
%\section{IMAGER principles}

\subsection{Overview of the data reduction and analysis}
\label{sub:overview}

Once the data has been acquired by an interferometer such as the 
NOrthern Extended Millimter Array  
(\NOEMA{}) or \ALMA{}, two different approaches may
be used for its reduction and analysis:
\begin{itemize}
\item The first possibility is to clearly separate 1) the calibration, 2)
  the imaging and deconvolution and 3) the analysis.
\item The second possibility is to merge in a single step calibration and
  imaging. This possibility is known as self-calibration.
\end{itemize}
While \casa{} uses the second paradigm, \gildas{} mainly implements 
the first approach, as the program and the data format used 
for each step is different, but still allows convenient self-calibration. 
 The basic instrumental calibration of \NOEMA{} data is
done inside \clic{} on the \NOEMA{} raw data format and the outcome is a
\uv{} table, which contains only calibrated visibilities of the
astronomical source.  The imaging and deconvolution are done inside
\imager{} on the calibrated \uv{} table and deliver mainly an \lmv{}
spectral cube (2 axes of coordinates and 1 axis of velocity/frequency).
\imager{} includes \viewer{}, a superset of \greg{}, for the  visualization, and provides additional
image analysis functionalities,  which are not specific to an interferometric use 
(\eg\ they can be used with the \IRAM{} 30\,m spectral cubes as well).

The choice of clearly separating calibration and imaging+deconvolution was
taken at start of the Plateau de Bure Interferometer, 
when the limiting number of antennas prevented 
the use of self-calibration. 
While many points of the calibration algorithms
inside \clic{} are specific to \NOEMA{} data (in particular its range of
Signal-to-Noise ratio), the algorithms of imaging+deconvolution can be used
in many different contexts and the visualization and analysis of spectra
cubes is mainly independent of the instrument that delivered the data. 
This last point implies that users can import data from \ALMA{} (mainly through FITS format)
in \imager{} for imaging and deconvolution, and in \viewer{} for
visualization. But the reverse is also true. While calibration
of \NOEMA{} data should be done inside \clic{}, imaging+deconvolution and
visualization+analysis can be done in other softwares (\eg\ \miriad{},
\aips{}, \casa{} for the imaging and deconvolution
and \karma{} for the visualization and analysis).

With the improvement of \NOEMA{} (increase of the number of antennas 
and better receiver sensitivities) and with the advent of a new 
generation of interferometer (\ALMA{}), the additional step of 
self-calibration may improve the consistency of the final results by 
imposing additional consistent constraints on the calibration. This 
self-calibration step is further presented in Section\ref{sec:self}.

\subsection{The structure of the \imager{} program}
\label{sub:structure}

The \imager{} program supports
\begin{itemize}
\item The manipulation (\eg\ resampling), visualization and flagging of
  \uv{} tables;
\item The imaging of \uv{} tables in dirty maps and beams;
\item The deconvolution of dirty maps;
\item The inclusion of short-spacings; 
\item The visualization and analysis of spectral cubes;
\item The self-calibration.
\item A simulator for observations with \ALMA{}  or \NOEMA{} (and other arrays).
\end{itemize}

It consists in a collection of commands, either dedicated to image display
  (the \lang{DISPLAY} language), UV handling and imaging with deconvolution
  (the \lang{CLEAN} language), implementing basic functionalities (the
  \lang{SIC}, \lang{GREG}, or \lang{CALIBRATE}  families of languages),
  advanced methods and image analysis tools (\lang{ADVANCED} language)
  or even complex suites of operations (the \lang{BUNDLES} language).
  A complete pipeline is available through the \lang{IMAGER} language.
    
A few additional widgets are grouped in the \imager{} main menu to
provide more integrated interfaces to the above possibilities, or
more elaborate control. While these widgets may still evolve
to offer more flexibility, the commands
are in general very stable, though minor syntax adjustments may occur
(usually to implement a more convenient syntax, but keeping as much as
possible backward compatibility as a major constraint). 

\subsection{The  user interface: SIC and GreG}
\subsubsection{User input: The SIC interpretor}
\label{sub:sic}

\imager{}, like all \gildas{} programs, uses the SIC command language
as a user interface. SIC is fully described at 
\begin{latexonly}
\href{https://www.iram.fr/IRAMFR/GILDAS}{https://www.iram.fr/IRAMFR/GILDAS/}.
\end{latexonly}
\begin{rawhtml}
  <A HREF="https://www.iram.fr/IRAMFR/GILDAS"> https://www.iram.fr/IRAMFR/GILDAS. </A>
\end{rawhtml}

In short, SIC is a pwerful command line interpretor, with structured programming  (\qom{FOR} \qom{NEXT} loops,
\qom{IF ELSE ELSEIF ENDIF}, \qom{BEGIN} \qom{END} constructs for procedures and ancillary data files,
\qom{@} control to execute procedures, etc\ldots..) and a syntax ressembling that of FORTRAN
for the command lines. \qom{HELP} provides help on each command and language (a logically connected
set of commands).   

Command syntax is the following
\begin{verbatim}
[LANG\]COMM [ARG1 [ARG2 [...]]] [/OPT1 [ARG11 [...]] [/OPTn [...]]
\end{verbatim}
Where \lang{LANG} is the language name, \qom{COMM} the command name, \qom{/OPT1} and \qom{/OPTn} are option names, 
\texttt{ARG}s are the arguments of command and options, and brackets indicate optional fields. 
Arguments are positional (order matters), while options can be given in any order. In the \qom{HELP} description,
arguments that appear in upper case should in general be taken from a list of specified keywords. Arguments
appearing in lower case can take any value.

SIC heavily uses variables. Variables are strongly typed named entities which must be declared before use through the 
\qom{DEFINE} command. They can be created by the user, or pre-defined by  the program.  Array variables are allowed, up 
to 7 dimensions. Variables may be of type Character to hold strings. Mathematical (arithmetic and logical) operations 
on numeric variables are supported. Variables can also be Structures, an ensemble of variables. In particular, data 
files can be imported as Structures that contain all the header information as variables, and an N-dimensional array of 
values (\qomm{DEFINE}{IMAGE} command).  Contents of variable is modified by command \qom{LET}.

SIC also allows the creation of Widgets, that can be used to assign variables and launch actions (commands or scripts)
whose behaviour is controlled by these variables, providing a more intuitive GUI (Graphics User Interface).

This document assumes you are reasonably familiar with SIC, mastering the basic syntax, and understanding
the use of Variables. Please refer to the  
\begin{latexonly}
\href{https://www.iram.fr/IRAMFR/GILDAS/doc/html/sic-html/sic.html}{SIC documentation}
\end{latexonly}
\begin{rawhtml}
  <A HREF="https://www.iram.fr/IRAMFR/GILDAS/doc/html/sic-html/sic.html"> SIC documentation</A>
\end{rawhtml}
for more information. The SIC version used inside \imager{} has minor differences compared to that described by
the manuals, but its internal HELP is up-to-date.

\subsubsection{Output user interface: the GREG package}
\label{sub:greg}

Besides running messages, generated by \imager{} through the \qom{MESSAGE} facility of SIC, \imager{} has advanced
visualization tools. 
Graphics display in \imager{} are done using the \greg{} package. It allows multi-window displays, with cursor
interaction, and publication quality plots in various format (PostScript, PDF, PNG). Please refer to the 
\begin{latexonly}
 \href{https://www.iram.fr/IRAMFR/GILDAS/doc/html/greg-html/greg.html}{GreG documentation}
\end{latexonly}
\begin{rawhtml}
  <A HREF="https://www.iram.fr/IRAMFR/GILDAS/doc/html/greg-html/greg.html"> GreG documentation</A>
\end{rawhtml}
for more information.

When only the visualisation is needed, the user can select the \viewer{} program instead of the full \imager{} package.

\subsection{Fundamental control variable: \sicvar{BEAM\_STEP} -- {\color{red}{Read carefully !}}}

In aperture synthesis, the angular resolution scales with the frequency: in fact the whole Fourier plane scales with 
this frequency, so that the angular scaled defined by the baselines varies across the frequency coverage. Accordingly, 
when using  sufficiently wide bandwidths (and/or imaging sufficiently large areas), it becomes important to account 
correctly for this effect. As a result, beams (whether primary or synthesized) 
can be 4-D arrays, as they may depend on Frequency and Field (for Mosaics).

The deconvolution implementation of \imager{} in the 
\com{UV\_MAP} and \com{CLEAN} commands is designed such that ensembles of contiguous channels 
(\textit{``chunks''}) are treated at once and share the same synthesized beam. 
Deconvolution with \com{CLEAN} then proceeds by using the synthesized 
beam with the appropriate frequency for each channel. The user can 
control the ``chunk'' size, and hence the precision of the process 
given the desired field of view.

The user control is done through \sicvar{BEAM\_STEP}.
\begin{itemize}
  \item \sicvar{BEAM\_STEP} = 0 instructs \imager{} to try to produce a single synthesized beam for all channels. This is 
appropriate for narrow bands and spectral channels with very similar noise (ideally identical). Unavoidably, the
synthesized beam is only an approximation, exact only at the reference frequency.
  \item \sicvar{BEAM\_STEP} = N instructs \imager{} to ensure that N consecutive channels share the 
same synthesized beam. This can be used when channels have the same weights (same effective UV coverage) but
the overall bandwidth is too large: the number of image channels being M, the number of different beams
will be M/N. 
  \item \sicvar{BEAM\_STEP} = -1 instructs \imager{} to derive the number N of consecutive channels
  that can share the same beam to within a certain precision in the imaged field of view. This precision
  is given by \sicvar{MAP\_PRECIS}. The derived N will depend of this number, on the relative bandwidth and on
  the size of the synthesized field (hence \sicvar{MAP\_FIELD} or \sicvar{MAP\_CELL} $\times$ \sicvar{MAP\_SIZE}).
\end{itemize}
The above works in an optimal way in all cases where all channels have the same weight distribution. 
This is in general true for \NOEMA{} data coming from \clic{}, but unfortunately,
this may not always be the case. Common cases where this does not occur are:
\begin{itemize}
 \item Band edges, where the noise may increase
 \item Data resulting from concatenation in frequency of different spectral windows
 \item Cases where the system temperature is  highly frequency dependent (and carried along in the weights) 
\end{itemize}
\com{UV\_CHECK}~\qom{BEAMS} will verify the channel ranges that share the same weights
and return in \sicvar{BEAM\_RANGES} the range boundaries. \com{UV\_MAP} uses this
information in an ``intelligent'' way: band edges will be dropped if needed, and small regions
(up to \sicvar{BEAM\_GAP}, default 3) with different weights are ignored in this process.
In most cases, this behaviour allows a common beam for all channels, or an automatic guess of N if
\sicvar{BEAM\_STEP} = -1.

If this fails, \qom{UV\_MAP} will complain and stop with an error. The user must then adjust 
\sicvar{BEAM\_STEP} to either
\begin{itemize}
  \item \sicvar{BEAM\_STEP} = 1, one beam per channel, mimicking to some extent the behaviour of CASA,
  at the expense of speed and memory/disk use (twice more needed) 
  \item \sicvar{BEAM\_STEP} = -2,  forcing \imager{} to produce a single synthesized beam,
but at the expense of some sensitivity loss in channels whose weight do not match the reference one, 
the so-called \textit{weight channel} \sicvar{WCOL}. 
\end{itemize}

\label{sec:sub:beamstep}
\subsubsection{CASA versus \imager{}}

The CASA and \imager{} approaches to this spatial resolution problem differ. CASA systematically produces one synthesized beam per
spectral channel, leading to a frequency variable spatial resolution.   In this approach, CASA can handle different
weighting for individual channels, optimizing signal to noise, but the interpretation of the data becomes more
difficult since there is no unique angular resolution after deconvolution.

\imager{} can mimick the CASA behaviour by using \sicvar{BEAM\_STEP} = 1, but \imager{} uses a common clean beam in 
all cases. This is achieved through the use of the so-called \textit{\bf JvM} factor, which approximately scales 
the undeconvolved residuals to match the flux scale of the Clean components. This behaviour is only obtained through 
the \com{UV\_RESTORE} command.

\sicvar{BEAM\_STEP} is a key control parameter for \imager{}.  The \imager{} pipelines (see Section\ref{sec:all})
use sensible default for each telescope, but users should be aware of this possible difference when comparing
with CASA results.



\subsection{Imaging/deconvolution: a brief sequence of commands}
For the user, \imager{} reduces the number of actions to the strict minimum. 
The imaging sequence is always the same: 
\begin{verbatim}
    1- Reading data
       READ UV MyData.uvt /RANGE Min Max Type
       ! here, optionally use UV_TIME, UV_COMPRESS, UV_BASELINE to average data
       ! or UV_FILTER, UV_CONT to filter lines or remove continuum
    2- Imaging
       UV_MAP         
    3- Deconvolving
       CLEAN           
       ! here, optionally use UV_RESTORE
    4- Looking at the result
       VIEW CLEAN      ! or SHOW CLEAN     
    5- Writing the result 
       WRITE * MyData  
\end{verbatim}

\begin{itemize}
\item \textbf{Step 1:} Reading  the  specified  internal  buffer (here 
UV) from the input file (\file{.uvt} file type), loading only the channels falling in 
the range defined by the variables Min and Max, of Type \qom{CHANNEL}, 
\qom{VELOCITY} or \qom{FREQUENCY}. \imager{} recognizes whether the UV 
table is for a single field or a mosaic. The only difference between 
the single field and mosaic cases is that \imager{} yields a Sky 
brightness image for Mosaics, while the computed sky brightness of a 
single field is not automatically corrected for the primary beam 
attenuation. Imaging for multiple fields will be presented in Section~\ref{sec:mosaic}.
Single Dish data can also be loaded  in the following way : 
\comm{READ}{SINGLE}~\file{File}.
\item \textbf{Step 2:} Computing a  dirty  map  and  beam from a UV data. 
\com{UV\_MAP} processes  single fields as well as Mosaics.
\item \textbf{Step 3:} Deconvolving the \sicvar{DIRTY} image map (a Single-field or Mosaic) 
using the dirty \sicvar{BEAM} with the current \sicvar{METHOD}. The default 
for the SIC variable \sicvar{METHOD} is \com{HOGBOM},  the other 
supported methods being \com{CLARK}, \com{MRC}, \com{MULTI} and 
\com{SDI}. See \qomm{CLEAN}{?} for the other SIC variables 
controlling the deconvolution process. The outputs are the \sicvar{CLEAN} and 
\sicvar{RESIDUAL} images, and the Clean Component Table \sicvar{CCT}, all being stored 
in dedicated SIC variables.
\item \textbf{Step 4:} Plotting the result in the specified internal 
buffer (\sicvar{CLEAN}). Optionaly, the user can restrict the plot to a 
subset of channels through the optional arguments First and Last. 
\comm{SHOW}{CLEAN} can also be used instead, and produces a different 
type of plot.
\item \textbf{Step 5:}  Writing all modified image-like buffers (not 
the UV  tables) under the common file name ''\file{MyData}''. In the case of the 
present example, the following files are produced: \file{MyData.lmv, 
MyData.lmv-clean, MyData.cct, MyData.beam}, which correspond to the 
buffers: \sicvar{DIRTY}, \sicvar{CLEAN}, \sicvar{CCT}, and \sicvar{BEAM}, respectively. 
\qomm{WRITE}{UV}~\file{MyData} would only write the internal buffer (\sicvar{UV}) in the 
file \file{MyData.uvt} (the default extension corresponds to the specified 
buffer data type).
\end{itemize}    

\subsection{Getting Help: the HELP command and the ? token}
\label{sub:help}

As with any \sic{} based program, simple call to \com{HELP} will 
display the various languages (e.g., \lang{SIC}, \lang{GREG}, 
\lang{CALIBRATE}, \lang{CLEAN}) accessible to the help documentation 
and list some commands with available documentation. Note that 
\qom{CLEAN} is a command and \lang{CLEAN} a language (i.e., a family of 
commands). The language of a given command is recalled in the help of 
each command. 

Example: the command \qom{APPLY} belongs to the language \lang{CALIBRATE}, 
it has one argument which can be \qom{AMPLI} or \qom{PHASE} exclusively, 
one optional argument \qom{gain}, and one option \qom{/FLAG}.
\begin{verbatim} 
IMAGER> help apply
        [CALIBRATE\]APPLY [AMPLI|PHASE [gain]] [/FLAG]
\end{verbatim} 
In the provided description, arguments within \texttt{[]} are optional. Upper case 
arguments are fixed keywords that can (in general) be abbreviated. A $|$ character separates
the allowed keyword values. Lower-case arguments are expected to be numbers, text, filenames, SIC variables or
expressions that can be evaluated.

A brief description of the imager program can be obtained through:
\begin{verbatim} 
IMAGER> help imager
USER\IMAGER = "@ welcome.ima"
 
      IMAGER is a interferometric imaging package, tailored for usage
  simplicity and efficiency for multi-spectral data sets.
 
     The basic concept of IMAGER is the use of a simple
         READ data  - ACTION(s) - [SHOW or VIEW] - WRITE
  sequence of commands, minimizing the data I/O as much as possible.
  Automatic guesses of appropriate default values for the ACTIONs
  parameters is implemented whenever possible.
 
Additional Help Available:
 Actions      BEAM_Handlin MAP_Handling UV_Handling  WebPage      
 HOW_TO       MAPPING      
\end{verbatim} 

In addition, finding documentation and help for the \imager\ commands 
can be done in three different ways:  
\begin{itemize}
\item a simple call to the \com{HELP} command provides a description of 
the command and its arguments and options
\item the command name followed by one or more questions marks will 
display some partial help on the command and its most useful 
parameters (''\qom{Command ?}''), its second level parameters for advanced 
users (''\qom{Command ??}''), all its parameters (''\qom{Command ???}''). 
%\item \com{INPUT Command} provides a list of default values for the most commonly used parameters.
\end{itemize}

Documentation on subtopics of a given command (e.g., Variables, 
Arguments, or Results) can be obtained though: \comm{HELP}{command subtopic}.
The list of available subtopics is found at the bottom of the documentation of each command: 
\begin{verbatim} 
IMAGER> help uv_map
[...] 
Additional Help Available:
 Mosaics      /FIELDS      /RANGE       /TRUNCATE    
 Variables:
              BEAM_STEP    MAP_CELL     MAP_CENTER   MAP_CONVOLUT MAP_FIELD
 MAP_POWER    MAP_PRECIS   MAP_ROBUST   MAP_ROUNDING MAP_SHIFT    MAP_SIZE
 MAP_TAPEREXP MAP_UVTAPER  MAP_UVCELL   MAP_VERSION  MOSAIC_BEAM
 Old_Names:
              convolution  map_angle    map_dec      map_ra       mcol
 uv_cell      uv_shift     uv_taper     taper_expo   wcol         weight_mode  
\end{verbatim} 
Subtopics that appear in UpperCase in the \texttt{Additional Help Available}
list are case insensitive. Subtopics that are in LowerCase of mixed case are
strictly case sensitive.

Example: the following command will list the control variables of the 
\com{UV\_MAP} function and describe the associated parameter(s): 
\begin{verbatim} 
IMAGER> help uv_map Variables
UV_MAP Variables:
                [CLEAN\]UV_MAP ?
        Will list all MAP_* variables controlling the UV_MAP parameters.
 
    The list of control variables is (by alphabetic order, with  the  corre-
    sponding old names used by Mapping on the right)
 
    New names       [   unit]       -- Description --    % Old Name
    BEAM_STEP       [       ]  Channels per dirty beam  % MAP_BEAM_STEP
    MAP_CELL        [ arcsec]  Image pixel size
    MAP_CENTER      [ string]  RA, Dec of map center, and Position Angle
    MAP_CONVOLUTION [       ]  Convolution function     % CONVOLUTION
    MAP_FIELD       [ arcsec]  Map field of view
    MAP_POWER       [       ]  Maximum exponent of 3 and 5 allowed in MAP_SIZE
    MAP_PRECIS      [       ]  Fraction of pixel tolerance on beam matching
    MAP_ROBUST      [       ]  Robustness factor        % UV_CELL[2]
    MAP_ROUNDING    [       ]  Precision of MAP_SIZE
    MAP_SIZE        [       ]  Number of pixels
    MAP_TAPEREXPO   [       ]  Taper exponent           % TAPER_EXPO
    MAP_UVCELL      [      m]  UV cell size             % UV_CELL[1]
    MAP_UVTAPER     [m,m,deg]  Gaussian taper           % UV_TAPER
    MAP_VERSION     [       ]  Code version (0 new, -1 old)
    MOSAIC_BEAM     [      %]  Mosaic truncation level  % MAP_TRUNCATE
  
    See HELP UV_MAP Old_names:  for deprecated variable names.
\end{verbatim} 

A more detailed description (type, size) of a given variable can be obtained 
through \texttt{help command variable}, as in this example:
\begin{verbatim} 
IMAGER> help uv_map map_uvtaper
UV_MAP MAP_UVTAPER
 
      MAP_UVTAPER[3]  Real
 
    Parameters of the tapering function (Gaussian if MAP_TAPEREXPO = 2): ma-
    jor axis at 1/e level [m], minor axis at 1/e level [m], and position an-
    gle [deg].
\end{verbatim} 
\sicvar{MAP\_UVTAPER} is an array that requires 3 values of type Real.
\vspace{0.5cm}

The default values of the useful parameters are checked through 
\qomm{Command}{?}\footnote{For users familiar with \mapping{}, the
question mark replaces the capabilities of the  \qomm{INPUT}{Command} although the output format may be slightly 
different.}
\begin{verbatim} 
 UV_MAP makes a dirty image and a dirty beam from the UV data
 
 Behaviour is controlled by a number of SIC Variables
 - BEAM_STEP and MAP_PRECIS control the dirty beam precision
 - MAP_CENTER controls shifting ang rotation
 - MAP_CELL[2], MAP_SIZE[2], MAP_FIELD[2] control the map sampling
 - MAP_UVTAPER[3], MAP_UVCELL and MAP_ROBUST
     control the beam shape and weighting scheme
 
---- Basic parameters                            Selected       Recommended
  Map Size (pixels)              MAP_SIZE        [ 0 0 ]   [ 0 0 ]
  Field of view (arcsec)         MAP_FIELD       [ 0 0 ]   [ 0 0 ]
  Pixel size (arcsec)            MAP_CELL        [ 0 0 ]   [ 0 0 ]
  Map center                     MAP_CENTER      [  ]      
  Robust weighting parameter     MAP_ROBUST      [ 0 ]
  UV cell size (meter)           MAP_UVCELL      [ 0 ]          [ 0 ]   
  UV Taper (m,m,°)               MAP_UVTAPER     [ 0 0 0 ] 
\end{verbatim} 

For commands that support the \qom{?} mechanism, \qomm{Command}{?}~\sicvar{ControlVariable} and, in most cases,
\com{Command}~\sicvar{ControlVariable}\qom{ ?} are shortcuts to \qomm{HELP}{Command}~\sicvar{ControlVariable}.

