%\section{Short and Zero spacings} 
\label{sec:short-spacings}

\subsection{In a nutshell}

\begin{verbatim}
  1  read uv gag_demo:demo-mosaic-imager.uvt
  2  read single gag_demo:demo-single-imager.tab
  3  uv_short
  4  uv_map; clean
  5  view clean
  6  write * MyDemo
\end{verbatim}
\begin{enumerate}\itemsep 0pt
\item Read your \uv{} data
\item read your single-dish data
\item Merge it with with the \uv{} data set,
\item Image and deconvolve as usual
\item check the result. You can use \texttt{CLEAN} as argument to \com{SHOW}
or \com{VIEW}: \imager{} will
automatically fall back to \texttt{SKY} if needed (and vice versa).
\item Save it. 
\end{enumerate}
You are done. This works for mosaics as well as single fields.

But, again, if the result are crazy, do not blame the software. 
Rather read carefully the information below: 
\com{UV\_SHORT} is simple to use, but can be tricky to use well !...

\subsection{Principle}
Let us note \textit{D} the diameter of the single-dish antenna (\textit{D\,=\,30}m for the
IRAM-30m telescope) used to produce the short-spacing information and $d$
the diameter of the interferometer antennas (\textit{d\,=\,15}m for \NOEMA{}). We
already mentioned that a multiplicative interferometer filters out all the
spatial frequencies smaller than $\sim d$ meters. When this information is
needed to get reliable results, the source should also be observed with a
single-dish antenna to produce the missing information.  The single-dish
antenna furnishes information about all spatial frequencies up to $\sim D$
meters (but this information is weighted by the single-dish beam shape,
\ie\ high frequencies are measured with a worse signal-to-noise ratio than
low frequencies). To recover all the information at spatial frequencies
smaller than \textit{d} meters, the diameter of single-dish antenna must be
larger or equal to the diameter of the interferometer antennae: $D \ge d$.

\subsection{Algorithms to merge single-dish and interferometer information}

The measurement equations of a single-dish and an interferometer are quite
different from each other. Indeed, the measurement equation of a
single-dish antenna is
\begin{latexonly}
 $$ I_\mathrm{meas}^\mathrm{sd} = B_\mathrm{sd} \star I_\mathrm{source} + N $$
\end{latexonly}
\begin{rawhtml}
 $$ I_\mathrm{meas}^\mathrm{sd} = B_\mathrm{sd} \star I_\mathrm{source} + N $$  
\end{rawhtml}
\ie\ the measured intensity ($I_{\mathrm{meas}}^{\mathrm{sd}}$) is the convolution of
the source intensity distribution ($I_{\mathrm{source}}$) by the single-dish
beam ($B_{\mathrm{sd}}$) plus some thermal noise, while the measurement equation
of an interferometer can be rewritten as
\begin{latexonly}
$$
  I_\mathrm{meas}^\mathrm{id} = B_\mathrm{dirty} \star \cbrace{B_\mathrm{primary}.I_\mathrm{source}} + N,
$$
\end{latexonly}
\begin{rawhtml}
$$  I_\mathrm{meas}^\mathrm{id} = B_\mathrm{dirty} \star \left(B_\mathrm{primary}.I_\mathrm{source}\right) + N, $$
\end{rawhtml}
\ie\ the measured intensity ($I_{\mathrm{meas}}^{\mathrm{id}}$) is the convolution
of the source intensity distribution times the primary beam
($B_{\mathrm{primary}}.I_{\mathrm{source}}$) by the dirty beam ($B_{\mathrm{dirty}}$) plus
some thermal noise. $B_{\mathrm{sd}}$ has very similar properties than
$B_{\mathrm{primary}}$ and very different properties than $B_{\mathrm{dirty}}$.  In
radioastronomy, $B_{\mathrm{sd}}$ and $B_{\mathrm{primary}}$ both have (approximately)
Gaussian shapes. Moreover, the fact that we will use the single-dish information to
produce the short-spacing information filtered out by the interferometer
implies that $B_{\mathrm{sd}}$ and $B_{\mathrm{primary}}$ have similar full width at
half maximum. Now, $B_{\mathrm{dirty}}$ is quite far from a Gaussian shape with
the current generation of interferometer (in particular, it has large
sidelobes) and the primary side lobe of $B_{\mathrm{dirty}}$ has a full width at
half maximum close to the interferometer resolution, \ie\ much smaller than
the FWHM of $B_{\mathrm{sd}}$.

Merging both kinds of information obtained from such different measurement
equations thus asks for a dedicated processing. There are mainly two
families of short-spacing processing: the \textbf{hybridization} and the
\textbf{pseudo-visibility} techniques.

\subsubsection{Hybridization technique (``feathering'')}

In this family, most of the processing is done on the interferometric data
alone. Indeed, the interferometric data is deconvolved and corrected for
the primary beam contribution to obtain
\begin{latexonly}
$$
  I_\mathrm{sky}^\mathrm{id} = B_\mathrm{clean} \star I_\mathrm{source} + N',
  \label{eq:clean}
$$
\end{latexonly}
\begin{rawhtml}
$$
  I_\mathrm{sky}^\mathrm{id} = B_\mathrm{clean} \star I_\mathrm{source} + N',
$$  
\end{rawhtml}
where $B_{\mathrm{clean}}$ is a Gaussian of FWHM equal to the interferometer
resolution and $N'$ is some thermal noise corrected for the primary beam
contribution.  Two main facts are hidden in this formulation: 1) the
field-of-view of the observation is obviously limited to the observed
portion of the sky and 2) more importantly, the lack of short-spacings has
not yet been overcome and a better formulation would be
\begin{latexonly}
$$
  I_\mathrm{sky}^\mathrm{id} = \mbox{Highpass-filter}\cbrace{B_\mathrm{clean} \star I_\mathrm{source}} + N'.
$$
\end{latexonly}
\begin{rawhtml}
$$
  I_\mathrm{sky}^\mathrm{id} = \mbox{Highpass-filter} \left(B_\mathrm{clean} \star I_\mathrm{source}\right) + N'.
$$
\end{rawhtml}
We shall keep the simpler formulation as notation, but bear in mind the
hidden filtering in the discussions. The hybridization method
consists in combining two images ($I_{\mathrm{meas}}^{\mathrm{sd}}$ and
$I_{\mathrm{clean}}^{\mathrm{id}}$) in the \uv{} plane.

\begin{enumerate}
\item Both images are first spatially regridded on the same fine grid.
\item The FFT of those two images are computed, and linearly combined by
  selecting the low spatial frequencies from FFT($I_{\mathrm{meas}}^{\mathrm{sd}}$)
  and the high spatial frequencies from FFT($I_{\mathrm{sky}}^{\mathrm{id}}$).
  \begin{latexonly})
  $$ \mathrm{FFT}(uv) = f(uv) \mathrm{FFT}(I_{\mathrm{meas}}^{\mathrm{sd}}) + (1-f(uv)) \mathrm{FFT}(I_{\mathrm{sky}}^{\mathrm{id}} $$
  \end{latexonly}
  \begin{rawhtml}
  $$ \mathrm{FFT}(uv) = f(uv) \mathrm{FFT}(I_{\mathrm{meas}}^{\mathrm{sd}}) + (1-f(uv)) \mathrm{FFT}(I_{\mathrm{sky}}^{\mathrm{id}}) $$
  \end{rawhtml}
  The transition $f(uv)$between low and high spatial frequency is selected
  to use the best regions of the \textit{uv} plane in both images.
\item The result is FFTed back to the image plane to produce a final,unique
  image, which takes into account both single-dish and interferometric
  information.
\end{enumerate}
The method has the following free parameters: the transition radius and the
detailed shape of that transition.  To avoid discontinuity, the transition
shape is chosen to be reasonably smooth.  When the low resolution image
is provided by a single-dish, the best signal-to-noise combination
is obtained using a function $f(uv)$ that is Fourier transform
of the single-dish beam. However, that is only optimal if the noise
in this image is small enough and no other instrumental effect
(such as pointing errors or baseline ripples) affect the data. So,
$f(uv)$ can also be chosen arbitrarily.  The spatial frequency of 
transition is selected close to the smallest spatial frequency 
reliably measured by the interferometer (\eg\ about 18~m for NOEMA), 
and/or the largest spatial frequency measured by the low 
resolution image (\eg\ about 20~m for data taken with the IRAM 30-m 
telescope). For combining ACA and ALMA data, this would be 15~m,
and of ACA and 12-m single dish, about 9~m.

\subsubsection{Pseudo-visibility technique}

\paragraph{General description}

In this family, the single-dish information is heavily processed before
merging with the interferometric information. The basic idea is to produce
from the single-dish observations pseudo-visibilities similar to the ones
that would be produced by the interferometer if they were not filtered out.
\begin{enumerate}\itemsep 0pt
\item The Single-Dish measurements are re-gridded and then FFTed into the
  \uv{} plane.
\item The data are deconvolved of the single-dish beam ($B_{\mathrm{sd}}$)
  convolution by division by its Fourier Transform (truncated to the
  antenna diameter).
\item The data are FFTed back to the image plane and multiplied by the
  interferometer primary beam, $B_{\mathrm{primary}}$.
\item The result is FFTed again in the \uv{} plane where the visibilities
  are sampled on a regular grid.
\item In the case of a mosaic, the two last operations are performed for
  each pointing center.
\end{enumerate}
Using the properties of the Fourier transform, we can rewrite the
measurement equation of an interferometer as
\begin{latexonly}
  $$
  V(u,v) = \cbrace{\mbox{FT}(B_\mathrm{primary}) \star \mbox{FT}(I_\mathrm{source})}(u,v)+N.
  $$
\end{latexonly}
\begin{rawhtml}
  $$
  V(u,v) = \left(\mbox{FT}(B_\mathrm{primary}) \star \mbox{FT}(I_\mathrm{source})\right)(u,v)+N.
  $$
\end{rawhtml}
This equation means that the visibility measured by an interferometer at
the spatial frequency $(u,v)$ is the convolution of the Fourier transform
of the source intensity distribution by the Fourier transform of the
primary beam. Hence, to get pseudo-visibilities truly consistent with
interferometric visibilities, we must be able to reliably compute the
convolution by the Fourier transform of the primary beam. This implies that
we can compute pseudo-visibilities only for spatial frequencies lower than
\textit{D-d}. The use of the IRAM-30m to produce the short-spacing information of
the \NOEMA{} is thus ideal as it enables to recover pseudo-visibilities up
to 15~m (=30\,\mbox{m}-15\,\mbox{m}). Once the pseudo-visibilities have
been computed, they are merged with the interferometric visibilities and
standard imaging and deconvolution are then applied to the merged data set.


\paragraph{Single-dish vs interferometer weight}

In all cases involving short spacings, the relative weight of the single
dish data to interferometer data is critical. Within the restrictions
imposed by the noise level, this relative weight is a free parameter. It is
all the more important that the Fourier transform of the \uv{} plane
density of weights is the dirty beam, a key parameter of the deconvolution.
The general goal is to have a dirty beam as close as possible to a
Gaussian. As the Fourier transform of a Gaussian is a Gaussian, we search
for obtaining a \uv{} plane density of weights as close as possible to a
Gaussian. In general, the short spacing frequencies are small compared to
the largest spatial frequency measured by an interferometer. This implies
we can use the linear approximation of a Gaussian, \ie\ a constant, in the
range of frequencies used for the short spacing processing. We thus end up
with the need to match (as far as possible) the Single-Dish and
interferometric densities of weights in the \uv{} plane. In practice, we
compute the density of weights from the single-dish in a \uv{} circle of
radius 1.25\,\textit{d} and we match it to the averaged density of weights from the
interferometer in a \uv{} ring between 1.25 and 2.5\,\textit{d}. Experience shows
that this gives the right order of magnitude for the relative weight and
that a large range of relative weight around this value gives very similar
final results.

When processing IRAM-30m data to combine to NOEMA data, this criterion
implies a large down-weighting of the IRAM-30m data which may make think
that too much observing time was used at the IRAM-30m data. However, just
using the above criterion to determine the observing time needed at the
IRAM-30m would in general lead to very low signal-to-noise ratio of the
single-dish map. In such a case, it is very difficult to detect problems
which would translate in strong artifacts in the IRAM-30m + NOEMA
combination. We recommend to ask for enough IRAM-30m time to get a
\emph{median} signal-to-noise ratio of 5 on the single-dish map. This ratio
should be achieved for all velocity channels of interest (which may include
the line wings).

\subsubsection{Comparison}

The simplicity of the hybridization technique is its main advantage. It is
simple to understand and simple to implement. However, this method works
badly in practice because it is truly difficult to obtain a
reliable deconvolution of interferometric data alone when short-spacing
information is important. An interferometer is a spatial pass-band
filter, filtering in particular the zero spacing. This implies that the total flux in the
dirty image is zero (\ie\ as much negative as positive flux in the dirty
image) but that the dirty beam integral is also zero (\ie\ as much negative
as positive sidelobes). Adding the short-spacing information (and in
particular the zero spacing) through the pseudo-visibility method, we
enforces the positivity of the dirty image total flux and of the dirty beam
integral. It is well-known that trying to deconvolve a mosaic built only
with interferometric data is quite difficult. It almost always requires the
definition of support where the \clean{} algorithms can search for clean
components with the clear risk to bias the final result. In contrast,
adding the short-spacing information through pseudo-visibilities enables an
almost straightforward \clean{} deconvolution \emph{without} the need of
any support.

For the sake of illustration, let us assume an intensity distribution 
made of a large scale structure (\eg{} a smoothly varying intensity) 
superimposed with a small scale distribution both in emission and 
absorption. An interferometer will filter out the smooth 
distribution. If there is no additional zero spacing 
information, the smooth distribution is completely lost with 
the important consequence that the final deconvoled image will have 
positive (emission) and negative (absorption) structures. Trying to 
reproduce both negative and positive structures is one of the most 
difficult task for deconvolution algorithms. In addition, the 
presence of large negative structures create instabilities in the 
algorithms of the \clean{} family (because it is difficult to 
distinguish between negative absorption structures and negative 
sidelobes of emission structures). Only the definition of support 
around positive emission peaks may succeed to stabilize the \clean{} 
algorithms with the drawback of biasing the result.

Both kind of algorithms  are implemented in \imager{}, under commands 
\com{FEATHER} for the hybridization and \com{UV\_SHORT} for the 
pseudo-visibilities. However, we strongly recommend to use the 
pseudo-visibility algorithm. 
\cite{pety01b,pety01a,pety01c} showed through
simulations that 1) the pseudo-visibility algorithm implemented in
\gildas{} enable extremely reliable results (fidelities of a few thousands)
on ideal observations and 2) the accuracy of the wide-field imaging is
limited by pointing errors, amplitude calibration errors and atmospheric
phase noise (and not by the used algorithms), even for ALMA.

\subsection{Hybridization technique and ALMA}
A special case where Hybridization can be extremely useful is that
of ALMA observations involving the main 12-m array, the ACA and single-dish
data with the 12-m antennas. These can be combined in several ways.

For example, the deconvolved mosaic made (using the pseudo-visibility method)
from ACA and the single-dish data can be used as a single-dish image,
providing short spacings 
that complete (either as pseudo-visibilities, or by feathering) the 12-m data.

In most circumstances, an optimal
result is obtained by hybridizing Cleaned images produced
from the mosaics obtained by combining (using the pseudo-visibility method)
ACA and Single-dish data
as short spacings, with another mosaic obtained with the 12-m
data and the Single-dish data (as zero spacing in this case) together.

The deconvolution of each mosaic is stabilized by the addition
of the 12-m single-dish zero or short spacings, and these good
mosaics are then merged in an optimal way by using the best
region of the \uv{} plane that they sample, typically using
a transition radius of 15 to 18~m.

See command \com{FEATHER} for details.

\subsection{The Zero spacing: an important subset}

An important subset of the pseudo-visibility method is the production of
only the zero spacing. Indeed, the zero spacing is just the total flux of
the observed field-of-view. Hence, if the observed field-of-view is small
enough to fit in the single-dish beam (this is in particular always the
case if $D = d$), a single spectrum observed with the single-dish telescope
in the direction of the interferometer phase center may be used as zero
spacing, only a scaling from Kelvin to Jansky is needed. This is the poor
man solution as only part of the short spacing information is recovered
by this technique.

\subsection{Short Spacings in practice: command \texttt{UV\_SHORT}}

Our algorithm to produce the short-spacing information is coded in the
\com{UV\_SHORT} command.  \com{UV\_SHORT} will \textbf{add} the 
short spacing information to the current \uv{} table (read by
command \comm{READ}{UV} and optionally transformed by further 
\texttt{\color{magenta} UV\_...} processing commands). 

\com{UV\_SHORT} has a substantial number (17)
of control variables, but with experience, they have been reduced
to 5 significant ones, among which only 3 really matter in most cases
but often can be used with their default values:
\begin{description}\itemsep 0pt
\item{\sicvar{SHORT\_SD\_FACTOR}} The single-dish brightness unit to 
flux conversion factor. If set to zero, \com{UV\_SHORT} will attempt
to derive it from the information available in the single-dish data   
\item{\sicvar{SHORT\_UV\_TRUNC}} The longest baseline retained in
the pseudo-visibilities. It defaults to the maximum theoretically possible,
the single-dish diameter minus the interferometer diameter. Smaller
values are allowed, and even recommended if the pointing quality
of the single-dish data is moderate.
\item{\sicvar{SHORT\_SD\_WEIGHT}} The relative weight scaling factor
between the pseudo-visibilities and the interferometer visibilities.
\end{description}
The relative weight of these visibilities is derived by \com{UV\_SHORT}
in order to optimize the shape of the overall synthesized beam. 
\sicvar{SHORT\_SD\_WEIGHT} is a scale factor to this optimum weight,
which may need to differ from 1 in case of poor \uv{} coverage in
the interferometer data or 
noisy single-dish data (it should be lower than 1 in this case).




\qom{UV\_SHORT ?} will list these 3 major ones, and \qom{UV\_SHORT ??} the
2 remaining main ones:
\begin{description}\itemsep 0pt
\item{\sicvar{SHORT\_TOLE}} The position tolerance in the single-dish map
\item{\sicvar{SHORT\_MIN\_WEIGHT}}  The minimum (relative) weight for a spectrum
in the single-dish map to be included.    
\end{description}
as well as four optional ones needed only if the original single-dish
and \uv{} data lacks the proper information (antenna diameter and beam sizes)

The \com{UV\_SHORT} command starts from data in a the format produced by the \class{} command
\texttt{TABLE}
command, and read in \imager{} through command \comm{READ}{SINGLE}. 
Basically, this is a GDF table containing one line per spectrum,
the columns representing the lambda offset, beta offset, weight, and the
spectrum intensities. \footnote{This format is subject to change: Please, refer to
the \com{TABLE} documentation for up-to-date information}. This data
must match spectrally the velocity sampling of the interferometric
data. This can be obtained using the \qom{/RESAMPLING} option of
command \qom{TABLE} in \class{}.

The \comm{READ}{SINGLE} and \com{UV\_SHORT} commands also support
a 3-D data cube (as produced by e.g. command \qom{XY\_MAP} in \class{})
as input instead of a \class{} table. Again, the
velocity axis must match that of the interferometric data.

\com{UV\_SHORT} will automatically produce the Zero spacing from
the single-dish data when the data does not allow other short spacings
to be evaluated. The temporary image produced by \com{UV\_SHORT}
when starting from a \class{} Table is stored in the \qom{SHORT}
buffer, and can be written by command \com{WRITE}. This image can
also be computed separately by command \com{XY\_SHORT}.

Finally, as \com{UV\_SHORT} \textbf{adds} the 
short spacing information, \comm{UV\_SHORT}{/REMOVE} allows to remove
it (there is no direct ``replace'' possibility because the $uv$ sampling
may change).

\subsection{Practical considerations}

\subsubsection{When are short-spacing information needed?}

\begin{itemize}
\item If the source size is smaller than 1/3 the primary beam size,
  short-spacing information is superfluous.
\item If the source size is between 1/3 and 1/2 the primary beam size of
  \NOEMA{} antennas, a single spectra obtained at the IRAM-30m telescope in
  the direction of the source can be used to produce the zero spacing
  information with the \com{UV\_SHORT} command. Indeed, the IRAM-30m diameter
  being twice the diameter of the \NOEMA{} antenna, all the flux of the
  source will be measured by a single IRAM-30m spectrum only if the size of
  the source is smaller than 1/2 the primary beam size of \NOEMA{} antennae.
\item If the source size is larger than 1/2 the primary beam size of
  \NOEMA{} antennas, short-spacing information under the form of an IRAM-30m
  map is almost always mandatory. The only exception could be wide-field
  imaging of a region made of unresolved or small (compared to the primary
  beam size) sources as it may happen when mapping close-by external
  galaxies for instance. However, adding short-spacing will anyway help the
  deconvolution. 
\item Short-spacing information is only useful if the brightness of the
  extended component is above the noise level. This requires a prior knowledge
  of the total flux in the imaged area to be determined. However, this
  information may be available from previous low-sensitivity single-dish
  observations. Checking this can avoid wasting a lot of telescope
  (and astronomer) time.
\end{itemize}
A generalization to \ALMA{} (12 m antenna) and ACA (7 m antennas)
is straightforward.


\subsubsection{How to optimize single-dish observations?}

One of the main difficulty of the short-spacing problematic is the need of
observations from a single-dish telescope at least as big as the
interferometer antennas\footnote{If there were no pointing errors,
a single-dish of the same size as the interferometer antennas
would be strictly sufficient.}. In this respect, the IRAM-30m and \NOEMA{} are very
complementary. Nevertheless, when observing with the single-dish telescope,
a few precautions are needed to avoid contaminating the interferometric
data with possible artifacts of single-dish data.
\begin{itemize}\itemsep 0pt
\item The field-of-view of the single-dish map must be twice the
  field-of-view covered by the mosaic. The only exception to this rule
  happens when the source intensity decreases to zero in a smaller
  field-of-view. Indeed, there is no point in observing an empty sky.
\item The observing strategy must enforce Nyquist sampling (or better) of
  the source at the resolution of the single-dish telescope.
\item A particular care should be taken of the pointing, tracking and
  amplitude calibration and baseline removal as those are critical issues
  in obtaining a high quality single-dish map to produce short-spacing
  information. For instance, data with too large tracking errors should be
  discarded.
\item Among ``baseline'' issues, the presence of continuum sources
  is to be treated with care. Continuum is difficult to measure with
  single-dish telescopes, and a (linear or polynomial) spectral baseline
  is often fitted to avoid atmospheric contamination. In such cases,
  the combination should be made with interferometer data where the 
  continuum has been removed, and added back later\ldots 
\item We advise to make many On-The-Fly coverages of the observed
  field-of-view to get homogeneous observing conditions. Scanning in
  perpendicular directions is needed to decrease stripping.  
\end{itemize}
Sometimes, single-dish telescope time is scarce and some of the above
criteria can not be fulfilled. In those cases, you can still try to use
your single-dish observations and our algorithm will try to make its best
to get a sensible result. However, any artifact in the combination may
directly come from wrong single-dish observations. In other words, do
\emph{not} blame the software unless you are sure of the quality 
of your single-dish (and interferometric) observations...

