\documentclass[11pt]{article}
\include{gildas-def}

\title{IRAM Memo 2012-?\\[3\bigskipamount]
  Preparing GILDAS for large datasets\\
  II - GILDAS Data Format Version 2}
\author{S. Guilloteau$^{1}$, S. Bardeau$^{2}$, J. Pety$^{2,3}$, V. Pietu$^{2}$\\[0.5cm]
  1. LAB (Bordeaux) \\
  2. IRAM (Grenoble) \\
  3. Observatoire de Paris}%
\date{24-jul-2019 \\
  version 1.4} %
\makeindex{}

\newcommand{\jp}[1]{\textbf{JP: #1?}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\maketitle{} %

% \begin{rawhtml}
%   Note: this is the on-line version of the ``GREG 2012 UV table format''
%   document for Gildas users and programmers.  A <A
%   HREF="../../pdf/greg-uvt-v2.pdf"> PDF version</A> is also available.
%
%   Related information is available in: <UL> <LI> <A
%   HREF="../greg-html/greg.html"> GREG:</A> Grenoble Graphics <LI> <A
%   HREF="../sic-html/sic.html"> SIC:</A> Simple Interpretor of Commands
%   </UL>
% \end{rawhtml}

\begin{abstract}
  With the advent of large data sets, the original GILDAS Data Format (for
  hypercubes and tables) became outdated due to its initial pure 32-bit
  implementation. The advent of new capabilities in interferometers, such
  as on-the-fly mosaics or polarization, also require modification in the
  way information are stored in UV tables.
  
  A new version of the GILDAS Data Format, called GDFV2, was designed and
  implemented to answer these challenges. This document details what was
  done and how it affects both programmers and users (in particular the
  backward compatibility aspects).
  
  The new data format is detailed. The public FORTRAN application program
  interface (API) which can be used in programs to access the data format
  is described.
  
  Related documents: \emph{GreG documentation}, \emph{Programming in
    GILDAS}, \emph{Task Programming Manual}.
\end{abstract}

\newpage
\tableofcontents{}

% -------------------------------------------------------------------

\newpage
\section{Introduction}

The size of the datasets produced by the (sub-)millimeter single-dish and
interferometers, including the IRAM 30m, Plateau de Bure interferometers
and ALMA, experience a tremendous increase (because of wide bandwidth
receivers, spectrometers with thousands of channels, multi-beams, and/or
new observing modes like the interferometric on-the-fly).

In the IRAM 30m context, the new FTS backend delivered during 2011 can
produce spectra of up to 37\,275 channels. Combined with On-The-Fly map
mode that can contain 100\,000's of spectra, the number of individual data
values can now easily reach 2 billions. The advent of NOEMA (a major
upgrade of PdBI capabilities) will also lead to large data sets.

While \clic{} and \class{} data formats have their own limitations (which
will be described in another document), imaging through gridding and
deconvolution uses the GILDAS data format (hypercubes and tables), which
were initially implemented for 32-bit machines. The original implementation
of this Gildas Data Format (hereafter GDFV1) stores the number of (4-bytes)
elements it contains in a signed integer value (\texttt{integer(4)}),
which, in IEEE arithmetics, is limited to $2^{31}-1 \simeq 2.10^9$
elements. Replacing this signed integer value (integer(4)) by a signed long
value (integer(8)) would have not helped as the dimensions of the data
array are also coded with a signed integer value. The only valid solution
was to also store these dimensions as signed long values with effect
everywhere in the GILDAS code (both kernel and packages). This way 1D
arrays can now contain as much as $2^{63}-1 \simeq 9.10^{18}$ elements.

In addition to these size limitations, new observing modes, such as
polarization or interferometric On-The-Fly, becomes available. Many new
requirements appeared to support these modes: Support of more than 4
dimensions, new header parameters, more flexible UV tables, ... Time had
come to revise the Gildas Data Format to a new version (hereafter GDFV2).

Section 2 summarizes the GDF design. Sections 3 and 4 describes how the GDF
format can store two main kinds of data, namely hypercubes and
tables. Changes for end-users and programmers are detailed respectively in
Sections 5 and 6. Backward compability and other miscellaneous issues are
dealt with in Section 7. For convenience, the data format and the public
application program interface of the GIO library is cut and pasted from the
code in Appendix A to D. Programmers are encouraged to look directly into
the source codes for up-to-date information, see
\texttt{http://www.iram.fr/IRAMFR/GILDAS/}.

\section{Design of the GILDAS Data Format}

The Gildas Data Format and its associated API are based on concept of
memory copy. In a program, access to a GDF file is done by reading into
memory the file header and part or all of the file data. These informations
are stored in a Fortran derived type named \texttt{gildas}.  This derived
type is described in Appendix \ref{app:gildas}.

It is essentially build on 5 items:
\begin{itemize}
\item The derived type \texttt{strings}, which contains character strings
  from the header;
\item The derived type \texttt{gildas\_header}, which contains essentially
  a direct copy of the file header (except for strings), but in the machine
  native representation;
\item The derived type \texttt{loca}, which contains memory related
  information;
\item A few ancillary information about the data, such as the \texttt{blc}
  and \texttt{trc} (Bottom Left corner and Top Right corner) which indicate
  which subset of the data is read, the data filename, etc...
\item And fortran pointers as placeholders for the data.  These are
  provided for \texttt{real(4)} arrays only, and only up to Rank 4. They
  may or may not be used by the programmer, who can elect to read the data
  in other Fortran arrays.
\end{itemize}

While numbers are always represented in the current hardware format in the
RAM memory (\eg, big-endian or little-endian), they may be stored in a
different way on the disk. The API (available is the so-called GIO library)
handles the data format conversion (in the data and in the header)
transparently. The data format, although described here, is \textbf{not}
intended to be used directly, but only through the API.

\section{GDF Images}

The Gildas Data Format can handle different type of data. In GDFV1, these
were Images (\ie, hypercubes), Tables and UV Tables, but the distinction
between Images and Tables was limited, as any Image could be opened as a
Table.  In GDFV2, we have improved this distinction by allowing different
kinds of Tables (simple tables, UV Tables, \class{} Tables, VO-like
Tables), which are identified through a specific code in the data
header. While provision for \class{} and VO-like Tables is taken, no
application code is yet available.

\subsection{Type description}

A GDF file is a binary file starting with a \emph{header} over \jp{at least
  two} \emph{header blocks}. The \emph{data} is found after the
\emph{header}, divided in its own \emph{data blocks}. We briefly describe
here the GDFV2 structure; the changes between GDFV1 and GDFV2 and the
backward support are detailed in the section~\ref{txt:backward}.

\subsubsection{Header description}
\label{txt:header}

The exact header definition can be found in appendix~\ref{app:gene}. In
short, it is divided in sections. The first one, named leading
pseudo-section, has a special status because it contains vital information
describing the file format. It (and thus all the GDF files, V1 or V2)
starts with 12 specific characters:
\begin{itemize}
\item the 6 firsts are always the word {\tt GILDAS} (uppercase),
\item the 7$^{th}$ is a unique character encoding the system type and the
  version of the GDF used (see Table~\ref{tbl:codes}),
\item the 5 last are a subsequent word used to recognize the kind of data
  in the file, namely {\tt IMAGE} (for standard Gildas images or
  hypercubes) or {\tt UVFIL} (for Gildas UV tables).
\end{itemize}
In GDFV2, this last character string is only a first order information
about the file kind. Distinction between Images, Tables, UV Tables, VO-like
Tables, is provided by an integer keywords (see
Table~\ref{tbl:kinds}). Then follows the data format, the various number of
blocks in the file, the version of the GDF currently in use, and the kind
of GDF file.

\begin{table}[tb]
  \begin{center}
    \caption{GDF encoding of the system and version. Most of the recent
      computers are little endian (IEEE) machines. The obsolete system VAX
      is not supported with GDFV2.}
    \vspace{1ex}
    \begin{tabular}{cccc}
      \hline
      Version  & IEEE & EEEI & VAX \\
      \hline
      1 & $-^1$ &  .  & $\_^2$ \\
      2 & $<$   & $>$ & N/A    \\
      \hline
    \end{tabular} \\ 
    $^{1}$ hyphen. \\
    $^{2}$ underscore.
    \label{tbl:codes}
  \end{center}
\end{table}
%
\begin{table}[tb]
  \begin{center}
    \caption{List of kinds available for GDFV2 files ({\tt
        h\%gil\%type\_gdf}). The values are integers that should not be
      used explicitely in the code. The developers must use the associated
      names.}
    \vspace{1ex}
    \begin{tabular}{l|l|l}
      \hline
      Name & Value & Comment \\
      \hline
      code\_gdf\_image & 0                 & Images or hypercubes \\
      code\_gdf\_uvold & 1                 & Old UV Data when the weights were inconsistent with the actual noise \\
      code\_gdf\_uvt   & 10                & UV Data in ``visibility'' order \\
      code\_gdf\_tuv   & -code\_gdf\_uvt   & UV Data in channel order (the transposed order) \\
      code\_gdf\_table & 20                & A simple Table, with no information \\
      code\_gdf\_vo    & -code\_gdf\_table & A simple Table, in the transposed way, as is the case for \\
                       &                   & Virtual Observatory Tables\\
      code\_gdf\_xyt   & 40                & A specific CLASS table (by symmetry with code\_gdf\_uvt) \\
      code\_gdf\_txy   & -code\_gdf\_xyt   & A specific CLASS table (by symmetry with code\_gdf\_uvt) \\
      % code\_gdf\_sheet & 50                & A spread sheet like table for the SIC Tabler...\\
      \hline
    \end{tabular}
    \label{tbl:kinds}
  \end{center}
\end{table}
% *** JP: I propose not to advertise the spreedsheet code.

The following sections are standard ones, named
\begin{description}
\item[dimension] The dimensions of the data in the file.
\item[blanking] The blanking and tolerance values.
\item[extrema] The value and position of the minimum and maximum value in
  the data.
\item[coordinate] The axes definition for each dimension, encoded as a 2D
  array of $N_{dim}$ (ref,val,inc) triplets.
\item[description] The physical unit of the data and the names of the
  axes. The first header block ends here.
\item[position] The source description. The second header block starts
  here.
\item[projection] The projection definition.
\item[spectroscopy] The description of the spectroscopic axis.
\item[resolution] The beam characteristics.
\item[noise] The data noise.
\item[astrometry] Proper motion parameters.
\item[telescope] Telescope(s) information.
\item[uv\_data] support for UV tables, e.g. description of extra-columns
  specific to this kind of tables.
\end{description}
Each section has a parameter describing its length. A zero value means that
this section is not present in the file and/or not filled in memory. For
the programmer or the end-user, any other value means it is enabled (then
all the parameters in the section are expected to be filled). In details,
some of the sections have a fixed length for any file (\eg, the length of
the \textbf{blanking} section is $2+2$ 4-bytes words) and some have a
length that depends on the number of dimensions in the file (\eg, the
length of the \textbf{coordinate} section is $2 + 2 \times (
N_{\mathrm{dim}} \times 3 )$ words).

\subsubsection{Data description}

The kind of data stored is described by the parameter {\tt form} in the
leading information of the header (see section \ref{txt:header}).  In most
radioastronomy applications, single precision floats are sufficient: The
code {\tt fmt\_r4} is used by default, but double precision floats ({\tt
  fmt\_r8}), standard ({\tt fmt\_i4}), or long integers ({\tt fmt\_i8}),
and single precision complex ({\tt fmt\_c4}) are also supported.

As for the header parameters, the values are stored in the native system of
the machine (little endian or big endian) at the file creation time. The
codes detailed in table~\ref{tbl:codes} ensure that the values can be
re-read and updated by any system.

The data is stored in the file in column-major order (Fortran-like).  Its
dimensions are described by the \textbf{dimension} section. While using
standard (32 bits) or long (64 bits) integers when in memory (depending on
the hardware capabilities), they are always stored in the file as long
integers. The limit is then $2^{63}-1 \simeq 9.10^{18}$ elements per
dimension.  The total number of elements in the data uses the same
limit. Note that cubes larger than 2 GB (512 mega-elements of single
precision floats) reach the RAM limit of 32 bits machines. This means that
the whole data block cannot fit in memory. Such big data files ($> 2$ GB)
are \textbf{not} supported on 32 bit machines.

The parameters \texttt{nhb} and \texttt{ndb} indicate the number of header
blocks, and of data blocks, respectively. Moreover, the parameter
\texttt{ntb} has been provisioned to handle trailing blocks after the
data. All blocks are 512 Bytes long in the current version, but the total
number of blocks \texttt{nhb+ndb+ntb} is rounded (upwards) to a multiple of
16, so that reading can proceed with physical blocks of 8192 Bytes for
better efficiency. A particular effort has also been made here to ensure
that there is no \texttt{integer(4)} limit on the number of Fortran records
(blocks) in the file, \ie, all computations and descriptive numbers related
to records use \texttt{integer(8)} values.

% zzzsb one word on the cut of data in blocks? DONE 

% zzzsg: here we can have a provision for using large blocks directly,
% through a different sub-version of GDFV2. We can change version_gdf for
% this when needed.

\subsection{Improvements between GDFV1 and GDFV2}

\subsubsection{Dimensions}

The maximum number of dimensions supported \emph{by the GIO library,
  including the GDF$\leftrightarrow$FITS converter} is now coded as a
unique parameter named {\tt GDF\_MAXDIMS}. It has been increased from 4 to
7\footnote{Up to the 2003 standard, Fortran used the same limit on its
  arrays. Fortran 2008 has raised it to 15.} between the two GDF
versions. The SIC arithmetic engine has been prepared during the summer
2011 to be able to deal with such data cubes.

% *** JP: I comment the following because it is difficult to follow and 
% *** JP: very hypothetical. Let's try first to find applications that 
% *** JP: requires more than 7 dimensions.

% Increasing this value is technically possible at the cost of hand-made
% adjustments of the GIO library, of the SIC arithmetic engine, maybe of
% the tasks and procedures, and probably on the global efficiency.

% The maximum number of dimensions supported \emph{by the GDFV2} is also
% set to {\tt GDF\_MAXDIMS}. Increasing it to 8 is technically possible but
% useless without the GIO and SIC libraries also upgraded. 9 dimensions and
% beyond can not be currently supported by the GDFV2, since the {\tt
% DESCRIPTION} section would overflow the first header block.

% In every case (GDFV2, GIO, SIC etc), increasing the maximum number
% dimensions beyond 7 implies the support by the compilers of the rank 15
% arrays from the Fortran 2008 standard. This, because we tend more and
% more to map the GDF or SIC data block on Fortran arrays.

In the GDFV1, the number of elements per dimension and the total number of
elements in the cube were stored in a standard integer, limited to
$2^{31}-1$. In the GDFV2, these elements are now stored as long integers
(under 64 bits machines) and are now limited to $2^{63}-1$.

\subsubsection{Transposition of data hypercubes}
\label{txt:transp}

The transposition engine has been moved to the GMATH library in order to
clearly isolate it from the GIO (for cube transpositions) and SIC (for
variable transpositions) libraries. It benefits several improvements.
\begin{itemize}
\item It now also supports up to 7 dimensions (but could support more,
  independently of GIO or SIC).
\item Timings and duration projections have been added for user feedback
  when transposing larges cubes.
\item More transposition codes are supported, including all the 3D
  permutations.
\end{itemize}

\subsubsection{New header parameters or facilities}
\label{txt:newparams}

A number of new objects have been added to the header in addition to all
the previous ones.
\begin{itemize}
\item For the programmer convenience, 3 pointers to the {\tt ref(:)}, {\tt
    val(:)} and {\tt inc(:)} values are now available, allowing easy
  writing of loops on the dimensions. They are real Fortran pointers to
  (and not duplicate of) the {\tt convert} array from the
  \textbf{coordinate} section\footnote{Variables \texttt{\%gil\%refI} with
    \texttt{I=1,4} (ibid. \texttt{\%gil\%valI} \texttt{\%gil\%incI}) which
    were marked as deprecated, have been removed. The new pointer arrays
    must be used instead.}.
\item In the \textbf{extrema} section, the min and max value coordinates in
  the cube are coded in memory as a {\tt MINLOC} and {\tt MAXLOC} arrays
  with 7 elements. In the disk file, they are coded as 2 \emph{flat}
  (scalar) address in the data.
\item The Doppler factor ({\tt dopp}) and the velocity type ({\tt vtyp})
  have been added to the \textbf{spectroscopy} section.
\end{itemize}

\subsection{Backward compatibility}
\label{txt:backward}

The GDFV2 is an improved and enlarged version of the GDFV1, but the GDFV1
is not directly compatible with the GDFV2. Changes for programmers and
end-users are described in the sections \ref{txt:endusers} and
\ref{txt:programmers} respectively.

Reading the GDFV2 file structure from disk with an old version (older than
may12) of \gildas{} is not possible. The file will not be recognized and
the following error message will be returned
\begin{verbatim}
E-RIS,  File xxx.gdf is neither a GILDAS data frame nor a SIMPLE FITS file
\end{verbatim}

Reading the GDFV1 file structure from disk (in any format, including VAX)
is still possible. It is mapped on the fly to a GDFV2 structure in
memory. On the other hand, writing GDFV1 is not possible anymore,
\jp{starting with the ??? version of \gildas{}}.

For convenience, some update operations will remain possible during a
transition phase. For example, extending a GDFV1 data file (via the
\texttt{DEFINE IMAGE A[n] gdfv1.gdf EXTEND} command), is still possible.
Updating the extrema is also possible.  Similarly, extending a pre-existing
GDFV1 UV Table in \clic{} is allowed. These specific facilities are only
provided to ease the transition between GDFV1 and GDFV2. They will
ultimately be removed from the \gildas{} package.  The user is thus
encouraged to convert its data files to the new format. The simple SIC
procedure \texttt{@ gdf\_convert} will do this.

% -------------------------------------------------------------------

\section{GDF UV Tables}

\subsection{Why a new implementation of the UV Tables in GILDAS?}

The UV Tables in the original GDFV1 data format, created in 1989, handled
visibilities and their required associated data parameters in a very
similar way as the UVFITS data format used to export data from the VLA at
that epoch.  A \textit{visibility} consisted in
\begin{itemize}
\item 7 associated data parameters (\textit{daps}), in fixed order:
  \texttt{u, v, w, date, time, first antenna, second antenna};
\item a so-called \textit{complex visibility element} for each channel,
  \ie, a real part, an imaginary part, and a weight.
\end{itemize}
The \textit{daps} precede the \textit{complex visibilities elements}.

The 7 (\textit{daps}) were real numbers. As Plateau de Bure is a 2-D array
with small field of view, \texttt{w} was most often replaced by the scan
number \texttt{scan} for debugging purpose.  The number of channels was
simply derived from the visibility size with
\begin{equation}
  N_{\mathrm{chan}} = (\mbox{VisiSize} - 7) /3.
\end{equation}
In the early 2000s, an extension was made to this constraining layout by
optionally adding two (real) numbers after the channel values.  This was
made necessary by the development of the ALMA simulator to study the impact
of the Atacama Compact Array (ACA).

With further developments coming on, such as polarization, on-the-fly
mosaicking, multi-source UVFITS files from CASA,...  this scheme was
becoming far too constraining. In addition, UV data sets could readily
exceed the 2 GByte size limit of the GDFV1 data format. We thus decided to
take advantage of the changes required to handle large data sets to
implement a more flexible UV data format.

% --------------------------------------------------------------------
% This section is a verbatim of (a part of) the discussion between SG
% and the other developers on 28-JAN-2015.

\subsection{UV tables subversions and Doppler factor}
A series of version of UV Tables is available, with increasing
precision for spectral coordinates and angular resolution.

\subsubsection{Version 2.0: Historical Frequency implementation}
This version is identified by {\tt code\_version\_uvt\_freq =
code\_version\_uvt\_v20 = 20}:
\begin{itemize}
\item u,v coordinates are "normal" lengths, as measured in the
  geocentric frame
\item they correspond to the frequency given in the spectral axis
\item the rest frequency is given separately
\item the spectral resolution does not distinguish between geocentric
  and selected system frame (usually LSR): it is approximate.
\end{itemize}
In conclusion, v2.0 has lousy spectral axis definition and lousy angular
resolution.

\subsubsection{Version 2.1: Global Doppler implementation}
This version is identified by {\tt code\_version\_uvt\_dopp =
code\_version\_uvt\_v21 = 21}:
\begin{itemize}
\item the frequency given in the spectral axis is also the rest
  frequency, in the source frame (at the source velocity in the
  selected system frame)
\item the spectral resolution \emph{is} in that same source frame
\item a mean Doppler factor, equal to $-v_G/c$, is specified, where
  $v_G$ is the Geocentric to selected system frame. It is stored in
  the {\tt \%gil\%dopp} variable.
\item The source velocity within the selected system frame comes in
  addition
\item u,v coordinates are in a "normal" length, as measured in the
    geocentric frame. The effective angular resolution is computed
    from the apparent observing frequency, given by
\begin{equation}
  F_{source} * (1+Doppler-V/C)
\end{equation}
\item When merging two UV Tables with different Doppler factor, u,v
  coordinates are NOT rescaled to a common Doppler factor. Instead a
  mean Doppler factor is re-computed, and a warning of precision loss
  given.
\end{itemize}
In conclusion, v2.1 has precise spectral axis definition, but lousy
angular resolution.

\subsubsection{Version 2.2: Local Doppler implementation}
This version is identified by {\tt code\_version\_uvt\_syst =
  code\_version\_uvt\_v22 = 22}:
\begin{itemize}
\item the frequency given in the spectral axis is also the rest
  frequency, in the source frame (at the source velocity in the
  selected system frame)
\item the spectral resolution \emph{is} in that same source frame
\item a time dependent Doppler factor, equal to $-v_G/c$, is
  specified, where $v_G$ is the Geocentric to selected system frame.
\item it is stored in an extra column in the UV Table. This column is
  identified by {\tt code\_uvt\_topo}. It can replace one of the 7
  mandatory column when in place modification is requested.
\item when Doppler variations are small, this column may be absent and
  in such a case, must be replaced by the global Doppler factor {\tt
    \%gil\%dopp}.
\item The source velocity within the selected system frame comes in
    addition
\item u,v coordinates are in a "pseudo-normal" length, as measured in the
    geocentric frame. The effective angular resolution is computed
    from the apparent observing frequency, given by 
\begin{equation}
        F_{obs} = F_{source} * (1+Doppler-V/C)
\end{equation}
\item when merging two UV Tables with different Doppler factors, the
  Doppler column must be propagated, or created from the mean Doppler
  factors of each table.
\item the UV Table header has an observatory section to store the
  observatory coordinates required to evaluate the Doppler factors.
  Time and Date are already in the 7 mandatory uv columns.
\end{itemize}
In conclusion, v2.2 has precise spectral axis definition and precise
angular resolution.

\subsection{The new layout}

\subsubsection{Data Format Identification}

UV Tables are now a special subset of GDF files.  As any GDF file, they are
identified by an integer code stored in variable \texttt{h\%gil\%type\_gdf}
(see Table~\ref{tbl:kinds}) which, for UV Tables must take the value
\texttt{code\_gdf\_uvt} or \texttt{code\_gdf\_tuv}, depending whether the
table is in native order (a visibility being contiguous in memory) or
transposed. For simplicity, we have imposed \texttt{code\_gdf\_tuv =
  -code\_gdf\_uvt}.

The new layout differs in several ways from the older one. In short:
\begin{itemize}
\item The \texttt{daps} are no longer at pre-defined positions in the data
  file, but their positions are described by the UV table header (\cf{} the
  following section for details).
\item The \textit{complex visibility elements} are not necessarily (real,
  imaginary, weight) triplets, but contain \texttt{h\%gil\%natom}
  elements. For example, a zero spacing UV table from a single-dish
  spectrum may have \texttt{h\%gil\%natom = 2} (as the imaginary part is 0
  in such a case).
\item Provision is made to handle polarization data, by indicating the
  number of Stokes parameters in a visibility, \texttt{h\%gil\%nstokes}.
\item The number of \textit{frequency} channels \texttt{h\%gil\%nchan} and
  the number of visibilities \texttt{h\%gil\%nvisi} are available
  independently of the transposition status of the UV data set.
\end{itemize}
The UV specific information is included in the \texttt{uvda} section of the
\texttt{h\%gil} header, as described in appendix~\ref{app:gene}.

\subsubsection{Dap positions and size}

The list of possible \texttt{daps} entities is defined in
appendix~\ref{app:uvcodes}, with the associated codes used to tagged
them. This list can be extended as required.

The \texttt{dap} position as well as its length can vary. For example, the
\texttt{item} dap would be located at
\texttt{h\%gil\%column\_pointer(code\_uvt\_item)}, and it could be a
\texttt{real(4)} or \texttt{real(8)} number depending on the value of
\texttt{h\%gil\%column\_size(code\_uvt\_item)}.

The \texttt{daps} element can be located either before the complex
visibilities, or after, but not intermixed with them.  Accordingly, there
can be \texttt{h\%gil\%nlead} leading columns before the complex visibility
values and \texttt{h\%gil\%ntrail} trailing columns after.  Note that these
number indicates the number of 32-bit columns, so if any of the
\texttt{dap} is a 64-bit number
(\texttt{h\%gil\%column\_size(code\_uvt\_dap) = 2}), \texttt{nlead+ntrail}
will be greater than the number of non-zero values in
\texttt{h\%gil\%column\_size(:)}.

The existence of \texttt{daps} entities of different lengths raise a
specific issue when transposing a UV tables.  By default in GILDAS,
transpositions are applied on the global data type. For UV tables, this is
\texttt{fmt\_r4}, i.e. 32 bit entities.  Assume we start with a natural
(UVT) order, with for example $u$ a 64 bit quantity.  After transposition
to TUV order, the first 32 bit of $u$ are in the first column, and the last
32 bits in the second. This is incoherent. Thus a second transposition is
required on these specific columns to establish the contiguity of the
64-bit values. This is handled transparently by the GIO library, in
\texttt{gdf\_transpose} as well as in \texttt{gdf\_read\_gildas}.

% *** JP: What about the SIC handling of UV tables?

\paragraph{Restriction on the use of \texttt{real(8)} daps}

The need for this capability was driven to provide simpler handling of
multi-source datasets coming from CASA (which have the RA and Dec of the
sources given for each visibility). However, the handling of such a
capability is complex. It thus is \textbf{NOT} intended to be of general
use.

In practice, it will be reserved for RA and Dec daps, when needed, and UV
Tables having such daps are intended to be transformed (split into
independent sources, or converted to a multi-field mosaic with only offsets
stored) by dedicated applications. For example, multi-source datasets
coming from CASA would be imported as FITS files, and converted to GILDAS
UV Tables by the GIO library. The resulting GILDAS data sets can then be
exploded to single-source UV data very simply.

% *** JP: I comment the following to avoid confusing the reader.
% *** JP: From our analysis (see paragraph below), our recommendation is to 
% *** JP: real(4) only.

% The capability of having \texttt{real(8)} also offers more flexibility
% for the future. However, most daps will still be \texttt{real(4)} only
% for the time being.

\paragraph{Are \texttt{real(4)} U,V,W coordinates sufficient?}

A peculiar case could be the U,V,W coordinates. With very long baselines
(20\,km with ALMA) and short wavelengths (0.3\,mm), one is tempted to say
that U,V,W should be \texttt{real(8)}, as absolute baseline lengths are
measured (and defined by the ultimate antenna stability) to a fraction of
wavelength, say 40 $\mu$m for example. 40\,$\mu$m/20\,km equals
$2\,10^{-9}$ which exceeds the precision of \texttt{real(4)} numbers by 2
digits. So in principle, \texttt{real(4)} are insufficient.

But this is only true if full astrometric calibration is required. GILDAS
UV tables are intended for imaging purposes, including self-calibration.
\texttt{real(4)} precision will only limit the field of view for a given
angular resolution. The phase error due to a (relative) numerical precision
$\delta$ is given by
\begin{equation}
  \Delta \phi = 2 \pi  u \delta    \Delta X  = 2\pi \frac{B}{\lambda} \delta \Delta X.
\end{equation}
Hence, we must have
\begin{equation}
  \Delta X \le \frac{0.1 \lambda} {2 \pi B \delta},
\end{equation}
to obtain for example a phase error lower than 0.1 radian or 6 degree.
Plugging extreme numbers ($\delta = 10^{-7}$, $B = 20$\,km, $\lambda =
0.3$\,mm), we obtain $\Delta X \le 2.4\,10^{-3}$ radian, or $0.13^\circ$,
which is quite a wide field of view.

In summary, the \texttt{real(4)} precision do not allow to perform normal
calibration (phase calibrators being in general more than a (few) degree(s)
away), but it is quite sufficient for even wide field imaging and
self-calibration.

\subsubsection{Complex visibilities}

The complex visibility values are always \texttt{real(4)}. The range of
32-bit columns for the complex visibility values is indicated by
\texttt{h\%gil\%fcol} and \texttt{h\%gil\%lcol}. We thus have the
following relations \\
\texttt{h\%gil\%fcol} = \texttt{h\%gil\%nlead+1} \\
\texttt{h\%gil\%lcol} = \texttt{h\%gil\%dim(visi\_axis) - h\%gil\%ntrail}\\
\texttt{h\%gil\%nlead+h\%gil\%ntrail} = \texttt{sum (h\%gil\%column\_size)}  \\
where \texttt{visi\_axis} is 1 for the ``natural'' (UVT) order, in which
each visibility is contiguous, and 2 for the ``transposed'' order (TUV).

The UV data format is in fact a generic telescope table format, and can
also handle tables from a Single dish telescope. As weights are often
common to all values, the complex visibility can be made of only 2
``atoms'' provided \texttt{h\%gil\%column\_pointer(code\_uvt\_weig) $\ne$
  0}. It can also have only 1 atom, if the imaginary part is always 0.  The
``atom'' types are explicitely described by the h\%gil\%atoms(1:4)
array. Currently defined atom types are
\begin{verbatim}
! Visibility atom description
integer(kind=4), parameter :: code_atom_real = 1
integer(kind=4), parameter :: code_atom_imag = 2
integer(kind=4), parameter :: code_atom_weig = 3
\end{verbatim}
Other codes may be added if needed, for example integration time (as Tsys
weighting is not necessarily the thing to do when handling different
polarizations).

\subsubsection{Telescope information}

In a UV table, there can be a \emph{telescope pair} column (code {\tt
  code\_uvt\_tele}, see value in Appendix~\ref{app:uvcodes}). It
describes the pair of telescopes used for the given baseline.\\

This column is directly linked to the telescope section possibly
available in any GDF file. The pair value is an integer equal to
$magic \times itele(iant) + itele(jant)$, where $magic$ is a factor
available in the telescope section, and $itele(xant)$ is the telescope
number from this section (corresponding to antenna $xant$ for the
baseline $(iant,jant)$). This allows to identify each telescope (in
particular its diameter). If there is only one telescope for the given
baseline, only $pair = itele(iant) = itele(jant)$ is saved. Decoding
is then simple and generic:
\begin{verbatim}
  if (pair.gt.magic) then
    itele(iant) = pair/magic
    itele(jant) = pair-itele(iant)*magic
  else
    itele(iant) = pair
    itele(jant) = pair
  endif
\end{verbatim}
The respective diameter for each antenna is then {\tt h\%gil\%teles(itele(xant))\%diam}.

\subsection{The new layout for polarimetry data}

\subsubsection{Nomenclature}

We introduce the following letters in order to identify the various
polarization states which can be encountered:
\begin{itemize}
\item X,Y for linear polarization in the sky frame,
\item H,V for linear polarization in the antenna frame,
\item R,L for circular polarization in the sky frame,
\item D,S\footnote{Letters for Latin words \emph{dextra} and
    \emph{sinistra}.} for circular polarization in the antenna frame.
\end{itemize}
We distiguish here the polarizations in the sky frame or in the
antenna frame\footnote{At this stage it is unclear if they are to be
  expressed in the antenna frame, the receiver frame, or the
  observatory frame.}. The conversion between the sky and antenna
frames is possible knowing the parallactic angle. However, some
observatories deliver natively polarizations (or Stokes products) in
one frame while other observatories deliver them in the other
frame. As we can not ensure the parallactic angle is always available
for the conversion, the polarization descriptors listed above allow to
store the polarizations in their native frame.\\
Note that the circular polarizations are also defined in a reference
frame, as their Stokes cross-products differ depending on the frame
(namely: $RR=DD$, $LL=SS$, but $RL \neq DS$, $LR \neq SD$).\\

At this stage, we can define the polarization codes and Stokes product
codes\footnote{The values H=1 and V=2 are found in the IPB/HPB files
  produced by CLIC from PdBI/NOEMA data since 2006. It is expected
  that CLIC will consistently use the other codes (for polarizations
  and stokes products) in the IPB/HPB in the future.}  (see
Tables~\ref{tbl:polar} and \ref{tbl:stokes}). Note that the values for
the Stokes products {\tt I, Q, U, V, RR, LL, RL, LR, XX, YY, XY, YX}
match the \emph{Conventional Stokes Values} from the FITS
standard. The code 0 for {\tt NONE} can be used in the data format to
indicate that no polarimetry data is present in the UV table (see next
section).

\begin{table}[htb]
  \begin{center}
    \caption{Polarization code names and values.}
    \vspace{1ex}
    \begin{tabular}{cc}
      \hline
      Name                  & Value \\
      \hline
      {\tt code\_polar\_h } & 1 \\
      {\tt code\_polar\_v } & 2 \\
      {\tt code\_polar\_x } & 3 \\
      {\tt code\_polar\_y } & 4 \\
      {\tt code\_polar\_r } & 5 \\
      {\tt code\_polar\_l } & 6 \\
      {\tt code\_polar\_d } & 7 \\
      {\tt code\_polar\_s } & 8 \\
      \hline
    \end{tabular}
    \label{tbl:polar}
  \end{center}
\end{table}

\begin{table}[htb]
  \begin{center}
    \caption{Stokes products code names and values.}
    \vspace{1ex}
    \begin{tabular}{cc|cc}
      \hline
      Name                    & Value & Name                 & Value \\
      \hline
      {\tt code\_stokes\_none } & 0 & & \\
      \hline
      {\tt code\_stokes\_i }  & 1 & & \\
      {\tt code\_stokes\_q }  & 2 & & \\
      {\tt code\_stokes\_u }  & 3 & & \\
      {\tt code\_stokes\_v }  & 4 & & \\
      \hline
      {\tt code\_stokes\_rr } & -1 & {\tt code\_stokes\_dd } & -9 \\
      {\tt code\_stokes\_ll } & -2 & {\tt code\_stokes\_ss } & -10 \\
      {\tt code\_stokes\_rl } & -3 & {\tt code\_stokes\_ds } & -11 \\
      {\tt code\_stokes\_lr } & -4 & {\tt code\_stokes\_sd } & -12 \\
      \hline
      {\tt code\_stokes\_xx } & -5 & {\tt code\_stokes\_hh } & -13 \\
      {\tt code\_stokes\_yy } & -6 & {\tt code\_stokes\_vv } & -14 \\
      {\tt code\_stokes\_xy } & -7 & {\tt code\_stokes\_hv } & -15 \\
      {\tt code\_stokes\_yx } & -8 & {\tt code\_stokes\_vh } & -16 \\
      \hline
    \end{tabular}
    \label{tbl:stokes}
  \end{center}
\end{table}

Then the following rules apply:
\begin{itemize}
\item any polarization (linear or circular) described in any frame
  (sky or antenna) can be found in a UV table,
\item there shall be a minimum of self-consistency, e.g. providing
  {\tt XX + VV} is probably non-sense,
\item the presence or absence of a parallactic angle column ({\tt
    code\_uvt\_para}) does not modify the interpretation of the DAPS
  provided in the table,
\item one can convert between sky and antenna frame by using the
  parallactic angle column, if available.
\end{itemize}

\subsubsection{In practice}

Polarimetry data can be handled in three different ways.
\begin{enumerate}
\item In the first mode, a single visibility can have only one
  polarization state, but different visibilities may have different
  states.  The polarization state for each visibility is defined by
  \texttt{h\%gil\%column\_pointer(code\_uvt\_stok)}.  Possible values
  are given by the \texttt{code\_stokes\_...} parameters specified in
  the previous section, while antenna polarizations are specified by
  the \texttt{code\_polar\_...}  parameters.  This mode is indicated
  by \texttt{h\%gil\%nstokes = 1} and \texttt{h\%gil\%order = 0}.  It
  will also have \texttt{h\%gil\%nfreq = 0} (see below). Note that
  there is no provision for mixed (circular / linear) polarization
  states.
\item In the second mode, a single visibility can handle several stokes
  parameters, but Stokes and Channels are regularly ordered. In this mode,
  channels are regularly spaced in Frequency, like in single Stokes mode,
  so no \texttt{h\%gil\%freqs} array is specified. Stokes parameters are
  also ordered. For every channel, the ordering of Stokes parameters must
  be the same.  Stokes parameters may vary first, and Frequencies next (the
  so-called \texttt{h\%gil\%order = code\_stok\_chan} ordering) or
  Frequencies may vary first, and Stokes next (the so-called
  \texttt{h\%gil\%order = code\_chan\_stok} ordering). Ordering is stored
  in variable \texttt{h\%gil\%order}.  Arrays
  \texttt{h\%gil\%stokes(h\%gil\%nstokes)} indicate the Stokes parameter
  for each frequency ``channel''.  This mode is indicated by
  \texttt{h\%gil\%nstokes $>$ 1} and \texttt{h\%gil\%order $\neq$ 0}, but
  still has \texttt{h\%gil\%nfreq = 0}.
\item In the third, most complex mode, the Stokes parameter can vary
  arbitrarily within one visibility. Then each \textit{complex visibility
    element} must have a Stokes parameter and a frequency indication, which
  are given in the arrays \texttt{h\%gil\%stokes} and
  \texttt{h\%gil\%freqs} respectively, of size \texttt{h\%gil\%nfreq =
    h\%gil\%nchan * h\%gil\%nstokes}.  As each \textit{complex visibility
    element} is given a specific frequency, this mode can also allow to
  store irregularly spaced frequency channels. Such a mode can be
  convenient for continuum bandwidth synthesis, for example. Proper
  weighting is ensured even though the channel width is not specified, as
  the weights are carried along with the Real and Imaginary parts.  This
  mode is indicated by \texttt{h\%gil\%nfreq $>$ 0}. \texttt{h\%gil\%order}
  is irrelevant in this mode, but should be set to 0.
\end{enumerate}

\section{Changes for end-users}
\label{txt:endusers}

The changes for users are minimal. As usual, compatibility has been a major
design goal in GILDAS. The new software can read transparently old GDF
files, including old UV tables. For UV data, it assumes old UV tables
contain \texttt{SCAN} as the third \textit{dap}, as this was the classical
behaviour for tables produced by CLIC.

\subsection{Behaviour}

The SIC command \texttt{V$\backslash$HEADER /EXTREMA} and the task {\tt
  EXTREMA} have now a different behaviour for UV Tables than for other GDF
data sets. They compute the minimum and maximum baseline lengths, which is
a more useful quantity in this case than the overall extrema of the data
array.

\subsection{SIC variables}

Some SIC variables mapping the file header (of any kind) have changed:
\begin{itemize}
\item \texttt{head\%dim[:]} is now a long integer array of dimension 7;
\item \texttt{head\%where[1]} and \texttt{head\%where[2]}, the ``flat''
  positions of the maximum and minimum values have been removed , replaced
  respectively by \texttt{head\%minloc[:]} and \texttt{head\%maxloc[:]}
  array coordinates in the datacube.
\item \texttt{head\%gene} is now the length of \emph{only} the
  \textbf{dimension} section.
% ZZZSB: What about length of the ``new'' Coordinate section? Do we want a
% new sic variable?  

% ZZZSG: May be, but I am not convinced. Only TABLES have no "Coordinate
% Section". For the others, it is always defined by the number of
% dimensions, so there is no surprise here...
\end{itemize}
New possibilities linked to the new UV data format are reflected in SIC.
The command \texttt{DEFINE UV} checks more strictly the type of data.  It
defines the following additional SIC variables
\begin{verbatim}
    head%basemin    Minimum baseline
    head%basemax    Maximum baseline
    head%nvisi      Number of visibilities
    head%nchan      Number of channels
    head%nstokes    Number of Stokes parameter
    head%natom      Complex visibility size
\end{verbatim}
In addition, for transposed UV tables, \texttt{Head\%U} and
\texttt{Head\%V} SIC arrays are defined to point towards the U and V
columns.

\subsection{New facilities}

Some tasks or commands may access indifferently UVT or TUV tables. Implicit
transposition is done when needed. This is currently the case for the
command \texttt{READ} of Mapping. See section~\ref{txt:transp} for details
on the new capabilities of the transposition engine.

% --- Programmers ------------------------------------------------------
\section{Changes for programmers}
\label{txt:programmers}

For images and hypercubes, the new format does not provide additional
features. So the changes are limited to the fact that dimensions and sizes
are now \texttt{integer(8)} integers. For Tables however, the new format is
much more flexible. It thus is more difficult to handle without
problems. In the context of UV tables, three features can cause
unanticipated problems if they are ignored or mis-used: 1) the possible
existence of \texttt{real(8)} \texttt{dap} elements, 2) the variable number
and ordering of \texttt{daps}, and 3) the multiple Stokes values or
irregularly spaced frequency channels. We have tried to simplify the
programming in several ways, by providing a more convenient data structure
and a comprehensive API.

\subsection{Dealing with GDF files (any kind)}
\label{txt:programmersgdf}

No access the Gildas derived type elements should be done before calling
the subroutine \texttt{GILDAS\_NULL}, as some elements are dangling
pointers.
\begin{verbatim}
  call gildas_null(hx,type)
\end{verbatim}
where \texttt{hx} is the Gildas header to be initialized, and \texttt{type}
an optional character string indicating the desired type of Gildas data
structure. Recognized values are {\tt 'IMAGE'}, {\tt 'TABLE'}, {\tt
  'UVT'}, {\tt 'TUV'} and {\tt 'VOTABLE'}, in link to the types described
in Table~\ref{tbl:kinds}. The default is set to \texttt{'IMAGE'}.

% *** JP: I don't understand the use of type. 

Others changes include:
\begin{itemize}
\item The 4 triplets {\tt (refj,valj,incj) } (with $j=1 \ \mbox{to} \ 4$)
  have been removed. For a more generic approach with 7 dimensions, these
  $4 \times 3$ scalars have been replaced by 3 vectors {\tt ref(:), val(:),
    inc(:)} (as described in section~\ref{txt:newparams}).
\item Similarly, the 4 pairs {\tt (minj,maxj)} (with $j=1 \ \mbox{to} \ 4$)
  have been removed. These $4 \times 2$ scalars have been replaced by 2
  vectors {\tt minloc(:)} and {\tt maxloc(:)}.
\item Each section length variable has been renamed as detailed in
  Table~\ref{tbl:seclen} (usually by adding the suffix {\tt \_words} to the
  GDFV1 name). The \textbf{general} section has been split into the 2
  sections \textbf{dimension} and \textbf{coordinate}. As described in
  section~\ref{txt:header}, any non-zero value is now valid to indicate
  that a section is enabled for reading or writing.
\end{itemize}
\begin{table}[t]
  \begin{center}
    \caption{Renaming of section length variables between the GDFV1 and
      GDFV2. Absent name indicate a section that has appeared or
      disappeared.}
    \vspace{1ex}
    \begin{tabular}{lcc}
      \hline
      Section      & GDFV1      & GDFV2 \\
      \hline
      General      & {\tt gene} & -                 \\
      Dimension    & -          & {\tt dim\_words}  \\
      Blanking     & {\tt blan} & {\tt blan\_words} \\
      Extrema      & {\tt extr} & {\tt extr\_words} \\
      Coordinate   & -          & {\tt coor\_words} \\
      Description  & {\tt desc} & {\tt desc\_words} \\
      Position     & {\tt posi} & {\tt posi\_words} \\
      Projection   & {\tt proj} & {\tt proj\_words} \\
      Spectroscopy & {\tt spec} & {\tt spec\_words} \\
      Resolution   & {\tt reso} & {\tt reso\_words} \\
      Noise        & {\tt sigm} & {\tt nois\_words} \\
      Astrometry   & {\tt prop} & {\tt astr\_words} \\
      Telescope    & -          & {\tt tele\_words} \\
      UV Data      & -          & {\tt uvda\_words} \\
      \hline
    \end{tabular}
    \label{tbl:seclen}
  \end{center}
\end{table}


\subsection{UV data format usage restrictions}

The Gildas UV tables signatures (12 first characters of the binary file)
{\tt GILDAS\_UVDAT} and {\tt GILDAS\_UVSOR} are not used
anymore\footnote{This restriction is actually older than the GDFV2
  definition.}. The initialization by \texttt{GILDAS\_NULL} sets the
generic signature {\tt GILDAS\_UVFIL} instead.

To simplify our life, we have imposed the following constraints
\begin{itemize}
\item There must be 7 leading \texttt{daps} before the visibilities. These
  \texttt{daps} have to be in sequence: \texttt{code\_uvt\_u, \_v, \_w}
  \textit{or} \texttt{\_scan, \_date, \_time \_anti, \_antj}. This
  \textit{default} ordering of ``mandatory'' \texttt{daps} follows the
  GDFV1 convention.
\item The 7 leading \texttt{daps} must be real.
\item Usage of \texttt{real(8)} \texttt{daps} is restricted to
  \texttt{code\_uvt\_ra, code\_uvt\_dec}.
\end{itemize}

\subsection{GDF API}
\label{txt:gdfapi}

Beside the definition of the GDFV2, the GDF API has been revised and
simplified. It is listed in appendix~\ref{app:api}. As the API provides
many tools, some redundancy exists, and one may legitimately ask which is
the best way to do things.  As usual, the answer depends on what you are
doing, and also pretty much on the actual size of your UV data sets.  Here
are some guidelines.

\begin{itemize}
\item Developing a new task.\\
  Use the integrated API \texttt{gdf\_read\_gildas} as much as possible. It
  provides consistency checks which you do not have to duplicate.  The
  drawback is that it reads all data at once.

\item  Consolidating a task.\\
  If your task requires some consistency checking before doing
  computations, consider using the integrated API
  \texttt{gdf\_read\_gildas} with optional argument \texttt{data =
    .false.}, and read the data later.  The drawback is that you have to
  allocate the data array yourself.  This can be done through
  \texttt{gdf\_allocate}.

\item  Accessing indifferently a .UVT or .TUV data set.\\
  Use the integrated API \texttt{gdf\_read\_uvdataset} as much as
  possible. To do so, you must use \texttt{gdf\_copy\_header} or
  \texttt{gdf\_transpose\_header} to generate the output header from the
  input one, depending on whether a transposition is required or not.

\item  Handling large data sets.\\
  If your application is likely to need to handle large data sets, you
  should make use of the abilities of the GDF API to read GDF files by
  blocks through the {\tt blc(:)} and {\tt trc(:)} arrays. Many operations
  are actually sequential, so this can use much less memory, and can be
  faster than considering the whole data set.

\end{itemize}

\subsection{Obsolete routines}

The following routines have been removed.
\begin{verbatim}
  gdf_blis    Change Starting Block. Not used anywhere...
  gdf_sris    Set Read status. Not used anywhere
  gdf_read    Obsolete step from the original gdf_readx, y or z
  gdf_writ    Obsolete step from the original gdf_writx, y or z
  gdf_ch      Obsolete step from the original gdf_chxy and others...
\end{verbatim}

These ones have become purely internal and been renamed to
\texttt{gio\_XYis} when applicable.
\begin{verbatim}
  gdf_cris    CReate Image Slot: now purely internal gio_cris
              Only used by SIC/defvar.f90
  gdf_geis    GEt Image Slot:
              Only used by SIC/defvar.f90
  gdf_gems    GEt Memory Slot: only the gio_gems equivalent
              Only used by SIC/defvar.f90
  gdf_reis    REad Image Slot: now purely internal
              Only used by SIC/defvar.f90
  gdf_clis    CLose Image Slot: now purely internal
              Only used by SIC/defvar.f90
  gdf_wris    WRose Image Slot: now purely internal
              Only used by SIC/defvar.f90
  gdf_exis    EXtend Image Slot: now purely internal, use gdf_extend_image
              Only used by SIC/defvar.f90
  gdf_crws    CReate Work Slot: now available by default on any "image".
              Only used by SIC/defvar.f90
  gdf_fris    Use gdf_close_image instead...
              Only used by SIC/defvar.f90 & delvar.f90
  gdf_frms    Free Memory Slot, purely internal now.
              One case in SIC/defvar.f90 to be checked...
  gdf_lsis    List Status of Image Slot
              Only used by SIC for debugging.
  gdf_rhsec   Read Header SECtion: now purely internal
  gdf_whsec   Write Header SECtion: now purely internal
  gdf_get_data  Use gdf_read_data instead, with an array allocated (e.g.
                by gdf_allocate) and set the loca%addr to point towards
                the allocated array.                 
  gdf_read_image
  gdf_free_image
  gdf_create_mapped_image
  gdf_upih
\end{verbatim}

There are a few obsolescent routines, awaiting renaming or deletion.
\begin{verbatim}
  gdf_stis      Test Read/Write status. Only used by SIC.
  gdf_geih      GEt Image Header. Only used by SIC.
  gdf_flih      FLush Image Header. Only used by SIC.
\end{verbatim}

\section{Miscellaneous and pending items}

\subsection{Compatibility}

In the development phase, the new GIO library is still able to write down
the old data format. Besides the inherent limits of the GDFV1 format (4
dimensions only, $< 2$ GBytes only), this means a number of additional
restrictions.

For UV tables, in particular, there must be no non-standard column
(i.e. the only authorized non-zero \texttt{h\%gil\%column\_pointer} are
those in the range 1-7), no polarization information or irregularly spaced
channels.

This facility is provided for debugging only, but not intended to be made
available to general users. It is controlled by the logical name
\texttt{GILDAS\_HEADERS} (which is used by all tasks) and, in SIC driven
programs, also by command \texttt{SIC HEADERS}, so that one can temporarily
write in the old format if needed. This capability will become obsolete
once the transition phase to GDFV2 is complete.


\subsection{Pending issues}

A number of items in the GDFV2 header have been kept for backwards
compatibility only. They may change in the future, when writing the GDFV1
format will no longer be supported.

\begin{itemize}
\item \texttt{\%loca\%getvm}\\
  This is clearly obsolescent, and perhaps can be removed from the gildas
  derived type.  Only SIC is likely to use this now. No application do so
  any longer.

\item \texttt{\%loca\%addr}\\
  This is now only used internally in SIC and by a few routines in CLIC,
  and one task of CLASS (\texttt{map\_ekh}. They are defined by applying
  \texttt{locwrd} to the appropriate array.

  The idea is to suppress this at some point.

\item \textbf{Rank of data set}\\
  A problem with the current code is the variable rank of the data. It
  would be convenient to use the new Fortran 2003 to have always a rank-1
  array declared, and using the Fortran 2003 capability of reshaping
  through a different rank pointer using this rank-1 array as a target.
  e.g. for a 2-D Real image:

  \texttt{ h\%r2d(1:h\%gil\%dim(1), h\%gil\%dim(2)) => h\%real }

  Allocation would be only of the \texttt{h\%real} array, so that automatic
  deallocation could be easily made when freeing the image. At least, we
  would \textit{always} know the data array name when no interface is
  required...

  Unfortunately, this is only possible with the latest version of the
  compilers being used so far (ifort 12.0 and gfortran 4.8.0).

  In the meantime, tools have been developped to adjust the rank of an
  image to the user need when possible (see \texttt{rank=} optional
  argument in \texttt{GDF\_READ\_GILDAS} and \texttt{GDF\_READ\_HEADER}.
\end{itemize}

\appendix{}

\section{GILDAS Fortran derived type}
\label{app:gildas}

\begin{verbatim}
  type :: gildas
    sequence
    character(len=filename_length)   :: file = ' '  ! File name
    type (strings)       :: char                    !
    type (location)      :: loca                    !
    type (gildas_header_v2) :: gil                  !
    integer(kind=index_length) :: blc(gdf_maxdims) = 0       ! Bottom left corner
    integer(kind=index_length) :: trc(gdf_maxdims) = 0       ! Top right corner
    integer(kind=4) :: header = 0                   ! Defined / Undefined
    integer(kind=4) :: status = 0                   ! Last error code
    real,         pointer :: r1d(:)       => null() ! Pointer to 1D data
    real(kind=8), pointer :: d1d(:)       => null()
    integer,      pointer :: i1d(:)       => null()
    real,         pointer :: r2d(:,:)     => null() ! Pointer to 2D data
    real(kind=8), pointer :: d2d(:,:)     => null()
    integer,      pointer :: i2d(:,:)     => null()
    real,         pointer :: r3d(:,:,:)   => null() ! Pointer to 3D data
    real(kind=8), pointer :: d3d(:,:,:)   => null()
    integer,      pointer :: i3d(:,:,:)   => null()
    real,         pointer :: r4d(:,:,:,:) => null() ! Pointer to 4D data
    real(kind=8), pointer :: d4d(:,:,:,:) => null()
    integer,      pointer :: i4d(:,:,:,:) => null()
  end type gildas
\end{verbatim}

\begin{verbatim}
  type :: strings
    sequence
    character(len=12) :: type    = 'GILDAS_IMAGE'  !     Image Type (see ITYP)
    character(len=12) :: unit    = ' '             !     Data Units (see IUNI)
    character(len=12) :: code(gdf_maxdims) = ' '   !     Axis codes (see ICOD)
    character(len=12) :: syst    = ' '             !     Coordinate system (see ISYS)
    character(len=12) :: name    = ' '             !     Source name (see ISOU)
    character(len=12) :: line    = ' '             !     Line name (see ILIN)
    ! For Even GDF_MAXDIMS only ! character(len=4)  :: pad_char
  end type strings

  type :: location
    sequence
    integer(kind=address_length) :: addr = 0  ! Address of image
    integer(kind=size_length) :: size = 0     ! Size of image
    integer :: islo         = 0               ! Image Slot number
    integer :: mslo         = 0               ! Memory Slot number
    logical :: read         = .true.          ! ReadOnly status
    logical :: getvm        = .false.         ! Memory / File indicator
    integer(kind=size_length) :: al64 = 0     ! Padding required for alignment
  end type location
\end{verbatim}

\newpage
\section{GDFV2 header}
\label{app:gene}
\begin{verbatim}
type :: gildas_header_v2
  sequence
    ! Trailer:
    integer(kind=4) :: ijtyp(3) = 0       !  1  Image Type
    integer(kind=4) :: form    = fmt_r4   !  4  Data format (FMT_xx)
    integer(kind=8) :: ndb     = 0        !  5  Number of blocks
    integer(kind=4) :: nhb     = 2        !  7  Number of header blocks
    integer(kind=4) :: ntb     = 0        !  8  Number of trailing blocks
    integer(kind=4) :: version_gdf = code_version_gdf_current !  9  Data format Version number
    integer(kind=4) :: type_gdf =  code_gdf_image  !  10 code_gdf_image or code_null
    integer(kind=4) :: dim_start = gdf_startdim    !  11  Start offset for DIMENSION, should be odd, >12
    integer(kind=4) :: pad_trail
    ! The maximum value would be 17 to hold up to 8 dimensions.
    !
    ! DIMENSION Section. Caution about alignment...
    integer(kind=4) :: dim_words   = 2*gdf_maxdims+2  ! at s_dim=17  Dimension section length
    integer(kind=4) :: blan_start !! = dim_start + dim_lenth + 2   ! 18  Pointer to next section
    integer(kind=4) :: mdim    = 4        !or > ! 19  Maximum number of dimensions in this data format
    integer(kind=4) :: ndim    = 0        ! 20  Number of dimensions
    integer(kind=index_length) :: dim(gdf_maxdims) = 0     ! 21  Dimensions
    !
    ! BLANKING
    integer(kind=4) :: blan_words = 2     ! Blanking section length
    integer(kind=4) :: extr_start         ! Pointer to next section
    real(kind=4) :: bval = +1.23456e+38   ! Blanking value
    real(kind=4) :: eval = -1.0           ! Tolerance
    !
    ! EXTREMA
    integer(kind=4) :: extr_words = 6    ! Extrema section length
    integer(kind=4) :: coor_start !! = extr_start + extr_words +2     !
    real(kind=4) :: rmin    = 0.0         ! Minimum
    real(kind=4) :: rmax    = 0.0         ! Maximum
    integer(kind=index_length) :: minloc(gdf_maxdims) = 0 ! Pixel of minimum
    integer(kind=index_length) :: maxloc(gdf_maxdims) = 0 ! Pixel of maximum
    !
    ! In data file, minloc and maxloc are not written. Instead, two Integer(8)
    ! mini and maxi indicate the position of the extrema in a rank-1 model of the data.
    !
    ! COORDINATE Section
    integer(kind=4) :: coor_words  = 6*gdf_maxdims   ! at s_coor  Section length
    integer(kind=4) :: desc_start  !! = coord_start + coord_words +2     !
    real(kind=8)    :: convert(3,gdf_maxdims)   !  Ref, Val, Inc for each dimension
    !
    ! DESCRIPTION Section
    integer(kind=4) :: desc_words  = 3*(gdf_maxdims+1) ! at s_desc, Description section length
    integer(kind=4) :: null_start  !! = desc_start + desc_words +2     !
    integer(kind=4) :: ijuni(3)   = 0        ! Data Unit
    integer(kind=4) :: ijcod(3,gdf_maxdims) = 0    ! Axis names
    integer(kind=4) :: pad_desc              ! For Odd gdf_maxdims only
    !
    !
    ! The first block length is thus
    !   s_dim-1 + (2*mdim+4) + (4) + (8) +  (6*mdim+2) + (3*mdim+5)
    ! = s_dim-1 + mdim*(2+6+3) + (4+4+2+5+8)
    ! = s_dim-1 + 11*mdim + 23
    ! With mdim = 7, s_dim=11, this is 110 spaces
    ! With mdim = 8, s_dim=11, this is 121 spaces
    ! MDIM > 8 would NOT fit in one block...
    !
    ! Block 2: Ancillary information
    !
    ! The same logic of Length + Pointer is used there too, although the
    ! length are fixed. Note rounding to even number for the pointer offsets
    ! in order to preserve alignement...
    !
    integer(kind=4) :: posi_start  = 1
    !
    ! POSITION
    integer(kind=4) :: posi_words = 15    ! Position section length: 15 used + 1 padding
    integer(kind=4) :: proj_start         !! = s_posi + 16     ! Pointer to next section
    integer(kind=4) :: ijsou(3) = 0       ! 75  Source name
    integer(kind=4) :: ijsys(3) = 0       ! 71  Coordinate System (moved from Description section)
    real(kind=8) :: ra         = 0.d0     ! 78  Right Ascension
    real(kind=8) :: dec        = 0.d0     ! 80  Declination
    real(kind=8) :: lii        = 0.d0     ! 82  Galactic longitude
    real(kind=8) :: bii        = 0.d0     ! 84           latitude
    real(kind=4) :: epoc       = 0.0      ! 86  Epoch of coordinates
    real(kind=4) :: pad_posi
    !
    ! PROJECTION
    integer(kind=4) :: proj_words = 9     ! Projection length: 9 used + 1 padding
    integer(kind=4) :: spec_start !! = proj_start + 12
    real(kind=8) :: a0      = 0.d0        ! 89  X of projection center
    real(kind=8) :: d0      = 0.d0        ! 91  Y of projection center
    real(kind=8) :: pang    = 0.d0        ! 93  Projection angle
    integer(kind=4) :: ptyp = p_none      ! 88  Projection type (see p_... codes)
    integer(kind=4) :: xaxi = 0           ! 95  X axis
    integer(kind=4) :: yaxi = 0           ! 96  Y axis
    integer(kind=4) :: pad_proj
    !
    ! SPECTROSCOPY
    integer(kind=4) :: spec_words  = 14   ! Spectroscopy length: 14 used
    integer(kind=4) :: reso_start  !! = spec_words + 16
    real(kind=8) :: fres       = 0.d0     !101  Frequency resolution
    real(kind=8) :: fima       = 0.d0     !103  Image frequency
    real(kind=8) :: freq       = 0.d0     !105  Rest Frequency
    real(kind=4) :: vres       = 0.0      !107  Velocity resolution
    real(kind=4) :: voff       = 0.0      !108  Velocity offset
    real(kind=4) :: dopp       = 0.0      !     Doppler factor
    integer(kind=4) :: faxi    = 0        !109  Frequency axis
    integer(kind=4) :: ijlin(3) = 0       ! 98  Line name
    integer(kind=4) :: vtyp    = vel_unk  ! Velocity type (see vel_... codes)
    !
    ! RESOLUTION
    integer(kind=4) :: reso_words = 3     ! Resolution length: 3 used + 1 padding
    integer(kind=4) :: nois_start !! = reso_words + 6
    real(kind=4) :: majo    = 0.0         !111  Major axis
    real(kind=4) :: mino    = 0.0         !112  Minor axis
    real(kind=4) :: posa    = 0.0         !113  Position angle
    real(kind=4) :: pad_reso
    !
    ! NOISE
    integer(kind=4) :: nois_words = 2     ! Noise section length: 2 used
    integer(kind=4) :: astr_start !! = s_nois + 4
    real(kind=4) :: noise   = 0.0         ! 115 Theoretical noise
    real(kind=4) :: rms     = 0.0         ! 116 Actual noise
    !
    ! ASTROMETRY
    integer(kind=4) :: astr_words = 3    ! Proper motion section length: 3 used + 1 padding
    integer(kind=4) :: uvda_start !! = s_astr + 4
    real(kind=4) :: mura     = 0.0        ! 118 along RA, in mas/yr
    real(kind=4) :: mudec    = 0.0        ! 119 along Dec, in mas/yr
    real(kind=4) :: parallax = 0.0        ! 120 in mas
    real(kind=4) :: pad_astr
    ! real(kind=4) :: pepoch   = 2000.0     ! 121 in yrs ?
    !
    ! TELESCOPE information section, normally after the UV Data section
    integer(kind=4) :: tele_words  = 2    ! Length of section: 8 used
    integer(kind=4) :: void_start  !! = s_tele + l_tele + 2
    integer(kind=4) :: nteles = 0        ! Number of telescopes
    integer(kind=4) :: magic  = 1000     ! Magic offset to code telescope pairs
    !
    ! UV_DATA information
    integer(kind=4) :: uvda_words  = 18+2*code_uvt_last ! Length of section: 18 used + codes
    integer(kind=4) :: void_start        !! = s_uvda + l_uvda + 2
    integer(kind=4) :: version_uv = code_version_uvt_current  ! 1 version number.
    integer(kind=4) :: nchan = 0         ! 2 Number of channels
    integer(kind=8) :: nvisi = 0         ! 3-4 Independent of the transposition status
    integer(kind=4) :: nstokes = 0       ! 5 Number of polarizations
    integer(kind=4) :: natom = 0         ! 6. 3 for real, imaginary, weight. 1 for real.
    real(kind=4)    :: basemin = 0.      ! 7 Minimum Baseline
    real(kind=4)    :: basemax = 0.      ! 8 Maximum Baseline
    integer(kind=4) :: fcol              ! 9 Column of first visibility information
    integer(kind=4) :: lcol              ! 10 Column of last visibility information
    ! The number of information per channel can be obtained by
    !       (lcol-fcol+1)/(nchan*natom)
    ! so this could allow to derive the number of Stokes parameters
    ! Leading data at start of each visibility contains specific information
    integer(kind=4) :: nlead = 7         ! 11 Number of leading informations
    ! Trailing data at end of each visibility may hold additional information
    integer(kind=4) :: ntrail = 0        ! 12 Number of trailing informations
    !
    ! Leading / Trailing information codes have been specified before
    integer(kind=4) :: column_pointer(code_uvt_last) = code_null ! Pointers to columns...
    integer(kind=4) :: column_size(code_uvt_last) = 0  ! Number of columns for each
    ! In the data, we instead have the codes for each column
    ! integer(kind=4) :: column_codes(nlead+ntrail)         ! Column code for each ...
    ! integer(kind=4) :: column_types(nlead+ntrail) /0,1,2/ ! Number of columns for each: 1 real*4, 2 real*8
    ! Leading / Trailing information codes
    !
    integer(kind=4) :: order = 0         ! 13  Stoke/Channel ordering
    integer(kind=4) :: nfreq = 0         ! 14  ! 0 or = nchan*nstokes
    integer(kind=4) :: atoms(4) = 0      ! 18  Atoms description
    !
    real(kind=8), pointer :: freqs(:) => null()     ! (nchan*nstokes) = 0d0
    integer(kind=4), pointer :: stokes(:) => null() ! (nchan*nstokes) or (nstokes) = code_stoke
  end type gildas_header_v2
\end{verbatim}

\begin{verbatim}
  type :: telesco
    sequence
     ! Then for each telescope
     real(kind=8) :: lon=0     ! Longitude (radians)
     real(kind=8) :: lat=0     ! Latitude  (radians)
     real(kind=4) :: alt=0     ! Altitude (m)
     real(kind=4) :: diam=0    ! Diameter
     character(len=12) :: ctele=' '  ! Telescope name
     integer(kind=4) :: pad    ! Padding for alignment
  end type telesco
\end{verbatim}

\newpage
\section{UV tables column codes}
\label{app:uvcodes}
\begin{verbatim}
! Column kind of the uv tables
integer(kind=4), parameter :: code_type_r4 = 1 ! Real number
integer(kind=4), parameter :: code_type_r8 = 2 ! Doubleprecision number
!
integer(kind=4), parameter :: code_follow   = -1 ! Column is a follower of the previous one
integer(kind=4), parameter :: code_uvt_u    =  1 ! u
integer(kind=4), parameter :: code_uvt_v    =  2 ! v
integer(kind=4), parameter :: code_uvt_w    =  3 ! w
integer(kind=4), parameter :: code_uvt_date =  4 ! Date
integer(kind=4), parameter :: code_uvt_time =  5 ! Time
integer(kind=4), parameter :: code_uvt_anti =  6 ! Antenna i
integer(kind=4), parameter :: code_uvt_antj =  7 ! Antenna j
integer(kind=4), parameter :: code_uvt_scan =  8 ! Scan number
integer(kind=4), parameter :: code_uvt_topo =  9 ! Topocentric to System Doppler effect
integer(kind=4), parameter :: code_uvt_loff = 10 ! Phase center offset
integer(kind=4), parameter :: code_uvt_moff = 11 ! Phase center offset
integer(kind=4), parameter :: code_uvt_xoff = 12 ! X Pointing Offset
integer(kind=4), parameter :: code_uvt_yoff = 13 ! Y Pointing Offset
integer(kind=4), parameter :: code_uvt_stok = 14 ! Polarization state
integer(kind=4), parameter :: code_uvt_el   = 15 ! Elevation
integer(kind=4), parameter :: code_uvt_ha   = 16 ! Hour angle
integer(kind=4), parameter :: code_uvt_para = 17 ! Parallactic angle
integer(kind=4), parameter :: code_uvt_int  = 18 ! Integration time
integer(kind=4), parameter :: code_uvt_weig = 19 ! Weight column for SD
integer(kind=4), parameter :: code_uvt_xofi = 20 ! Pointing error Ant I
integer(kind=4), parameter :: code_uvt_yofi = 21 ! Pointing error Ant I
integer(kind=4), parameter :: code_uvt_xofj = 22 ! Pointing error Ant J
integer(kind=4), parameter :: code_uvt_yofj = 23 ! Pointing error Ant J
integer(kind=4), parameter :: code_uvt_ra   = 24 ! RA of reference
integer(kind=4), parameter :: code_uvt_dec  = 25 ! DEC of reference
integer(kind=4), parameter :: code_uvt_if   = 26 ! VLA IF identification
integer(kind=4), parameter :: code_uvt_tele = 27 ! Telescope pair ID
integer(kind=4), parameter :: code_uvt_id   = 28 ! Any counter (e.g. source ID from UVFITS)
integer(kind=4), parameter :: code_uvt_last = 28 ! Last uvt code
! currently up to code_uvt_last = 29 can fit in a single block
\end{verbatim}

\newpage
\section{GDF API and programming tools}
\label{app:api}

\subsection{GILDAS\_NULL}

The subroutine \texttt{GILDAS\_NULL} is the principal initialization
routine for GILDAS data structures. No access to the Gildas derived type
elements should be done before calling this routine, as some elements are
dangling pointers.  Subroutine \texttt{GILDAS\_ERROR} will return an error
and issue a message if the gildas data structure has not been
initialized. \footnote{However, the whole GIO library can still work
  consistently on unitialized gildas data structure, as it never accesses
  the pointer elements.}

\begin{verbatim}
interface
  subroutine gildas_null(hx, type)
    use image_def
    !---------------------------------------------------------------------
    ! @ public
    ! GDF API
    !   Initialize a Gildas structure and reset its content
    !   TYPE can be: 'IMAGE' (default), 'TABLE', 'UVT', 'TUV', 'VOTABLE'
    !---------------------------------------------------------------------
    type(gildas), intent(out), target :: hx ! Gildas structure
    character(len=*), intent(in), optional :: type
  end subroutine gildas_null
end interface
\end{verbatim}
It has an optional argument named \texttt{type}, which indicates which type
of Gildas data structure is used. The default is \texttt{'IMAGE'}.

Calling \texttt{GILDAS\_NULL(h, type = 'UVT')} will set appropriate
defaults for UV data set in natural order, while \texttt{GILDAS\_NULL(h,
  type = 'TUV')} should be used for the transposed order.  Existing
pointers are nullified, and then associated to their appropriate target.

\subsection{GDF\_READ\_HEADER}

Subroutine \texttt{GDF\_READ\_HEADER} reads the header of the requested
Gildas data set (specified in \texttt{imag\%file}), and optionally change
its rank.

\begin{verbatim}
subroutine gdf_read_header(imag,error,rank)
  use gio_dependencies_interfaces
  use gio_interfaces, except_this=>gdf_read_header
  use image_def
  use gbl_message
  !---------------------------------------------------------------------
  ! @ public
  ! GDF API
  !       Read an image header from the requested file
  !---------------------------------------------------------------------
  type(gildas), intent(inout) :: imag   ! Image structure
  logical,      intent(out)   :: error  ! Error flag
  integer,      intent(in), optional :: rank  ! Desired rank
\end{verbatim}

If the optional \texttt{rank=value} argument is present, the header is
modified according to the following rules.  \texttt{value < 0} means the
image must have \texttt{hx\%gil\%ndim=-rank}.  \texttt{value = 0} means
trim all possible trailine degenerate dimensions, i.e. set
\texttt{hx\%gil\%ndim} to the last dimension $>1$.  \texttt{value > 0}
means extend or trim rank to desired value.

\subsection{GDF\_TRIM\_HEADER}

Subroutine \texttt{GDF\_TRIM\_HEADER} changes the rank of an image
header. See \texttt{GDF\_READ\_HEADER} for interpretation of the rank.

\begin{verbatim}
subroutine gdf_trim_header(imag,rank,error)
  use gio_dependencies_interfaces
  use gio_interfaces, only : gio_message
  use image_def
  use gbl_message
  !---------------------------------------------------------------------
  ! @ public
  ! GDF API
  !       Read an image header from the requested file
  !---------------------------------------------------------------------
  type(gildas), intent(inout) :: imag    ! Image structure
  integer(4),   intent(in)    :: rank    ! Requested Rank
  logical,      intent(out)   :: error   ! Error flag
\end{verbatim}

\subsection{GDF\_ALLOCATE}

The \texttt{gdf\_allocate(header,error)} routine returns the appropriate
Header\%XnD pointer, depending on the data type and rank. It uses the
\texttt{\%blc, \%trc} information to derive the proper size, given the data
rank in \texttt{\%gil\%ndim}.

\subsection{GDF\_COPY\_HEADER}

Subroutine \texttt{GDF\_COPY\_HEADER} copy the information from one header
to another.  \textbf{The output header must have been initialized by a call
  to \texttt{GILDAS\_NULL} before.}
\begin{verbatim}
interface
  subroutine gdf_copy_header(input,output, error)
    use image_def
    !---------------------------------------------------------------------
    ! @ public
    ! Copy (a part of) the input header into the output header.
    ! Since the arguments are the full gildas type, 'output' must be
    ! inout to avoid resetting the data part of the type. This implies that
    ! the other header components must also have been set/initialized
    ! before.
    !---------------------------------------------------------------------
    type (gildas), intent(inout), target :: input   ! Input header
    type (gildas), intent(inout), target :: output  ! Output header
    logical, intent(out) :: error
  end subroutine gdf_copy_header
end interface
\end{verbatim}

The behaviour depends on the type of headers, defined as
\texttt{h\%gil\%type\_gdf}.  If the destination header is of same type as
the source header, which means the same \textit{absolute value} of
\texttt{h\%gil\%type\_gdf}, all the relevant information is copied,
including the source type, as this indicates the transposition status.  If
the destination header has a different type, only the common information is
copied, so that the information which is only present in the destination
header is preserved.  This typically happens when copying an Image header
to a UV header or vice-versa.

The routine may return an error. This is a change of interface compared to
GILDAS V1.


\subsection{GDF\_RANGE}

Function \texttt{gdf\_range} converts an input range to a valid output one
given a total number of items. It is a generic function, accepting any kind
of integers.

\begin{verbatim}
interface gdf_range
  function gdf_range(nc,nchan)
    use gbl_message
    !---------------------------------------------------------------------
    ! @ public
    ! GDF API
    !       Return a range of (pixel / channel / items)
    !       Any Kind of integers is accepted
    !---------------------------------------------------------------------
    integer(4) :: gdf_range  ! Intent(out)  0 if no error
    integer(kind=*), intent(in) :: nchan     ! The actual number of channels
    integer(kind=*), intent(inout) :: nc(2)  ! The interpreted range of selected channels
  end function gdf_range
end interface
\end{verbatim}

Positive values of \texttt{nc} indicate an absolute channel number, while
negative values are offsets from the last channel. Null values default to 1
(first channel) and \texttt{nchan} (last channel) respectively.


\subsection{GDF\_READ\_GILDAS}

Subroutine \texttt{gdf\_read\_gildas} is the basic entry point to read a
GILDAS data file into a GILDAS data structure. Based on the
\texttt{hx\%gil\%type\_gdf} code (previously set by an appropriate call to
\texttt{gildas\_null} or inherited by \texttt{gdf\_copy\_header} or
\texttt{gdf\_transpose\_header}, it will perform different operations.

\begin{verbatim}
interface
  subroutine gdf_read_gildas(hx,name,ext,error, rank, data)
    use image_def
    use gbl_message
    !-----------------------------------------------------------------
    ! @ public
    ! Read a Gildas Data File of name "name" and extention "ext"
    ! and return it into the "hx" Gildas data type.
    !
    ! Desired Form & Rank can be specified: an error will happen
    ! in case of mismatch.
    !-----------------------------------------------------------------
    type (gildas), intent(inout) :: hx    ! Gildas data type
    character(len=*), intent(in) :: name  ! Filename
    character(len=*), intent(in) :: ext   ! Extension
    logical, intent(out) :: error         ! Error flag
    integer, intent(in), optional :: rank
    logical, intent(in), optional :: data
  end subroutine gdf_read_gildas
end interface
\end{verbatim}
Read a GILDAS data set and (optionally) return data from it.

Optional argument \texttt{data} is a logical indicating whether the data is
read (\texttt{data=.true.}, which is the default) or only the header is
read (\texttt{data=.false.}). The data is returned in the \texttt{hx\%XNd}
where \texttt{X = r, d} or \texttt{i} according to the data type, and
\texttt{N = 1-4} is the data rank. The array is allocated by the routine.

Optional argument \texttt{rank=value} indicates the desired rank of the
result.  \texttt{value<0} means the image must have an intrinsic rank equal
to \texttt{-value}.  \texttt{value>0} means that the data must be trimmed
of its last, or extended by, (degenerate) dimensions to match the requested
rank; an error is returned if not possible. \texttt{value=0} means the
header should take the rank from file, but the data should be returned as a
1D array.

Note: if preset on input, \textbf{hx\%gil\%form} indicates the desired type
of values.  An error will occur in case of mismatch: no conversion between
real or integers is performed.

For UV data sets (in UVT or TUV order), it reads the header and the data
according to the file layout.

\subsection{GDF\_READ\_UVDATASET}

\texttt{subroutine gdf\_read\_uvdataset(huvin,huvou,nc,duvou,error)}

Subroutine \texttt{gdf\_read\_uvdaset} is a somewhat more elaborate entry
point which performs the possible layout conversion.

\begin{verbatim}
interface
  subroutine gdf_read_uvdataset(huvin,huvou,nc,duvou,error)
    use image_def
    use gio_image
    use gbl_message
    use gbl_format
    !---------------------------------------------------------------------
    ! @ public
    !       Read UV data and Associated parameters from a GILDAS UV
    !       structure and place it in the specified array
    !---------------------------------------------------------------------
    type(gildas), intent(inout) :: huvin     ! Input UV header (data file)
    type(gildas), intent(inout) :: huvou     ! Output UV header (program used)
    integer(kind=4), intent(in) :: nc(2)     ! Selected channels
    real(kind=4), intent(inout) :: duvou(huvou%gil%dim(1),huvou%gil%dim(2))  ! Data
    logical,      intent(out)   :: error     ! Flag
  end subroutine gdf_read_uvdataset
end interface
\end{verbatim}

\texttt{huvin} must contain the data file header, while \texttt{huvou} can
have a different layout. Layout can differ in several ways: different
\texttt{column\_pointer} and / or \texttt{column\_size} arrays,
transposition order (i.e. we may have \texttt{huvou\%gil\%type\_gdf = -
  huvin\%gil\%type\_gdf}).  Transposition is done on-the-fly if needed.
The two headers must otherwise be conforming.

\texttt{nc(2)} indicate the channel range to be retrieved.  This is
performed by calling function \texttt{gdf\_range(nc, huvin\%gil\%nchan)}
(or on a local copy, as \texttt{nc} is \texttt{intent(in)} only).

If the destination requires information not present in the source, or vice
versa, an error is raised and a status code returned in
\texttt{huvou\%status} \textbf{TO BE CODED...}. It is up to the caller to
react accordingly.

\textbf{The routine currently assumes only one polarization. Polarization
  handling still needs further debate.}

\subsection{GDF\_READ\_UVONLY\_CODES}
\begin{verbatim}
subroutine gdf_read_uvonly_codes(huv,uv,codes,ncode,error)
  use gio_interfaces, except_this=>gdf_read_uvonly_codes
  use image_def
  use gio_image
  use gbl_message
  !---------------------------------------------------------------------
  ! @ public
  ! GDF API
  !       Read the Time-Baseline data of a GILDAS UV structure
  !       UVT and TUV order are allowed...
  !---------------------------------------------------------------------
  type (gildas), intent(inout) :: huv               ! UV table descriptor
  integer(4), intent(in) :: ncode                   ! Number of codes
  integer(4), intent(in) :: codes(ncode)            ! Desired codes
  real(kind=4), intent(out) :: uv(ncode,huv%gil%nvisi)  ! Return Values
  logical, intent(out) :: error                     ! Flag
\end{verbatim}
Subroutine \texttt{gdf\_read\_uvonly\_codes} allows to read only the
columns containing the requested codes.  On input, UVT and TUV order are
supported.

For example, if the requested codes contains
\texttt{\[code\_uvt\_u,code\_uvt\_v\]}, it would return the U and V values
for all visibilities (in this example, \texttt{gdf\_read\_uvonly} would
return the same information, but in a transposed way).

  
\subsection{GDF\_SETUV}

Subroutine \texttt{gdf\_setuv} verifies and finalizes a UV header. It
computes the secondary parameters \texttt{\%nlead, \%ntrail, \%fcol} and
\texttt{\%lcol} from the primary information present in
\texttt{\%column\_pointer, \%column\_size, \%nchan} and \texttt{\%nstokes}.
It also verifies the conformance to the programming restrictions.

\begin{verbatim}
interface
  subroutine gdf_setuv (hx,error)
    use image_defgrep
    use gbl_message
    !---------------------------------------------------------------------
    ! @ public
    ! Define the UV section consistently.
    ! It could also define the column_types and column_codes if needed
    !---------------------------------------------------------------------
    type(gildas), intent(inout) :: hx
    logical, intent(out) :: error
  end subroutine gdf_setuv
end interface
\end{verbatim}


\subsection{Other less used or too specific or hidden routines... To be
  debated}

\begin{verbatim}
interface
  subroutine gdf_get_baselines (mine,error)
    use image_def
    use gbl_message
    use gbl_format
    !---------------------------------------------------------------------
    ! @ public
    ! GDF API
    !   Compute the baseline range from a Virtual Memory UV data (one with the
    !   appropriate address field). Note that the address field
    !   can have been set independently of the %getvm status
    !---------------------------------------------------------------------
    type (gildas), intent(inout) :: mine   !
    logical,       intent(out)   :: error  !
  end subroutine gdf_get_baselines
end interface
\end{verbatim}
Computes the min max of the baseline lengths.  Used to update the header.


\begin{verbatim}
interface
  subroutine gdf_read_uvall (huv,array,error)
    use gio_interfaces
    use image_def
    use gio_image
    use gbl_message
    use gbl_format
    !---------------------------------------------------------------------
    ! @ no-interface (because of argument type mismatch)
    !       Read data from a GILDAS UV structure and
    !       place it in the specified array
    !---------------------------------------------------------------------
    type(gildas), intent(inout) :: huv       ! Image descriptor
    real(kind=4), intent(inout) :: array(*)  ! Data
    logical,      intent(out)   :: error     ! Flag
  end subroutine gdf_read_uvall
end interface
\end{verbatim}
Read the full UV data set (all channels included). Implicitely used by
\texttt{gdf\_read\_data} when possible.

\begin{verbatim}
interface
  subroutine gdf_write_uvall(huv,array,error)
    use gio_interfaces
    use image_def
    use gio_image
    use gbl_message
    !---------------------------------------------------------------------
    ! @ no-interface  (Yet...)
    !       Write UV data to an image file specified
    !       by its image structure. The image must have been
    !       opened for write before
    !---------------------------------------------------------------------
    type(gildas), intent(inout) :: huv        ! Image descriptor
    real, intent(inout) :: array(*)           ! Data
    logical, intent(out) :: error             ! Flag
  end subroutine gdf_write_uvall
end interface
\end{verbatim}
Write the full UV data set (all channels included). Implicitely used by
\texttt{gdf\_write\_data} when possible.


\begin{verbatim}
interface
  subroutine gdf_read_uvonly(huv,uv,error)
    use image_def
    use gio_image
    use gbl_message
    !---------------------------------------------------------------------
    ! @ public
    ! GDF API
    !       Read the UV data of a GILDAS UV structure
    !       UVT and TUV order are allowed...
    !---------------------------------------------------------------------
    type (gildas), intent(inout) :: huv               ! UV table descriptor
    real(kind=4), intent(out) :: uv(huv%gil%nvisi,2)  ! Return U,V coordinates
    logical, intent(out) :: error                     ! Flag
  end subroutine gdf_read_uvonly
end interface
\end{verbatim}
Read only the U and V values.

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
