Vis5D Documentation
<<< Previous |  | Next >>>

# Putting Your Data Into Vis5D

Vis5D can work with data organized as a 5-D rectangle. The first 3 dimensions are spatial: rows, columns, and levels (or latitutude, longitude, and height). The 4th dimension is time. The 5th dimension is the enumeration of multiple physical variables such as temperature, pressure, water content, etc.

In addition to the data itself, there are a number of parameters needed to describe a Vis5D dataset: the sizes of the five dimensions (number of rows, columns, levels, timesteps, and variables), geographic position and orientation of the data (map projection), the names of the variables, the actual times and dates associated with each timestep, etc.

The Vis5d visualization program accepts two standard file formats: v5d files and comp5d files. Both store 3-D data in a compressed format which vis5d can use quickly and efficiently. Comp5d files are those which were produced by the comp5d program in previous versions of Vis5D. The v5d file format is the new, and prefered, file format used in version 4.0 and later of Vis5D. It is intended to be a replacement for the comp5d format because it more flexible and may be extended in the future.

Vis5d can also accept a user defined file format. A user can create a new file format and the functions to read this format. See section 3.4 below.

To view your data with vis5d you will typically write a conversion program to convert your data files to v5d format. To help you do this we've included four sample conversion programs to guide you. Basically, you just add the instructions to read your file format, we provide the instructions to write the v5d file. See section 3.1 below. You then link your program with `-lv5d` (the installed `libv5d` library).

If you have used Vis5D in the past, you may continue to convert your data to McIDAS format and use comp5d to make a compressed file. However, to take full advantage of the new map projections and vertical coordinate system in version 4.0 and higher, you should write a new conversion program to make v5d files.

Another option for getting your data into Vis5D is the v5dimport utility. v5dimport is a program for file conversion, combining, and resampling. It reads a number of different file formats and can be extended to read new formats. See chapter 7 for more details.

If your data is in HDF5 format, you may find the h5tov5d conversion program useful; it is available as part of the free h5utils package.

Vis5D can also work with non-gridded data. This type of data is stored as records. Each record has a geographic location and a set of variables containing character or numerical data. Vis5D does not have it's own file format for this irregular data. The two methods for getting irregular data into Vis5D are using the current irregular import program to read certain NetCDF files, or creating a new import process using calls from the `irregular_v5d` library. See section 3.5 for more details.

# Converting Your Data to v5d Format

Files in the v5d format are created with functions from the v5d library. We've included four sample conversion programs which outline how to make a v5d file. They are located in the convert/ subdirectory. You can choose which one to use as a template for your data converter:

`foo_to_v5d.f` | A Fortran program which assumes a rectangular lat/lon map projection and equally spaced linear vertical coordinate system.
`foo2_to_v5d.f` | A Fortran program which allows any map projection and vertical coordinate system as well as a different number of vertical levels for each variable.
`foo_to_v5d.c` | A C program which assumes a rectangular lat/lon map projection and equally spaced linear vertical coordinate system.
`foo2_to_v5d.c` | A C program which allows any map projection and vertical coordinate system as well as a different number of vertical levels for each variable.

| The files in the `convert/` subdirectory link directly with the object and header files in the `src/` directory. In your own program, you'll want to instead link with `-lv5d` (the installed `libv5d` library) and include the installed headers via:
    #include <vis5d/v5d.h>

In any case, each conversion program uses three functions to write the v5d file: `v5dCreate` (or `v5dCreateSimple`), `v5dWrite`, and `v5dClose`. `v5dCreateSimple` is used to create v5d files that only specify the most basic parameters. `v5dCreate` allows more complicated parameters. There are versions of these functions for C and Fortran programs.

Here are the descriptions of the `v5dCreate` and `v5dCreateSimple` functions in a format similar to man page documentation. C programmers should note that in the argument descriptions we describe arrays by Fortran convention, i.e. `A(1)` is the first element of `A`, whereas in C this would be `A[0]`.

Fortran-callable functions:

    	  integer function v5dcreatesimple( name, numtimes, numvars,
    	  nr, nc, nl, varname, timestamp, datestamp, northlat,
    	  latinc, westlon, loninc, bottomhgt, hgtinc )

    	  character* (*) name
    	  integer numtimes
    	  integer numvars
    	  integer nr
    	  integer nc
    	  integer nl
    	  character*10 varname(MAXVARS)
    	  integer timestamp(*)
    	  integer datestamp(*)
    	  real northlat
    	  real latinc
    	  real westlon
    	  real loninc
    	  real bottomhgt
    	  real hgtinc

    	  integer function v5dcreate( name, numtimes, numvars, nr, nc,
    	  nl, varname, timestamp, datestamp, compress, projection,
    	  proj_args, vertical, vert_args )

    	  character* (*) name
    	  integer numtimes, numvars
    	  integer nr
    	  integer nc
    	  integer nl(*)
    	  character*10 varname(MAXVARS)
    	  integer timestamp(*)
    	  integer datestamp(*)
    	  integer compress
    	  integer projection
    	  real    proj_args(*)
    	  integer vertical
    	  real    vert_args(*)

C-callable functions:

    	  int v5dCreateSimple( name, numtimes, numvars, nr, nc,
    	  nl, varname, timestamp, datestamp, northlat, latinc,
    	  westlon, loninc, bottomhgt, hgtinc )

    	  char  *name;
    	  int   numtimes;
    	  int   numvars;
    	  int   nr, nc, nl;
    	  char  varname[MAXVARS][10];
    	  int   timestamp[], datestamp[];
    	  float northlat, latinc;
    	  float westlon, loninc; float bottomhgt, hgtinc;

    	  int v5dCreate( name, numtimes, numvars, nr, nc, nl,
    	  varname, timestamp, datestamp, compress, projection,
    	  proj_args, vertical, vert_args )

    	  char  *name;
    	  int   numtimes, numvars;
    	  int   nr, nc, nl[];
    	  char  varname[MAXVARS][10];
    	  int   timestamp[], datestamp[];
    	  int   compress;
    	  int   projection;
    	  float proj_args[];
    	  int   vertical;
    	  float vert_args[];

Arguments used by v5dCreate and v5dCreateSimple:

`name`

  The name of the v5d file to create

`numtimes`

  Number of timesteps (at least 1)

`numvars`

  Number of variables (at least 1)

`nr`

  Number of rows in all 3-D grids (at least 2)

`nc`

  Number of columns in all 3-D grids (at least 2)

`varname`

  Array of variable names:
  varname(1) = name of first variable
  varname(2) = name of second variable
  ...
  varname(numvars) = name of last variable

`timestamp`

  Array of time labels for the timesteps in HHMMSS format:
  timestamp(1) = time of first timestep
  timestamp(2) = time of second timestep
  ...
  timestamp(numtimes) = time of last timestep

`datestamp`

  Array of date labels for the timesteps in YYDDD format
  datestamp(1) = date of first timestep
  datestamp(2) = date of second timestep
  ...
  datestamp(numtimes) = date of last timestep

Arguments used only by v5dCreateSimple:

`nl`

  Number of levels in all 3-D grids (at least 1)

`northlat`

  Latitude of northern edge of box in degrees

`latinc`

  increment between rows in degrees (positive)

`westlon`

  Longitude of western edge of box in degrees (positive Westlongitude)

`loninc`

  Increment between columns in degrees (positive)

`bottomhgt`

  Bottom boundary of box in km

`hgtinc`

  Increment between levels in km (positive)

Arguments used only by v5dCreate:

`nl`

  Number of levels in the 3-D grids per variable:
  nl(1) = number of levels for first variable
  nl(2) = number of levels for second variable
  ...
  nl(numvars) = number of levels for last variable

`compress`

  Compression mode (1, 2 or 4 bytes per grid point)

`projection`

  Indicates type of map projection:
  0 = linear, rectangular, generic units
  1 = linear, rectangular, cylindrical-equidistant
  2 = Lambert Conformal
  3 = Stereographic 4 = Rotated

`proj_args`

  Projection arguments:

  if projection=0 then

    proj_args(1) = North boundary of 3-D box
    proj_args(2) = West boundary of 3-D box
    proj_args(3) = Increment between rows
    proj_args(4) = Increment between columns

  else if projection=1 then

    proj_args(1) = North Latitude bound of 3-D box
    proj_args(2) = West Longitude bound of 3-D box
    proj_args(3) = Increment between rows in degrees
    proj_args(4) = Increment between cols in degrees

  else if projection=2 then

    proj_args(1) = Standard Latitude 1
    proj_args(2) = Standard Latitude 2
    proj_args(3) = Row of North/South pole
    proj_args(4) = Column of North/South pole
    proj_args(5) = Longitude parallel to columns
    proj_args(6) = Increment between columns in km

  else if projection=3 then

    proj_args(1) = Latitude of center (degrees)
    proj_args(2) = Longitude of center (degrees)
    proj_args(3) = Row of center of projection
    proj_args(4) = Column of center of projection
    proj_args(5) = Spacing between columns at center

  else if projection=4 then

    proj_args(1) = North boundary on rotated sphere
    proj_args(2) = West boundary on rotated sphere
    proj_args(3) = Increment between rows
    proj_args(4) = Increment between columns
    proj_args(5) = Earth Latitude corresponding to (0,0)
    proj_args(6) = Earth Longitude corresponding to (0,0)
    proj_args(7) = Rotation angle

  endif

`vertical`

  Indicates type of vertical coordinate system:
  0 = equally spaced levels in generic units
  1 = equally spaced levels in km
  2 = unequally spaced levels in km
  3 = unequally spaced levels in mb

`vert_args`

  Vertical coordinate system arguments:

  if vertical=0 then

    vert_args(1) = height of bottom level
    vert_args(2) = spacing between levels

  else if vertical=1 then

    vert_args(1) = height of bottom level in km
    vert_args(2) = spacing between levels in km

  else if vertical=2 then

    vert_args(1) = height (km) of grid level 1 (bottom)
    vert_args(2) = height (km) of grid level 2
    ...
    vert_args(N) = height (km) of grid level N (top)
    where N is the maximum value in the nl array.

  else if vertical=3 then

    vert_args(1) = pressure (mb) of grid level 1 (bottom)
    vert_args(2) = pressure (mb) of grid level 2
    ...
    vert_args(N) = pressure (mb) of grid level N (top)
    where N is the maximum value in the nl array.

  endif

The v5dWrite function is used to write a single 3-D grid of data to a v5d file. The grid is identified by a timestep and physical variable number. Here is the synopsis of v5dWrite:

Fortran-callable function:

     integer function v5dwrite( time, var, data )
    	  integer time
    	  integer var real data(*)

C-callable function:

    	  int v5dWrite( time, var, data )
    	  int time;
    	  int var;
    	  float data[];

Table 1. Arguments descriptions:

time | A timestep number in the range [1..numtimes]
var | A variable number in the range [1..numvars]
data | 3-D array of grid values; number of values = nr*nc*nl(var) ordered as data[row+nr*(col+nc*lev)] where row increases from North to South, col increases from West to East, and lev increases from bottom to top

The v5dClose function closes the v5d file after the last grid has been written. No arguments are needed. Here is the synpsis of v5dClose:

Fortran-callable function:

     integer function v5dclose

C-callable function:

     int v5dClose()

Each of the create functions returns 1 when successful and 0 when an error occurs.

Looking at any of the example data conversion programs, you'll see that there are variables which directly correspond to the arguments to v5dCreate/v5dCreateSimple. It is up to you to initialize these variables. For example, you'll have to assign to numtimes the number of timesteps in your dataset, assign to numvars the number of variables in your dataset, etc. After you've initialized all these variables, the v5dCreate (or v5dCreateSimple) call will create the v5d file. If you've failed to initialize any of the variables you will see an appropriate error message.

Next, the conversion program will enter a nested loop inside of which you must insert the code to read your data for the appropriate time step and physical variable number. Read your data into the array specified. The v5dWrite call will then compress and write the data to the v5d file. Finally, the v5dClose function will be called after all the data has been written.

After you've written and compiled your file converter, you should test it with one of your data files then check that it worked by running the v5dinfo and v5dstats utility programs on the v5d file. If everything looks OK, try running vis5d.

Here is an example of typical values that might be assigned to each variable if one were using the `foo_to_v5d.f` program:

Assignment | Comments
--- | ---
numtimes = 5 | 5 time steps
numvars = 4 | 4 physical variables
nr = 30 | 30 rows in each 3-D grid
nc = 40 | 40 columns in each 3-D grid
nl = 20 | 20 levels in each 3-D grid
varname(1) = "U" | U (east/west) wind component
varname(2) = "V" | V (north/south) wind component
varname(3) = "T" | Temperature
varname(4) = "P" | Pressure
timestamp(1) = 140000 | 2:00:00 pm
timestamp(2) = 141500 | 2:15:00 pm
timestamp(3) = 143000 | 2:30:00 pm
timestamp(4) = 144500 | 2:45:00 pm
timestamp(5) = 150000 | 3:00:00 pm
datestamp(1) = 94036 | 36th day of 1994 (February 5)
datestamp(2) = 94036 | "
datestamp(3) = 94036 | "
datestamp(4) = 94036 | "
datestamp(5) = 94036 | "
northlat = 60.0 | Northern boundary of box is at 30 degrees latitude
latinc = 1.0 | There is 1 degree of latitude between each of the 30 rows
westlon = 100.0 | Western boundary of 3-D box is at 100 degrees longitude
loninc = 0.5 | 0.5 degree of longitude between each of the 40 columns
bottomhgt = 0.0 | Bottom of box is at 0km (sea level)
hgtinc = 1.0 | 1 km between each of the 20 grid levels (top at 19.0km)

The product of the number of rows, columns, levels, timesteps, and variables is the total number of data points. In this example: 30*40*20*5*4 = 480,000. A real dataset may be 100 rows by 100 columns by 20 levels, have 50 timesteps, and 10 variables for a total of 100,000,000 data points.

The difference between the foo_to_v5d program (which uses v5dCreateSimple), and the foo2_to_v5d program (which uses v5dCreate), is the later allows you to specify any map projection, vertical coordinate system, a different number of grid levels for each physical variable, and to control data compression. To specify a map projection, you must set the value of projection to 0,1,2 or 3 to indicate which projection, then specify the projection-dependent parameters in the proj_args arrray. Specifying the vertical coordinate system is done similarly.

It is sometimes useful to specify a different number of grid levels for each variable. For example, suppose most of your variables have 30 grid levels but a some variables have fewer grid levels, perhaps only one. Prior to version 4.0 of Vis5D, you would have had to fill in the extra levels with redundant, missing or dummy data values. With the v5dCreate function you can specify how many grid levels are present for each individual physical variable with the nl array parameter. Be aware that the amount of data passed to the v5dWrite call will depend on which variable you're writing. For example, if your grid has C columns and R rows then the number of values in the data array passed to v5dWrite for variable V must equal C*R*nl(V).

By default, the bottom-most grid level of each variable is displayed at the bottom of the 3-D box; each grid extends upward for how ever many levels are present. Sometimes, however, the bottom-most grid level of a particular variable should be positioned higher up. An example of this is a combined ocean/atmosphere dataset. There may be a total of 18 grid levels: the bottom 8 grid levels being ocean data and the top 10 grid levels being atmospheric data. In this case, the bottom of the atmospheric data should be offset or shifted upward by 8 grid levels.

Elaborating on the ocean/atmosphere example, suppose we have 2 ocean variables named S (salinity) and T (temperature) and 2 atmosphere variables named P (pressure) and T1 (temperature). There are 8 layers of ocean data and 10 layers of atmospheric data. Here is a summary showing how the lowlev array is the solution to this situation:

varnum | varname(varnum) | nl(varnum) | lowlev(varnum)
--- | --- | --- | ---
1 | S | 8 | 0
2 | T | 8 | 0
3 | P | 10 | 8
4 | T1 | 10 | 8

The lowlev array is not specified in the v5dCreate function because it was developed after the v5dCreate function was well established. Instead, the new v5dSetLowLev function is called with the lowlev array. This separate function was added to extend the functionality of v5dCreate without changing its calling sequence. Here is the synopsis of v5dSetLowLev:

Fortran-callable function:

    	  integer function v5dsetlowlev( lowlev )
    	  integer lowlev(*)

C-callable function:

    	  int v5dSetLowLev( lowlev )
    	  int lowlev[];

Argument description:

lowlev: Specifies the vertical offset, in grid levels, for each variable.

lowlev(1) = offset for first variable

lowlev(2) = offset for second variable ... = ... lowlev(numvars) = offset for last variable

v5dSetLowLev may be called at any point between v5dCreate and v5dClose.

The v5dCreate and v5dcreate functions allow you to control how the grid data are compressed. The default is for grid values to be linearly scaled to one byte integers. This works very well for most data sets, since the scaling factors are chosen independently for each combination of time step, variable and vertical level. Furthermore, the compression to one byte per grid point enables Vis5D's high degree of interactivity, since compression allows entire data sets to be resident in memory. However, the compress argument of the v5dCreate and v5dcreate functions lets you pick whether grid point values are scaled to 1-byte integers, scaled to 2-byte integers, or left as 4-byte floating point values (no compression).

| We recommend that you try compression to 1-byte integers first, and only use 2 or 4 bytes if you have precision problems at 1-byte.

Vis5D version 4.2 and later allow you to specify the physical units for each variable in your dataset. The v5dSetUnits() function takes two arguments: a variable number and a units character string. If the first variable in your file is P and the units are millibars then you can specify that with:

C: v5dSetUnits( 1, "millibars" )

Fortran: call v5dsetunits( 1, "millibars" )

The units will be displayed by the v5dinfo program and in Vis5D when using the probe.

To compile your program which uses v5dCreate, v5dWrite, and v5dClose you must link with the `src/v5d.o` and `src/binio.o` files. Alternatively, link with `-lv5d` to link with the installed `libv5d` library. See the makefiles in the convert/ directory for examples.

Finally, if your data is generated by an atmospheric or oceanic model, you may want to consider modifying your model to generate v5d files directly using the v5dCreate, v5dWrite, and v5dClose functions. Look at the sample data conversion programs for ideas.

<<< Previous | Home | Next >>>
Installing Vis5D |  | Map Projections and Vertical Coordinate Systems