Making HIPASS/ZOA cubes

0. Preliminaries

1. Which Parts of the Sky have been observed?

Find out directly from the HIPASS and ZOA on-line logs, use coverage.pl (below), hipass_status.pl, or consult the cube archive.

2. List the cdroms required

Type:

      coverage.pl -c cube_number [-m scan_letters]
where cube_number is the standard HIPASS cube number. Normally, all scan letters "abcde" are read (i.e. the default is "-m abcde". If only a specific scan letter, e.g. "a" is required, use "-m a". Scans for non-standard HIPASS cubes can be read using the switches "-s" (start scan number), "-e" (end scan number) and "-d" (declination band), instead of "-c". HIPASS scans known to be in error are not listed. Quiet levels -q 0, 1 or 2 are possible.

For ZOA scans, use: 

      coverage-zoa.pl -l long [-s start -e end -m scan_letters]
where long is the Galactic longitude of the scans you want to grid. All scan numbers (start=1 and end=17) and scan letters (scan_letters=a-y) are read unless specified otherwise. Regular expressions for scan letters are supported (e.g. a-m,q). Additionally, for ZOA scans, the start and end observation dates may be specified (-b = start_date and -c = end_date). The format is YYMMDD. ZOA scans known to be in error are not listed.

You will get an output that looks something like this: 

      ************************************************************
      coverage.pl starting Mon Jun 23 16:23:08 Australia/NSW 1997
 
      ************************************************************
      File type: SDF
      Longitude band: 304
      Sequence start, stop (latitude): 1, 17
      Files will not be read
      Parsing file SDF_970228_1551.size
      Parsing file SDF_970301_0919.size
      Parsing file SDF_970301_1702.size
      .
      .
      .
      001     agm
      002     a m
      003     a m
      004     a m
      005     a m
      006     a m
      007     a m
      008     a m
      009     a m
      010     a m
      011     a m
      012     a m
      013     a m
      014     a m
      015     a m
      016     a m
      017     a m
      Required CDs are:
      29 57 80 
      **********************************************************************
In this example, the required cdroms are 29, 57 and 80.

NB. coverage.pl interrogates the environment variable MB_CATALOG_PATH to find the whereabouts of the listing of archive files. This is normally set to /nfs/atapplic/multibeam/archive (which points to a local disk at Parkes and Epping). This directory is updated once a day. At Parkes, it may be preferable to use the `real-time' listing, in which case the variable should be reset, i.e. 


setenv MB_CATALOG_PATH /p/multi/archiver/hipass_zoa/size_files

3. Load the cdroms required

Having extracted the required cd's, you have to load the data. This process involves reading the sdfits files from the cd's, converting to mscal files, and dumping the data onto disk. Allow 11 MB for each mscal file. A fully gridded HIPASS region (a-e scans) should have 75 scans, so allow around 1 GB for all the mscal files. A fully gridded ZOA cube (a-y scans) needs 5 GB.

To load the data, you need to run coverage.pl again with a few extra parameters: 

      coverage.pl -c cube_number -r cdpath -w writedir -x rextn -y wextn -a n
where cdpath is the path of the cdrom (usually /cdrom), writedir is the place you want to write the data to (e.g. /data/multi/mscal), rextn is the read extension (i.e. sdfits), wextn is the write extension (use mscal), and n is the auto-recurse option. n=1 will search subdirectories under the specified directory; n=0 will only search the specified directory for files.

You will need to run this for each cd that is listed, i.e. load the cd, then run coverage.pl, eject the cdrom, then load the next. On attila (Epping), the following example will extract all scans for cube H014 from all slots in the cdrom changer: 

      coverage.pl -c 14 -r '/cdrom/cdrom[2-7]' -w /DATA/MULTI_1/mscal -x sdfits -y mscal -a 0
Use the command cd_mountall on attila to mount the cdrom changer (cdroms label down), and cd_umountall to dismount the cdrom changer. Note that the cdrom changer can generate I/O errors. To overcome this, repeat the sdfits2ms operation on the appropriate file. coverage.pl can also be used to copy sdfits files (instead of converting to mscal) using -x sdfits -y sdfits and to copy rpfits data files from the raw cdroms (-t HPF).

4. Check the mscal files

Corrupt mscal files can cause positional information to be jumbled whilst making a cube. It is therefore VERY important to check the integrity of all mscal files to be gridded.

In each directory containing mscal files, type: 

   mscheck [-d]
The "-d" option will also delete any corrupt mscal files.

If cubes are to made using the "autosize" option (i.e. ZOA or general cubes), the positions in the mscal files should also be checked with the script: 

   msposcheck
This will report mscal files for which any beam position lies greater than 10 degrees from the MEAN of all beam positions contained in that mscal file.

Finally, to check if all HIPASS mscal (or sdfits) files are present in the current directory, type: 

   hipass_check.pl -x mscal/sdfits -s start cube -e end cube
e.g. 
   hipass_check.pl -x mscal -s 340 -e 342
will check that all mscal files needed to grid cubes 340, 341 and 342 are present.

5. Creating cubelets with the Large-Scale Gridder

In Epping or Parkes, type "gridzilla" for the GUI version.

The details of the the gridding script and and how to use it are documented here.

The output of gridzilla is a series of FITS cubelets, Their names will be, for example, H010_001.fits, H010_002.fits,...

6. Concatenating cubelets (mosaicing in velocity), Hanning smoothing

In Epping and Parkes, use cube_helper.

cube_helper is self-documenting. Enter the names of the input cubes (either individually or using the appropriate wildcard), select the output cube name and directory, then "go"!

ZOA cubes should be made in both hanned and unhanned versions. cube_helper can do hanning smoothing also.

7. A Note on Spatial Mosaicing of cubes

Areas larger than the standard field size may be gridded by selecting the appropriate scans, and driving the gridder manually. However, it is often impractical to go back to the cdrom sets every time.

A better way is to mosaic the standard cubes together. The fact that they already have substantial overlap means that little information, if any, is lost. At present, we don't have any mosaicing software that correctly feathers the edges together. However, routines in miriad allow a reasonable job to be done.

Firstly, gather the cubes to be mosaiced and locate the central cube.

Then regrid the other cubes so that the tangent point (and projection geometry) is the same:

Then, combine the cubes together with equal weight everywhere:

Note that if you are handling cubes that were made with "smallcube=F" (i.e. IEEE FITS cubes), they will not be blanked. In this case, you will have to run maths exp=file mask=file.ne.0.0 out=newfile on each cube before the above procedure.

Lister Staveley-Smith and David Barnes
Last modified: Fri May 14 11:08:15 EST 1999