Next: LBACOR Up: index Previous: Correlating the Next Set   Contents

Obtaining Your Final Files

The RPFITS *.rpf data files are written out by the correlator to the data disks on lbaccc
/data/LBACCC_1/corr/dat or /data/LBACCC_2/corr/dat
(which can be read accessed from other network machines on the ATNF computer system as /DATA/LBACCC_1/corr/dat and /DATA/LBACCC_2/corr/dat). You should watch the space on these disks, as they can fill up quickly with some correlator configurations with lots of channels (ie spectral line) or bins (ie pulsar data). Which of the two disks is used depends on the enviroment variable cor_fits_dir set on lbaccc (setenv cor_fits_dir /data/LBACCC_1/corr/dat). Also, you should watch the size of the individual RPFITS files ( MB), since you will want to back up your data on your own export CD and the data will also be archived onto CD.

The hardware and software available to write CDs at Epping/Marsfield has changed over the last few years, as computers are upgraded, so here is a current summary (May 2003). See the ATNF web pages
http://www.atnf.csiro.au/internal/computing/help/cdwriter/cdwriter_mkiso...
for any changes.

The writing of CDs can be done from one of three public machines workstations on the ground floor, above the basement correlator room
apus and ajax in the Northern Workstation Room, Rm94, and virgo in the Southern Workstation Room, Rm 23.

The writing of CDs currently occurs in 2 steps. The format that files are written onto CD is ISO9600 (data files, not audio or video disks), so firstly the files in a directory are combined and translated into this format with the program mkisofs. Secondly, the ISO file is written onto the CD with the program cdrw. The second step is time critical, with any glitches to the data stream to the CDROM writer spoiling the writing, so this cannot be done over the network, with the ISO file needing to be from the local machine's scratch disk. Therefore the first step in creating the ISO file can be used to copy the data across the network onto the local CD writing machine.

First we move the recorded datafiles to a temporary directory. All VLBI archive CDs are labelled lba_aNNN, where NNN is the number of CD we're up to. In this example, I will give the steps for making archive CD lba_a442, so the subdirectory was 442. Most of the ``housekeeping'' for archive CDs is not needed if you are writing your own export CD for your own use, but it is very important for the archive to have these records ! The maximum capacity of 1 CD-ROM is 1.247 million blocks, or 640 Mbytes, so we put about 600 MB of RPFITS files into this subdirectory. We need to be logged in to lbaccc as ``corr'' to have permission to do this. We then make a listing of the files for the archive records from this subdirectory
ls -l ../lba_a442.lis
mv ../lba_a442.lis .
cksum * ../lba_a442.cksum
mv ../lba_a442.cksum
putting the listing of the files in the subdirectory, and making a checksum record of the files.

On ajax (Rm 94), or one of the other machines with CDROM writers, we then create a scratch area on the local disk, eg. /DATA/AJAX_1/AIPS/vlbi, for the ``vlbi'' login, and create the ISO file with a command like
mkisofs -o lba_a442.iso -a -l -L -A "LBA_A442" /DATA/LBACCC_2/corr/dat/422
. This may take about 15 minutes. Do not do this while the correlator is running, as copying large files puts a strain on network resources, and may upset the correlator writing to the lbaccc data disk. See the above mentioned website or ``man mkisofs'' for more details of the parameters.

The CDROM is then written with a command like
cdrw -v -i lba_a442.iso
This should take about 25 minutes for a full CD.

The CD should be checked in a CD reader (most workstations have readers) and the listing of files (ls) checked against the previous list from disk (eg. lba_a422.lis in above example). You don't want to find out later that the CD (particularly the archive) is unreadable ! For a more comprehensive check, which may take some minutes, do the checksum of bits compared to the previous one from the hard disk (eg. lba_a422.cksum).

For the archive CDs, put these listings into the area /u/vlbi/lba (logged in as ``vlbi''), eg. lba_a422.lis, lba_a422.cksum, lba_a422.cdlis, lba_a422.cdcksum.

VLBI export CDs have been labelled lba_xNNN, again where NNN is the number of export CD we're up to. You could use this labelling system, or just the experiment code (eg. v145a1_1) for your own export CDs. Ask correlator personnel Paul Jones or Tasso Tzioumis to find out where things are up to.

Congratulations - you are done !

Next: LBACOR Up: index Previous: Correlating the Next Set   Contents