ATLOD: Reading RPFITS Files
ATCA data will be initially in RPFITS format. This needs to be converted to
Miriad's format, by task atlod, before any further processing
can be done.
We will discuss the various input parameters to Miriad atlod.
- The in parameter gives the name of the input UNIX file, or files,
to load. Many files can be given, and wildcards are supported.
Almost all ATCA RPFITS files can now be downloaded directly from the AT
Online Archive (ATOA). Any requested RPFITS files can be saved onto your
local disk, and atlod
can be directed to load this data from there.
RPFITS files from the old correlator are commonly archived on CDROM or DVD:
a CDROM/DVD appears much like
a disk to the operating system, and atlod
can load RPFITS files
directly off these. To use a CDROM/DVD drive,
first check that the drive is not in use (an orange light flashing).
To load a disc, press the Open/Close on the front of the drive, place
the disc in
and again press the Open/Close button. After about 10 seconds, the operating
system will mount the CDROM/DVD, and it can be accessed.
CDROM/DVDs written at the ATCA will have the RPFITS data in the
You can do a directory listing to see what you have - its like a normal
You can set the in parameter to the RPFITS files in this directory.
For example, to select all files belonging to project C561, use
When you are done with a CDROM/DVD drive you eject it with the command
(the Open/Close button will not work).
RPFITS files are (or at least were once) commonly written to exabyte
tapes. Unlike other Miriad tasks, atlod
can read exabytes
directly. In this case you should use the physical device name as the
input name. This is usually
displayed on the exabyte drive. On the Epping Solaris systems, the device
name is usually
/dev/rmt/0lbn or similar
(this is the so-called non-rewinding, raw, BSD compatibility interface).
On older Solaris systems, it may be
/dev/nrst4 or something similar
(non-rewinding, raw interface).
An alternative to directly reading from exabyte is to load the data to
disk with the Unix commands ansiread
or ansitape, which are
available on a number of the ATNF systems. See the
pages for more information.
- The out parameter gives the name of the output Miriad visibility
dataset. There is no default.
- The ifsel parameter makes atlod
extract only the IFs that are
selected here from the RPFITS files. The IFs to extract should be specified as a comma-separated list.
This can be very useful if you are interested
only in a specific frequency, or if the data has two or more IFs with different
numbers of channels that would normally need to be read in with options=noif.
- The options parameter gives miscellaneous processing options.
Several values can be given, separated by commas.
Some of the options described below control on-the-fly corrections that
makes to the data, while others can reduce the amount of disk
space that the dataset will consume. It is very important therefore to
pass the correct options to atlod, to save yourself time later doing
corrections that could have been done more easily right at the start.
At frequencies above 10 GHz, the opacity of the atmosphere can be significant:
in mediocre conditions, the signal loss can be 20% at low elevations. With
high-frequency data, an atmospheric opacity correction should be applied to
the data; atlod
will perform this correction if it is passed the
opcorr option. This option can be safely included for data of any wavelength,
as it does very little (and certainly does not harm the data) at low frequencies,
and is completely ignored at 3mm wavelengths, where it is not appropriate.
For high frequency data taken with the old ATCA correlator, it is also generally desirable to
load both observing bands together, to allow joint calibration.
A shortcoming of
Miriad is that it is poor at handling multiple observing bands
when the polarisation products differ between the bands. However
the old ATCA correlator configurations that combined narrowband
and wideband observations generally did measure different polarisation
products in the two bands. As polarimetry
with the ATCA 3-mm system was not possible until recently, the simplest approach
to ensuring the polarisation products of the two bands are the same is by
discarding polarisation products that are not needed. The
nopol causes atlod
to discard any XY and YX
A few sets of permutations of the options will be appropriate with most
observations. These include:
- For CABB data:
- For old-correlator continuum:
options=birdie,reweight,xycorr. See below for a discussion on
whether to use
reweight or not.
- For old-correlator non-polarimetric line:
options=birdie,compress,hanning. See Section 16.3
for more on using atlod
with spectral line data and
Section 24.2 for high time resolution bin-mode data.
- For old-correlator 3mm data:
Possible options are:
- The old ATCA correlator suffered from self-interference at frequencies
that were a multiple of 128 MHz, while the CABB correlator displays self-interference spikes
at frequencies related to the 640 MHz sampler clock. The birdie option flags out the channels
affected by this self-interference. This option is strongly recommended.
Additionally, in the 33 channel/128 MHz mode, the birdie option
also discards some edge channels and every other channel. This operation
does not incur a sensitivity penalty, as correlator channels are not
independent in this mode. The net result is that
the output consists of either 13 or 14 good channels.
For CABB data, this option also flags 100 channels on each edge of the band, and any part
of the 20cm or 13cm band that is not within the recognised frequency range of the receiver.
- In polarimetric correlator configurations, the
ATCA makes an on-line measurement of the phase difference between the
X and Y channels. Although it is generally only a few
degrees (assuming the observation was correctly set-up), this
phase difference should be corrected when doing polarimetric work.
Since a hardware upgrade in November 1992, the recommendation has
been that the on-line measurements should
be applied, without averaging, using the
xycorr option in atlod.
For data prior to November 1992 - seek expert advise.
- In the 33 channel/128 MHz mode, a Gibbs
phenomena (see Albert Bos in the Indirect Imaging proceedings)
ATCA. This introduces a non-closing error into the data.
The effect is moderately subtle, and is not significant for
dynamic ranges of less than about 500. The `reweight' option reweights
the visibility spectrum in the lag domain to eliminate this problem.
This option is recommended for high dynamic range work (more than
about 500). However, the option also reduces the effectiveness of the birdie
option to reject self-interference. This can be a significant effect for
20cm continuum observations, where the usual observing band straddles
the 1408 MHz birdie. Particularly at 20cm, the reweight option is not
recommended where dynamic ranges of less than 500 are expected.
This option is not required for CABB data.
- Normally the correlation data are stored as
32-bit floating point numbers.
Alternatively, the compress switch can be used to instruct
to store the correlations as 16-bit integers,
with a scale factor associated with each spectrum. This approximately
halves the disk space occupied by a dataset, and so may be very
advantageous for large spectral-line observations.
- Hanning smooth the data, and
discard every second
channel. This can be useful for spectral line experiments. Task atlod
will refuse to Hanning smooth data with 33 channels or less.
This is unlikely to be beneficial for CABB data, as each CABB channel is
independent, and boxcar smoothing (or channel binning) is preferred.
- Use the barycentre (often called heliocentre
and usually used for extra-galactic work) as the rest frame when calculating
velocity information. The default is to compute velocity information
relative to the LSR frame (which is usually used for Galactic work).
- This option may be appropriate for observations
using the ATCA's high time resolution bin mode. See Chapter 24
for more information.
- This option corrects the visibility data for
atmospheric opacity. It is since October 2003 that there is sufficient
information in the visibility dataset to allow this.
This options should not be used for 3mm data. Generally it is relevant
for 12mm observations only. See Sections 22.1 and 23.2.2
for more information.
- Correct the measured visibilities for bad
sampler statistics. Normally the samplers adjust their various threshold
levels to ensure optimal operation. However various transients can cause
the sampler levels to be wrong. All is not lost - you can correct the
data for the imperfect sampler levels after the fact. Use the samcorr
(sampler-correct) option for this, and is the recommended
approach for data observed before December 1993. Since December 1993,
has been done automatically on-line. For this data, the
is (quietly) ignored.
- This option is generally not recommended.
flags data if it has not
previously read the appropriate calibration record or if the information
in the calibration record suggest the data are bad. Although calibration
records generally precede the data, there are some instances when this
is not the case, and the data are still good. In this case, the
relax option causes atlod
to be more lenient, and not flag
- Normally atlod
will discard a visibility record
(i.e. not write it to disk) if all the data in it are flagged bad.
Flagging usually indicates that there was
significant reason to suspect the data - for example the telescope was not on
source or was off-line, etc. The unflag option causes all data to
be saved to disk, although flagged
data is still marked as such. This option is generally not recommended,
although the disk-space penalty for using it is often modest.
- and nocross cause atlod
any autocorrelation or cross-correlation data respectively. It's
fairly unlikely that you will want to discard the cross-correlation data!
- When there are multiple frequency bands being
observed simultaneously, Miriad normally maps the frequency bands
to its spectral windows (IF axis in AIPS terminology), and writes
them out in a single record. Alternatively,
using the noif switch causes atlod
to write the
simultaneous frequencies as sequential records, making it appear
somewhat like a frequency-switch. You will need to use the
option if you have two spectral windows which sample different
polarization parameters, or that have different numbers of channels.
- If at least one polarization of a set of 2 or 4 polarimetric
spectra are bad, atlod
normally flags all of the
polarizations. With this option only
the nominally bad spectrum is flagged.
will usually mark as bad any scans that are labelled
as a reference pointing calibration or a paddle measurement. Normally, this data is of
little use as corrections from these scans are applied on-line. However, occasionally,
the data in these scans may be of use, and including this option causes atlod
keep this data instead of discarding it, although it will still be flagged as bad.
- At low frequencies, the large bandwidth of the CABB correlator makes
it impossible to avoid most of the known RFI. Including this option causes atlod
automatically flag regions of known interference as bad. It is also possible to specify
flagging regions manually using the file rfiflag.txt in the current directory, or the
default version in the MIRCAT directory. Each line in this file should have two numbers
representing the lower and upper frequencies to flag in MHz.
- nscans gives two numbers which are the number of scans to skip
over (before processing), and the number of scans to save. The default is
to save all scans.
- nfiles, like nscans, gives two numbers, which are the
number of files to skip and process. This parameter is only really
useful when reading from exabyte. The default is to process the first file
Note, however, that the mechanism used to skip files is rather
inefficient. If you wish to skip a large number of files, you should
do this with the Unix command mt. A
complication is that every RPFITS file appears as three files to mt - so you will want to skip three times as many tape files as RPFITS
files. For example, to skip 10 RPFITS files, you would use the Unix
mt -f /dev/nrst4 fsf 30
See the Unix man
page on mt
for more information.
Typical inputs for atlod
are given below.
||Input is either exabyte name
||or RPFITS file.
||Output visibility data set.
||Normal CABB data options, or
||Normal old-correlator continuum mode options, or
||possible old-correlator spectral line options.
||Skip 0, then read 3 files (use only for tape reading).
||Unset to save all scans
saves a number of
on-line measurements as visibility variables. These measurements may by
helpful in analysing and flagging the data. They can be plotted and
listed with task varplt
(see Section 10.7). These
on-line parameter, and its Miriad variable name, are described below.
- This is the on-line measurement of the XY phase.
It is important to examine this measurement if you are doing polarimetry.
- This is the amplitude of the correlation between the
X and Y polarization channel of a given antenna.
- xsampler, ysampler:
- These variables give the sampler statistics for
the X and Y polarization channels. There are three numbers per
antenna per IF, which reflect the sampler levels. They should have values of
17.3, 50 and 17.3. Thus for six antennas, the xsampler variable
will consist of 18 numbers per IF. These values are not available for CABB data.
- xtsys, ytsys:
- These give the system temperature, in Kelvin, for the
X and Y polarization channels. Unfortunately Miriad tasks were
originally developed around a single polarization model of a telescope, and so
most tasks that concern themselves with system temperature do not handle
dual polarization systems. The partial workaround used in Miriad has been
to store the geometric mean of the X and Y system temperatures in the
It is this variable that Miriad tasks use when they need to use the
system temperature value.
- axisrms, axismax:
- These give the antenna rms and maximum tracking
errors in a particular cycle. The units are arcseconds.
- These give various
meteorological measurements at the observatory as a function of time.
- This gives a measure indicative of the seeing conditions
at mm wavelengths, and is obtained by a two-element interferometer observing a
communications satellite at 30 GHz. This number is the RMS of the difference between
the expected path length difference and that observed, and is due to the atmospheric
conditions above the Narrabri site. It may not however directly represent the atmosphere
along the line-of-sight to the object currently being observed. A smaller number here
means a more stable atmosphere, and better conditions for mm observing.