ATUC Meeting
ATCA on-line data reduction
at the ATNF in Epping, Sydney
- March 23, 1999 -
AT Compact Array on-line data reduction
SUMMARY: A trial system for on-line imaging of ATCA visibility data has been produced and is in limited use. It is suggested here that users be encouraged to use this system when possible. Experience gained should be used to design a system which would: (i) augment the data quality monitoring functions of VIS, CASNAP and CORDISP; (ii) provide complete data reduction for a limited set of observation types ; (iii) provide another educational tool for students and other Array users. The new system should make use of AIPS++ tools to provide the flexibility needed to cater for different observation types and to provide a familiar graphical interface to the user.
BACKGROUND:
In the past efforts have been made to give observers views of the incoming data. These views provide valuable means of checking the integrity of the data and in some cases give limited scientific information which influences decisions on how to proceed with the observation. The two viewing facilities, the programs VIS and CASNAP, have used spectrally averaged summaries of the visibility data. A third, CORDISP, displays the frequency spectrum for a single, user selected baseline.
VIS - Plots visibility amplitudes and phases, usually against time. Other plots can be requested such as amplitude vs. baseline length and real vs. imaginary.
CASNAP - This program has been offered intermittently since 1992. It produces 1-dimensional images from short integrations (typically 1 minute) of visibility summaries. It produces good results for simple fields - dominated by a small number of point sources.
CORDISP - This displays the spectrum for one polarisation product for one baseline. It can show the spectrum for each correlator integration (typically 10 seconds), or an accumulated spectrum can be displayed.
The next step would be to produce 2-dimensional images in real-time and this has been discussed with the ATNF and also at the 1998 meeting of the AT Steering Committee. During 1998 a modest effort from Frederic Badia has produced a trial system which currently runs on a work station in the control room. Frederic's description of his system is included below. It uses Miriad for most of the processing and constitutes a straw-man system which can be the basis for discussions of future developments. There is a wide variety of observations made with the ATCA and it is difficult for a simple on-line system to cater for each observation type. Examples are:
- continuum imaging of a single field;
- imaging a mosaic of adjacent fields;
- imaging (with sparse u-v coverage) a number of distinct fields;
- any of these but in a (sometimes large) number of spectral channels;
- imaging in time bins (usually for pulsar binning).
What are the aims of on-line imaging?
For many observations some amount of off-line data reduction is necessary, such as those involving the combination of data from different arrays. For all observations the on-line imaging should extend the data monitoring functions of VIS and CASNAP. For some observations it should be possible to completely reduce the data by the end of the observations. Examples are flux measurement of point sources, continuum imaging of single fields, measure spectra (emission or absorption) at a single point in the image plane.
Education. Both VIS and CASNAP have been very useful in illustrating the theory and practice of aperture synthesis to both students and experienced observers, as well as observatory staff. On-line 2-dimensional imaging would add to this suite of educational tools.
How to proceed?
- 1. Encourage ATCA users to use the prototype system. This will require advertisement and user documentation. Observatory staff should actively seek comments and suggestions from users.
- 2. Specify and design and new system to accommodate perceived requirements. Effort should be expended on those classes of observation for which complete on-line data reduction is possible.
- 3. Use AIPS++ for the next generation system. This may provide, ultimately, the flexibility needed to provide a complete data reduction facility for a variety of projects. The AIPS++ user interface, both user input and graphical display, should be used in that observers are presented with a familiar looking product.
A prototype on-line imaging system for the ATCA
The purpose of this project is to allow astronomers to have a look at their images while they are doing the observations. The system presented here is based on Miriad scripts written in C-shell with a Motif GUI on top of them. Eventually it will be written in Glish when Aips++ has become stable. At the moment we
- Load the data from the rpfits file and dispatch it into source.frequency files in a incremental manner. This is done using a modified version of the miriad tasks atlod and uvsplit
- Apply a flagging/calibration/imaging process to the data we already have every hour or so.
- Record an image of the current state of each observation in the server memory and add it to a movie illustrating the synthesis process.
Flagging
The flagging of the data is optional and automatic (based on average absolute deviations). The user can specify the clip level applied.
Calibration
Calibrators are associated to sources in a automatic way. Every source which is found in one of the most common catalogues (ATCA, VLA ...) is used for the secondary calibration of the source which immediately follows it (in time). A control panel allows the user to change interactively this association just by dragging the mouse from a source to a calibrator.
Imaging
By default every field is imaged at every frequency and the calibrators are also imaged using the observation of a previous calibrator. This can lead to a large amount of computing time, especially if the images are deconvolved. So the user has the possibility to turn on and off the imaging process of each source, to choose the frequencies being imaged, and if he wants to deconvolve or not.
Frederic Badia
Dave McConnell
22 February 1999