-------------------------------------------
next meeting: Thursday, March 2; 11:00 am.
--------------------------------------------
CA-forum #6 - minutes
3 Feb 1995
A. December Upgrade. (WEW/DMcK)
1. Dynamic Hold now installed and working. CACOR will accept a request
(prior to any integration) for a hold which is a multiple of the
CAIN-specified HOLD INCREMENT.
Warnings:
a. The Increment should be a multiple of the switched CAL cycle (128 ms);
512ms is recommended.
b. CACOR allows a Hold of up to 50 multiples of Increment
- This puts a limit of 20 sec max integration cycle with the default.
>>> CAIN should perhaps check that these conditions are not violated.
>>> It would be desirable to be able to monitor the HOLD - camon/status?
2. The sampler statistics are now collected and the sampler levels
reset within a cycle. (contrary to the statement in CA-forum #5 minutes).
-- Narrabri to run some further tests (eg, on Centaurus) to confirm that all
is well.
3. The drive-time algorithm is now in place and linked to the
dynamic hold.
==> Except for the digital synch demods (see below) the mosaicing
questions raised earlier have now been resolved.
4. The (uvw) written to RPFITS are not corrected for HOLD. The
modifications to CAOBS (M.Calabretta) and CACOR (WEW) will be installed
in the april shutdown.
5. Other problems now fixed/warnings:
- Integration times up to 30 seconds are now possible. (Narrabri to run
some further tests to confirm).
- Averaging during mosaicing should be avoided.
B. Digital Sync Demods.
George Graves has completed the Seti work, and has committed himself to an
April 3 target.
C. Fractional MHz at Narrabri.
It was agreed that the on-line option should be defeated, and all
references to it be removed from the documentation (eg, the user-guide).
Sched will be modified (MJK) to enforce this, with a warning.
One additional argument advanced by WEW was the problem of the 128 MHz
harmonics. With integral MHz LO settings, the interference will be
confined to a single channel; The interference will distributed
over many channels if the correlator provides a fractional MHz shift.
D. Data quality.
Neil Killeen provided a list of discussion points - attached to these minutes.
This list was not given all the attention it deserved - we should return to
this subject next month. A preliminary list of recommendations is attached:
1. Visibility phase stability. An effort should be made to characterise
the site -- just how good/bad is the site? Is the LO system working?
Observations of strong calibrators will be made whenever time is free
(eg, night-time after reconfiguration, maintenance).
All calibrator observations should be siphoned off to a separate file,
along with the LO round trip phase data.
>>> DMcK to investigate.
2. Standard fields (eg, a strong point source offset from the field
centre) should be observed on a regular basis, possibly after
each reconfiguration.
3. How often do we really need to run CACAL? Only after a power problem
should a new delay calibration be required.
>>> Reference CACAL could be run on all bands after a reconfiguration.
These should provide good delay settings. Users should be free to change
these, but most observers should be able to use the reference settings.
Neil Killeen adds:
Delay Matters
-------------
There was discussion over delays. Why do we continually need to set
them (i.e. every 12 hours). The discussion was centred largely on
spectral-line observations where observers cannot usefully set their
delays with CACAL because there is insufficient SNR. Of course, at
narrow band, the delays do not have to be very good, so there is
actually no need for a high SNR setting. However, they need something
in the right ball park. Also there was the question of the N*128MHz
birdies causing bad CACAL computations.
Birdies
-------
It is suggested that CACOR automatically excise channels containing
birdies when it computes the delay from the phase slope across the
bandpass. This should be straightforward as the birdies occur at
integral multiples of 128MHz and the frequency at the centre of the band
must be an integral MHz. At 128MHz bandwidth observations, the
birdie will ring into adjacent channels. Thus, eliminating three
channels centred on the birdie should be adequate at all bandwidths.
>>>> WEW advises that the modifications are in hand and will be implemented
at the next shutdown (april).
Note that in the future when we devise a system to observe with the band
centre at a fractional MHz (necessary for the narrow band filters to
come), that the birdie may straddle channels. However, eliminating
3 channels should still deal with the bulk of the interfering signal
so that delay and gain are adequately measured.
Discussion about eliminating the birdies themselves indicated that
progress is slow in tracking down exactly where they are entering the
system. It was felt that it is important they be eliminated, but that a
substantial effort would be required to do so.
Computation of delays
---------------------
The delay system works as follows. There is a table of transmission
times for each antenna station, that accounts for the basic large delay.
This is updated as necessary, but should be basically static and is
always applied. All delays referred to below are offsets to these
values which will be taken as implicitly applied. There is then a
global delay worked out by the staff at reconfiguration. This global
delay should be small, of the order of tens of ns (as it is relative to
the transmission times file). This delay is common to all frequencies.
Running CACAL then generates an offset which is added to the global
solution to generate the working delay.
The system uses the delays from the closest frequency if you do not set
them yourself. In principle, the offset delay table should be reinitialized
at each reconfiguration. It is possible that this does not always
happen, whereupon you might end up with delays from several months ago, but
this should not pose any problems. (Experiments are needed to determine
whether differential delays within a band vary with time - at this stage
we know only of variations in the global delays).
Here are two ways to deal with the spectral-line delay issue.
1) Work out the delays in a 128MHz bandwidth first at the frequency of
your interest. Then switch to your spectral-line configuration. This
suffers from the problem that you have to mess about reprogramming the
correlator. In addition, the broad band measurement may suffer from
intereference (e.g. 18 cm) and still not be useful.
2) Examination of the current set of delays in the data base suggests
that the working delay varies between the 4 frequency bands by an amount
of the order of 10ns. Thus, using the global delay should be adequate
for any narrow band experiment. It is unlikely anyway, that even at
8MHz bandwidth and using 1934-638 at 20cm that you could set them much
more accurately than this.
CACAL offers you the possibility to reset the offset so that you can use
the global value. However, as most people are not aware of the details
of how this system works (nor should they need to be), or when they
should reset, this causes considerable confusion.
We suggest then that CACAL be modified, so that when it determines that
there is insufficient SNR to adequately determine the delays in a
spectral-line observation, that it automatically sets the appropriate
offset to zero and offers the global delay as the working delay with
some soothing messages. This would be straight forward to implement.
In the longer term, perhaps we should consider working towards
eliminating the need for observers to have to set the delays, or at
least offer them an excellent fall back, even in broad band
observations. In this case, we must characterize the variation of the
delay across each band as well as from band to band. If the variation
across the band is very small (less that a couple of tenths of a ns),
then a predetermined global offset for EACH BAND (requiring changes to
the appropriate data base [SACCOM]) would be adequate. However,
seasonal and probably to a lesser degree diurnal variations will cause
these global delays to vary. Thus, we would need to determine them
periodically, probably weekly (as is the case at the VLA).
Some data will be taken to investigate some of these latter issues.
--------------------------------------------------------------------
Attachment - Neil Killeen's thoughts on data quality.
We will return to these matters at next month's meeting.
Fundamental question. What is RAW data and what isn't. Where do
we draw the line between trying to undo things if they get stuffed
up ? When should the observer have to reobserve ? This issue usually
causes severe emotional responses from some people.
What do observers need in order to assess the quality of their data ?
I.e., why is this piece of data mucked up. Is it recoverable or not.
I am thinking of this in an offline sense rather than online,
as we will evolve towards a system where observers do not go to the
ATCA all the time, and may just receive their data on a CD Rom.
Is the quality of ATCA data sufficiently good now that we can just
discard obviously wrong data and not be concerned with why it was
wrong ?
Do subtle things still go wrong for long periods of time that this
approach is not good enough ? How can we improve this (FAULTY
or ASSISTANCE) ?
Are there additional corrections that could be made for high dynamic
range imaging that require ancillary data currently not stored
or measured ?
What additional pieces of data would be useful to look into subtle
problems, after the data have been taken. That is, what should be
stored with the RPFITS file ?
Should the FAULTY project (expert system) be resurrected ?
Are the XY phases good enough ? Can we apply them on-line and
forget about them ?
Should we have a simple algorithm running on line that expunges
obviously bad data ?
Do the inexperienced observers get sufficient good advice
from DA's and local staff. That is, are observations made
as effectively as possible ?? Is the DA system good enough ?
What we store with the RPFITS file now
---------------------------------------
. Visibilities
. Pulsar bin number
. uvw
. Observed frequency
. UT time
. Astronomical coordinates
mean only is stored. Apparent is lost and although recomputed,
not with the accuracy of the online ephemeris. This also means
that uvw and parallactic angle are in error if recomputed.
. Integration time (allowing for HOLD and BLANK)
. XY phases and amplitudes (averaged over band, no spectrum saved
unless special correlator config. concocted with losses elsewhere)
. Tsys
. Sampler statistics (corrections for imperect sampler stats done
on line now. Relies on DC level being close to correct [usually
true], and sampler not being too far wrong [hard to quantify,
efforts so far have failed).
. Parallalactic angle
. Antenna locations
. Flag. Spectrum flagged good or bad by on-line system.
What is there already space for in the RPFITS file that we do not use ?
-----------------------------------------------------------------------
. Basically, a variety of tables. A new table can always be created
for any new quantity, but a table is SCAN based by definition.
Currently the only unused or empty table included in the original
definition of RPFITS is the MT (meteorological) table,
containing time, pressure, temperature, and humidity
Can we add other integration based ancillary data to an RPFITS file ?
---------------------------------------------------------------------
. Yes, the table where quantities such as Tsys, XYphase etc are
stored can be extended (with some pain). This allows for
integration based quantities for each frequency and antenna.
Quantities that are polarization dependent (samplers, Tsys) are
stored by as a different parameter. I.e. polarization is
not a column in the table.
What else might we like to store/know ?
---------------------------------------
. CACAL (online calibration program) determined quantities such as
the applied delay, gain (amplitude and phase), polarization
leakage (not yet). Scan based.
. Local Oscillator Round Trip Phase which provides a measure of
the phase stability of the system with time. Integration based.
Should be undoable but is it worth it ?
. What the pointing model being applied was. This would
be SCAN based. Local pointing solutions would cause this
to change with time. In principle the pointing can be undone
if you know the primary beam to large radii and azimuthally.
VERY nasty. May be relevant for high precision flux density
measurement.
. What the atmosphere is doing (P,T, Humidity, wind velocity)
This could be integration based, but SCAN bases is probably good
enough. This offers better refractive index phase corrections
with standard atmosphere models. SHould go into local
pointing model ?
. Doppler velocity of observatory
. What the focus position is. Integration based ???
. Ephemeris information allowing accurate recomputation of
uvw etc
. Pulsar ephemeris
. Where the SUN is
. What the SCAN type is
. What the ionosphere is doing with time. This is impractical in real
time, but we do have software to compute the RM(t) from the
IPS data base (A Kukushkin). Needs to be integrated with data.
. What Array the observation was made with
. Latitude and longitude of observatory (usualyy programs use
a data base, which may or may not be accurate). Altitude
of observatory (relevant to atmospheric models ??)
.
Major types of data problems encountered today are:
-----------------------------------------------------
. Atmosphere
. Interference (await total collapse of Russian Economy)
. Mismatch between data and array (i.e. dynamic range / uv
coverage)
. Lousy calibrators at 13/20 cm
. LO distribution
. Probably pointing
. New observing modes (e.g. Mosaicing tricks)
. ???