Description: gridzilla: "Simple weight" switch ignored.
Type of affected data: Data cubes generated with gridzilla (the GUI interface to pksgridzilla).
Date of affected data: From version 1.0 of gridzilla and on-going.
Problem in detail: Gridzilla presents an "on/off" switch labelled "Simple weighting"; This switch is no longer relevant to the underlying gridder (pksgridzilla), and is consequently ignored. Altering the state of this switch does not change the way the data is gridded, and the resultant behaviour is the default, which is to apply "order=1" weighting. Note that this switch is only able to be modified when gridzilla is used in "general" mode, and so if you have not used this mode, this problem does not affect you or your data.
Solution: You cannot correct an already-generated data cube. At this stage, and until further notice, if you wish to generate data cubes using different weighting schemes, you must generate the glish gridding script yourself, and put the option "weight_order = 0" (or 1 or 2).
Signed: David Barnes (04-Nov-1998)
Description: acquisition: RPFITS date errors.
Type of affected data: All data.
Date of affected data: 22-Aug-1998 to 02-Nov-1998.
Problem in detail: As part of an upgrade to the RPFITS writer to 2000-compliance, a bug was introduced which caused errors in timestamps within RPFITS files. Specifically, the observing date was clocked on to the next day at 0h local time, rather than 0h UT. For survey data, the only important consequence of this is that velocity tracking solutions will be slightly in error, typically on the order of 0.5 km/s. In principle this deviation could be corrected during gridding, but for the HIPASS and ZoA, this is not likely to be worthwhile, given that they each have 13 km/s channels, and 16 and 26 km/s resolution respectively. It is also possible that the apparent 24hr time jump in the files may cause bandpass correction failures for one scan per day during the period that the bug remained unnoticed. Note that the RPFITS filenames remained correct throughout this period.
Solution: There is no software solution provided at present, but the RPFITS library may in the future be fixed to handle the affected data files correctly. As far as Multibeam data reduced on-line using LiveData, there is no fix available at present, and as noted above, is not required for the HIPASS or ZOA surveys. Narrowband Multibeam observations taken during this time will need to be fixed.
Signed: David Barnes (04-Nov-1998)
Description: livedata: Bandpass calculation error.
Type of affected data: LiveData-processed scan files having more than 100 integration cycles.
Date of affected data: 27-Feb-1997 onwards.
Problem in detail: An error in calculating which spectra to use to generate a bandpass estimate has been detected by Mark Calabretta. The error may apply to calibrated scan files (ie. .mscal and .sdfits extensions) having more than 100 integration cycles of data, but does not apply to shorter scan files. The error would manifest itself as a visually detectable worsening in the bandpass-corrected spectra beyond the 100th integration cycle in a calibrated dataset (ie. a .mscal or .sdfits file).
Solution: The only possible fix for affected scan files is to reprocess the data through the latest version of LiveData. We have decided not to do this for the HIPASS and ZoA surveys, since survey scan files rarely contain more than 100 integrations, and integrations beyond the hundredth cycle lie in the overlap band with a lower or higher declination band for HIPASS, or a more easterly or westerly band for the ZoA survey.
Signed: David Barnes (04-Nov-1998)
Description: livedata: Bandpass calculation error.
Type of affected data: LiveData-processed scan files with option fast=F(alse).
Date of affected data: 27-Feb-1997 onwards.
Problem in detail: A logical error in the bandpass calculation routine has meant that the "fast" option given to pksbandpass (the binary bandpass calculation client) was ignored, and assumed to be True. Thus any data reduced with "fast=F" may not be quite the data you think it is! When "fast" is True, the bandpass calculator is only required to check for field name consistency and for the absence of time and position jumps for the central beam of the Multibeam receiver. When False, all beams are checked individually. There are no known observing projects which used a scanning mode for which this switch is important.
Solution: There is no software solution. The bug has been fixed in the latest version of Livedata, post version 4.2.
Signed: David Barnes (07-Dec-1998)
Description: livedata: Bandpass calculation error.
Type of affected data: LiveData-processed scan files having more than 52 integration cycles.
Date of affected data: 27-Feb-1997 onwards.
Problem in detail: An error in calculating which spectra to use to generate a bandpass estimate has been detected by Mark Calabretta. The error applies to calibrated scan files (ie. .mscal and .sdfits extensions) having more than 52 integration cycles of data (for the survey scan modes). Technically, for bandpass correction with "x" precycles and "y" postcycles and a bandpass interval of "j", the cycles affected are (n*(x+y)+j) to (n*(x+y)+j+x), with n = 1, 2, 3, ... In these cycle ranges, the bandpass estimate is incorrectly calculated, neglecting various valid precycles. At worst, the bandpass correction is as good as that at the start of a scan. The error manifests itself as approximately a 5 to 10 per cent deterioration in the SNR of calibrated spectra in these cycle ranges.
Solution: The only possible fix for affected scan files is to reprocess the data through the latest version of LiveData. We have decided not to do this for the HIPASS and ZoA surveys, since the effect has not been noticed previously, and in any case is very small. The integrity of the data is not affected, only the noise level.
Signed: David Barnes (12-Dec-1998)
Description: acquisition: bandwidth error.
Type of affected data: All affected HIPASS data has been flagged.
Date of affected data: 26-28 Dec-1998.
Problem in detail: After the Dec 98 narrow-band observing session (starting Dec 13-17 1998), there were a few weeks where some data was accidentally labelled with a bandwidth of 8 MHz (although the data was taken at 64 MHz bandwidth). Because of the mislabelled bandwidth, the LO calculation was done wrongly (although the LO was reset manually to the correct frequency by the observers at one stage). The problem arose because the HIPASS/ZOA sched files did not contain a BANDWIDTH parameter. It is believed that a rogue tkmulti save file (perhaps in a sub-directory) with a saved bandwidth parameter of 8 MHz caused the problem.
Solution: Data where the LO was not manually reset is not recoverable. All affected data (even where it is thought the LO was OK) has been flagged by I. Stewart. This data is between 26 and 28 Dec 1998. The ZOA sched files and the HIPASS -82 to -2 deg sched files have now been upgraded to include BANDIDTH=64.
Signed: Lister Staveley-Smith (29-Jun-1999)
Description: acquisition: RPFITS Tsys error.
Type of affected data: HIPASS/ZOA HPF files.
Date of affected data: 26 to 29-Mar-1999.
Problem in detail: Some data was taken where the T(sys) values AND the corresponding 13-beam spectral data were too low by the cal noise diode conversion factor (about 1.8). All data known to be affected were deleted before being archived. The source of the problem was multi not updating these factors following a correlator reconfiguration.
Solution: Although no known data is affected, the solution is to multiply T(sys) and Data values by the cal noise diode table which is:
# Beam, CAL(A), CAL(B) 1 1.51 1.73 2 1.68 1.86 3 1.60 1.75 4 1.43 1.86 5 1.89 2.10 6 1.65 1.93 7 1.72 2.07 8 1.81 2.17 9 1.71 2.11 10 1.82 2.05 11 1.86 2.18 12 1.92 2.29 13 1.91 2.03
Signed: Lister Staveley-Smith (29-Jun-1999)
Description: acquisition: RPFITS syscal error.
Type of affected data: SYSCAL efficiency tables in HIPASS/ZOA RPFITS files.
Date of affected data: UT 1100 20-May-1999 to 0700 24-May-1999.
Problem in detail: One of the SYSCAL arrays in the HIPASS/ZOA HPF/RPFITS files was set to unity following the transfer from multi/tkmulti to TCS. Normally, this array contains the Kelvin to Jansky conversion.
Solution: This array is not yet used by any known software.
Signed: Warwick Wilson, Lister Staveley-Smith (29-Jun-1999)
Description: FONT SIZE=+1>acquisition: Position errors.
Type of affected data: HIPASS/ZOA data files.
Date of affected data: UT 1100 20-May-1999 to 29-June-1999.
Problem in detail: A corrupt parameter file caused the multibeam receiver zero-point angle to be read as 0 deg rather than 3.75 deg. This means that although the multibeam receiver was at the correct requested position and rotation angle, the positions written to the HPF files were WRONG by 1.9 arcmin for beams 2-7 and 3.3 arcmin for beams 8-13.
Solution: The position correction is straightforward in principle, but has not been implemented. Note that calibrations will not be affected, since the receiver was always at the correct position.
Signed: Mike Kesteven, Lister Staveley-Smith (29-Jun-1999)
Description: acquisition: Frequency errors.
Type of affected data: P248(HIPASS/ZOA)/P290/P307 data files.
Date of affected data: UT 03:33-07:54 31 May 1999 (P290), UT 02:38-12:19 18 June 1999 (P307), UT 11:08-22:32 25 July 1999 (P248).
Problem in detail: Following a "calibration scan", a bug in TCS caused Doppler tracking to switch from "topocentric" to LSR mode, causing an unintended frequency offset in all data. TCS log files contain the exact frequency offsets.
Solution: HIPASS/ZOA data will be flagged as the velocity-channel relationship is compromised. However, other data should be recoverable as long as gridding is done in the velocity or frequency domain, rather than the "channel number" domain.
Signed: John Reynolds, Lister Staveley-Smith (26-Aug-1999)
Description: acquisition: RPFITS syscal error.
Type of affected data: 14th SYSCAL record in HIPASS/ZOA rpfits files.
Date of affected data: UT 00:00 03-Dec-1999 to UT 02:40 21-Jan-2000.
Problem in detail: Following a correlator software upgrade on 03-Dec-1999, all Multibeam/HIPASS RPFITS files omitted the additional (14th) SYSCAL record. This record normally contains the telescope parallactic angle, azimuth and elevation translator coordinates, reference beam number, and the datestamp of the active calibration file. These quantities were all left zero or blank for the above period.
Solution: The defect has been corrected, and does not affect standard Multibeam/HIPASS processing. The parallactic angle can be recomputed from the observation data, time and sky direction if necessary.
Signed: Warwick Wilson, Lister Staveley-Smith (01-Feb-2000)
Description: livedata: HI line rest frequency used to compute Doppler correction.
Type of affected data: All observations processed by livedata before pksbandpass revision 15.6 (UTC 2001/03/09); all HIPASS/ZOA observations processed after this and any others which use the HIPASS/ZOA method of Doppler correction.
Date of affected data: 1997/02/27 onwards.
Problem in detail: The HIPASS/ZOA method of Doppler correction uses FFT interpolation to shift the spectra by an appropriate number of channels. In doing so it introduces three approximations, one of which may be significant.
The Doppler relativistic formula is
f' = f * D
where f is the observed (e.g. topocentric) frequency, f' is the frequency in
the new frame of reference (e.g. LSRK), and
D = sqrt((1-beta)/(1+beta)).
The velocity (beta = v/c) is positive for motion towards the new frame of
reference in which case f' < f. These formulae show that the Doppler effect
simply scales the frequency by a constant factor. Note that the frequency
increment (i.e. channel spacing) is scaled by the same amount
df' = df * D.
The HIPASS/ZOA Doppler tracking method is an approximation because it does not take into account the change in frequency increment. That is, the Doppler frequency shift is actually proportional to the observed frequency
f' - f = (D - 1) * f.Thus the first approximation was to ignore this frequency dependence. The second approximation was to use the non-relativistic velocity formula
D ~= 1 - beta,
whence
f' - f ~= -beta * f.In fact, this is quite accurate for the LSRK correction but nevertheless was unnecessary. However, the third approximation (really an error) was that the frequency shift was computed from the HI line rest frequency (f0 = 1420.40575 MHz),
shift = -beta * f0.
Since f0 is towards one end of the 1024 channel HIPASS/ZOA spectrum (in
channel 97), using it rather than the central reference frequency (1394.5 MHz
in channel 513) accentuates the error at the far end of the spectrum (i.e. in
channel 1024). Now the error in frequency is
E(f) = -beta(f0 - f)
so that the error in channel number is
E(C) = -beta(C0 - C).
Thus the error in channel number is proportional to the channel separation.
For the earth's orbital velocity of 30 km/s the Doppler factor may vary
between 0.9999 and 1.0001. Thus for C0 = 97 the error in channel 1 is small
while in channel 1024 it varies between -0.1 and 0.1 channel.
While the error in channel 1024 is acceptable for HIPASS/ZOA spectra it could be roughly halved by using the central reference frequency to compute the Doppler shift since this would balance the error between the two ends of the spectrum.
However, in the general case, particularly for observations in other frequency bands, use of the HI line rest frequency in computing the Doppler shift may lead to a much larger error. For example, for high frequency Mopra observations the frequency shift would be so far underestimated as effectively to be equivalent to not applying the Doppler correction at all.
Solution: The correct way to account for the Doppler effect is to rescale the frequency axis - both the reference frequency and frequency increment must be scaled by the Doppler factor, D.
The reason for shifting the spectrum rather than rescaling the frequency axis was to facilitate combining observations taken at different times - provided that the observations were made with the same LO settings (which includes disabling Doppler tracking in TCS) then they would all have the same frequency axis parameters. However, in this case the frequency increment in the Doppler shifted spectra is not strictly correct.
It was considered first and foremost that the data output by livedata should be correctly labelled. The solution derived thus consists of two parts
The bandpass calibration client and GUI were modified to provide the new Doppler tracking scheme but with the old method as an option which HIPASS/ZOA observations will continue to use for the sake of consistency.
gridzilla was also modified to allow small differences in the frequency increment between input spectra sufficient to allow combination of 1024 channel spectra with the worse case Doppler correction.
Signed: Mark Calabretta (12-Mar-2001)
Description: acquisition: Sky coordinates for beams 8-13 are wildly erroneous
Type of affected data: HIPASS/ZOA, P349.
Date of affected data: 25 Febuary 2001 - 26 April 2001 inclusive
Problem in detail: Livedata display for beams 8-13 show positions which are about 50 degrees away from the actual on-source position.
Solution: An incorrect (7-beam) "feedhorn.dat" file had been defined for the new TCS version, which resulted in junk RA/Dec in the headers for beams 8-13 (rest of the data OK).
The (only) purpose of having a 7-beam feedhorn.dat file is to ensure that TCS calibrations with 7-beam configs terminate after just 7 beams, but it is clear that the inconvenience of having to manually stop the calibration after beam 7 in such cases is negligible compared with the risk of corrupting the data by leaving the 7-beam file in place. We will have to devise another means of making the calibration procedure halt after the correct number of beams.
Affected data for HIPASS/ZOA is contained on CDs 1152-1180, whilst patched data files have been reprocessed with Livedata and archived onto CDs 1181-1209. In order for programs like coverage and scanlog to skip over CDs 1152-1180, references to these CDs have been removed from the cd_db database file.
Signed: John Reynolds, Stacy Mader (24-May-2001)
This page is maintained by
David Barnes
Last Modified: