RPFITS - ATNF data-recording format

RPFITS is the data-recording format used since the mid-1980s within the Australia Telescope National Facility (ATNF).

RPFITS is used to record synthesis visibility data obtained from the Australia Telescope Compact Array (ATCA) at Narrabri, NSW. It is also used for single-dish spectral line data obtained from Parkes and Mopra, including Parkes multibeam data.

While RPFITS superficially resembles FITS it differs in the following important respects:

• The block size is 2560 bytes.
• All data (other than ASCII header information) are stored in VAX single-precision floating point format.
• Angles in header keyvalues are expressed in radians rather than degrees.
• Several "scans" can occur in one file, each scan starting with an RPFITS header. However, none of the headers follows the standard for FITS extensions. They start with 'SIMPLE =' rather than 'XTENSION='. Moreover, while so-called "RPFITS tables" are mostly implemented using a private convention as a contiguous set of keyrecords at the end of the header, the flag (FG) table is written after the data in a header that begins with the string 'TABLE FG'.

These differences make RPFITS incompatible with standard FITS software such as FITSIO and FTOOLS and, in particular, precludes the use of fv, a powerful GUI-based FITS browser/editor. However, a translator, rp2sdfits, is available that converts RPFITS single-dish data to SDFITS format which is based on standard FITS binary tables. rp2sdfits is available as part of the livedata/gridzilla package.

The Fortran library distributed here contains routines for reading and writing RPFITS files. A header file, RPFITS.h, is provided to facilitate usage by C and C++ applications. Also included is rpfhdr, a utility for viewing RPFITS headers (it also works for standard FITS), and rpfex for extracting selected scans from an RPFITS file.

In addition to the source code distribution kit, the following RPFITS binary distributions are available:

• Libraries and binary utilities for x86 linux systems compiled on an i686 machine running Debian 4.0 "etch" using g77/gcc 3.4.6.

• Libraries and binary utilities for x86_64 (64-bit) linux systems compiled on a machine running Debian 4.0 "etch" using g77/gcc 3.4.6.

• Libraries and binary utilities for Sun Sparc systems compiled on a sun4u machine running SunOS 5.6 (Solaris 2.6) using Sun WorkShop 6 update 2 Fortran and C.

• Libraries and binary utilities for Apple Macintosh i386 systems compiled on a machine running Darwin 8.9.1 using the HPC distribution of g77 3.4.0 and gcc 4.0.1.

Note that each binary distribution kit contains a GNU makefile which supplies architecture-specific definitions for the template GNU makefile supplied with the source code kit.

MacOSX (PowerPC) binaries are no longer supplied as of version 2.18.
DEC alpha binaries are no longer supplied as of version 2.5.
SunOS 4.1.3 binaries are no longer supplied as of version 2.5.
SGI binaries are no longer supplied as of version 2.2.

Dates in the version 2 RPFITS header are in the new standard year-2000-compatible format which is not readable by the version 1 RPFITS reader. Version 1 dates are, however, backwards compatible with the version 2 reader.

Data has been written at the Australia Telescope Compact Array (ATCA) in the new format since 1998/10/06. Similarly for the Parkes telescope.

Version 2.19 (2008/10/27): Ensure that non-negative source RAs are written by RPFITSOUT and read by RPFITSIN. Such were written for ATCA mosaic data with pointing centres straddling RA = 0h, causing problems for Miriad task GPSCAL.

Version 2.18 (2008/09/11): Fortran STOP statements were removed from the "table"-reading routines used by RPFITSIN and replaced with an error return mechanism. The STOP statements caused livedata to crash on a corrupted input file.

Error message handling in RPFITSIN was generalized to provide control over what LUN to write to (default 6), or to disable output altogether. In either case the message is passed out via COMMON, refer to the prologue comments in rpferr.f. Application programs should now have much better control over message logging.

Version 2.17 (2007/07/16): When assigning a Fortran logical unit number, check whether it may already be in use by non-RPFITS code.

Version 2.16 (2007/01/30): RPFITS.h was modified for better conformance with C and C++ usage. There were no changes to the RPFITS library itself (being written in Fortran it does not use RPFITS.h).

In particular the declaration of subroutine arguments as global variables has been removed to avoid multiple definitions if RPFITS.h is included in two or more library modules. The MaxNChan and MaxNPol enums that were used to dimension the vis[] and weight[] arrays have also been removed - C and C++ code should allocate memory as required based on parameters read from the RPFITS header. These two changes will probably necessitate changes to C or C++ application code that uses RPFITS.h. A function prototype has also been added for RPFITSOUT together with a number of lesser changes.

According to the online manpage for freopen (cf. the COMPATIBILITY section), Darwin supports large files natively; it does not have freopen64 or fseeko64 but freopen and fseeko should suffice to read files >2GiB. rpfhdr and rpfex were modified accordingly and binaries are now supplied.

Version 2.15 (2006/10/26): replaced usage of the non-standard TIME and GMTIME Fortran intrinsics (which are not supported by gfortran) with a wrapper on the corresponding unix system C-library functions, time(2) and gmtime(3).

The "darwin" architecture has now been split into "darwin_ppc" and "darwin_x86".

Version 2.14 (2006/08/01): rpfhdr and rpfex now support large files. Also added comments to the linux GNUmakefile indicating options to use for the gfortran compiler (no changes to the RPFITS library itself).

Version 2.13 (2006/07/11): Renamed the INDEX COMMON to IOSTAT to avoid a conflict with the intrinsic function of the same name; gfortran objected to this violation of the Fortran 77 standard. The distribution kit now also includes rpfex, a utility for extracting selected scans from an RPFITS file.

Version 2.13.1 (2006/07/17): Added support for building on Mac OS X systems ("darwin" architecture).

Version 2.12 (2006/07/07): The distribution kit now includes the rpfhdr utility (no changes to the RPFITS library itself).

Version 2.11 (2006/07/06): RPFITSIN now extracts the BUNIT and OBSTYPE keyvalues into new CHARACTER*16 variables of the same name in the NAMES COMMON, and RPFITSOUT now sets the BUNIT keyrecord to the value supplied via this COMMON.

Version 2.10 (2006/07/04): Changed the CARD mechanism in RPFITSIN so that only the first eight characters are tested and all blanks are significant (i.e. the "FITS" keyword). Previously 12 characters were tested with trailing blanks ignored, and the whole header keyrecord then replaced the particular CARD variable. However, since these 12 characters might have included a portion of the keyvalue the test was thereby changed.

Version 2.9 (2006/06/20): Increased the maximum number of IFs supported from 8 to 16 for the MOPS correlator.

Version 2.8 (2004/08/25): RPFITSIN checks for corrupted "data format" values stored as the eleventh random parameter.

Version 2.7 (2004/07/01) accounts for DATE-OBS in the form DD/MM/YYYY. This is only known to have happened once, at the ATCA on 18/11/1998 when the date format was changed from DD/MM/YY to YYYY-MM-DD. Also fixed a bug in decoding datsys from the DATE-OBS header keyrecord.

Version 2.6 (2004/06/10) increases the input file name length in RPFITSIN from 80 to 256 characters. It also includes RPFITS.h which defines global enums and structs for RPFITS usage in C and C++ code.

Version 2.5 (2003/11/03) fills SU_PRA(K) and SU_PDEC(K) in the DOUBLES COMMON.

Version 2.4 (2001/06/04) fixes the floating point conversion problem on linux systems similar to that of version 2.2 but arising in a different context. It sporadically affected the random parameters, in particular the MJD (UTC) was occasionally found to be in error by 0.5 day. The problem affected both RPFITSOUT and RPFITSIN.

Version 2.3 (2001/05/31) fixes a problem with logical unit number handling that restricted the number of files that could be processed to 100. This affected livedata which runs continuously at Parkes processing hundreds of RPFITS files in succession. However, it is unlikely to be of general concern.

Version 2.2 (2001/02/09) differs from 2.1 mainly in fixing a problem with RPFITSOUT on linux systems. The conversion from the native IEEE little-endian floating point to output VAX floating was found to introduce errors of order 2^-17 in certain circumstances. Specifically, this occurred for values where the VAX floating representation happened to correspond to a "signalling NaN" when interpreted as an IEEE value. The signalling NaN was implicitly converted to a "quiet NaN" through floating point assignment by flipping one bit. The code was rewritten to avoid such assignments.

The problem manifested itself when integration times of 9.5s or 10.5s (instead of 10.0s) were computed as a result of subtracting two moderately large, closely separated UTs.

Version 2.1 (2000/06/14) is the first versioned release of the RPFITS library.

RPFITS is maintained by Mark Calabretta
Last Modified: 2008/10/27