PSRFITS - A Standard for Pulsar Data Storage

There are several compelling reasons for the pulsar community to move towards a standard data storage format. For example, standardisation of file formats would significantly decrease the amount of effort required to integrate new instruments into the analysis chain. The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Centre provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging analysis software.

In accordance with FITS standards, PSRFITS consist of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, observation start time etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data itself. A feature of the standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Unused HDUs need not be written, even though they are present in the definition. This feature allows, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. While primarily designed to accommodate folded or profile data, the definition can also be used for fast-sampled or search data. A novel feature of the definition is the inclusion of HDUs containing 'history' information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.

Further details about PSRFITS and a complete description of the open pulsar data analysis system PSRCHIVE are contained in Hotan, van Straten & Manchester (PASA, 21, 302-309, 2004). Major version 2 of the definition includes provision for use of TEMPO2 for predictions (See Hobbs, Edwards & Manchester, MNRAS, 369. 655-672, 2006; astro-ph/0603381).

The PSRFITS Definition

The Table below describes the basic features of the current definition. A history table keeps a record of the file creation and processing history. Two different digitiser HDUs are included; one stores the complete histogram of count-rate data and the other stores the summary statistics for instruments where the histogram data are unavailable. Entries in the Ephemeris history HDU are based on the pulsar timing program TEMPO2 -- they form the basis for predictions of the apparent pulsar period at the time of observation which are contained in the 'polyco' (TEMPO) or 'predictor' (TEMPO2) tables. As improved parameters become available, the data may be reprocessed, leading to new rows in the ephemeris and polyco/predictor history tables. Some tables, e.g., those related to calibration, are normally only filled during off-line processing. They need not be written at file creation time. Provision is made for breaking very large (search-mode) data sets into several separate PSRFITS files, split by either frequency or time.

PSRFITS - A summary of the current definition

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au