PSRFITS - A Standard for Pulsar Data Storage
There are compelling reasons for the pulsar community to utilise a standard data storage format. For example, standardisation of file formats significantly decreases the amount of effort required to integrate new instruments into the analysis chain. Importantly, they facilitate provision of consistent and complete file metadata, essential for archival material.
The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Center provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging software. Accordingly, we have defined PSRFITS as a standard FITS-based format for pulsar data files. Both mean pulse profile ("fold-mode") and streamed ("search-mode") multi-channel full-polarisation data are supported. PSRFITS is fully integrated with the open-source pulsar data-analysis system PSRCHIVE.
In accordance with FITS standards, PSRFITS consist of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, receiver parameters, observation start time, etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data formatted as a series of sub-integrations. A feature of standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Furthermore, unused HDUs need not be written, even though they are present in the definition. These features allow, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. A novel feature of the definition is the inclusion of HDUs containing processing history information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.
PSRFITS and PSRCHIVE are described in Hotan, van Straten & Manchester (PASA, 21, 302-309, 2004). The primary copy of the PSRFITS definition file is maintained under sourceforge as part of PSRCHIVE. Click on PSRFITS template for the current version.
The Table below describes the basic components of the template.
PSRFITS - A summary of the current definition (V5.4)
|Main header||Observer, telescope and receiver information, source name and observation date and time|
|HISTORY||Date, program and details of data acquisition and each subsequent processing step|
|OBSDESCR||Free-format ascii description of the observation or signal processing|
|PSRPARAM||Pulsar ephemeris used to create or modify pulse profile data|
|POLYCO||History of the TEMPO polyco files used to predict the apparent pulsar period|
|T2PREDICT||The TEMPO2 predictor file used to predict the apparent pulsar period|
|COHDDISP||Parameters used for coherent dedisperion of baseband data|
|BANDPASS||Observed bandpass in each polarisation averaged over observation|
|FLUX_CAL||System temperature and injected noise calibration data as a function of frequency across the bandpass|
|CAL_POLN||Apparent polarisation of injected noise calibration signal as a function of frequency|
|FEEDPAR||Parameters of feed cross-coupling as a function of frequency|
|SPECKURT||Statistics for spectral kurtosis RFI excision|
|SUBINT||Pulse profiles or streamed data as a function of time, frequency and polarisation|
|DIG_STAT||Digitiser mode, attenuator settings and count statistics|
|DIG_CNTS||Digitiser mode and count rate distribution|
The PSRFITS Documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.
PSRFITS Version History
PSRFITS Version History gives the date, author and summary of the main changes for each version of the PSRFITS template.
Email distribution list
If you would like to join the PSRFITS email distribution list, please sign up here.
For further information, please contact Dick.Manchester at csiro.au