Main

Psrfits

Main.Psrfits History

Hide minor edits - Show changes to markup

May 28, 2014, at 05:44 PM by RNM -
Changed lines 48-49 from:

Click on PSRFITS template for the current version or download it at psrheader.fits?.

to:

Click on PSRFITS template for the current version.

May 28, 2014, at 05:44 PM by RNM -
Changed lines 48-49 from:

Click on PSRFITS template for the current version or download it at <a href="psrheader.fits">psrheader.fits</a>.

to:

Click on PSRFITS template for the current version or download it at psrheader.fits?.

May 28, 2014, at 05:41 PM by RNM -
Changed lines 48-49 from:

Click on PSRFITS template for the current version. The Table below describes the basic components of the template.

to:

Click on PSRFITS template for the current version or download it at <a href="psrheader.fits">psrheader.fits</a>.

The Table below describes the basic components of the template.

May 28, 2014, at 05:37 PM by RNM -
Changed lines 14-16 from:

wide array of software is available for FITS file manipulation. The [[http://heasarc.gsfc.nasa.gov/docs/software/|NASA High Energy Astrophysics Science Archive Research Centre]] provides useful

to:

wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Center provides useful

May 28, 2014, at 05:33 PM by RNM -
May 14, 2014, at 02:48 PM by RNM -
Added line 58:
OBSDESCRFree-format ascii description of the observation or signal processing
Deleted line 59:
OBSDESCRFree-format ascii description of the observation or signal processing
May 12, 2014, at 04:33 PM by RNM -
Changed lines 52-53 from:

PSRFITS - A summary of the current definition (V5.3)

to:

PSRFITS - A summary of the current definition (V5.4)

Added line 59:
OBSDESCRFree-format ascii description of the observation or signal processing
November 29, 2013, at 05:38 PM by RNM -
Added lines 79-82:

Email distribution list

If you would like to join the PSRFITS email distribution list, please sign up here.

November 29, 2013, at 02:31 PM by RNM -
Changed lines 52-53 from:

PSRFITS - A summary of the current definition

to:

PSRFITS - A summary of the current definition (V5.3)

November 29, 2013, at 02:04 PM by 150.229.106.28 -
Added line 66:
SPECKURTStatistics for spectral kurtosis RFI excision
November 14, 2013, at 11:59 AM by RNM -
Changed lines 76-77 from:

PSRFITS Version History gives the date and a summary of the main changes for each version of the PSRFITS template.

to:

PSRFITS Version History gives the date, author and summary of the main changes for each version of the PSRFITS template.

November 14, 2013, at 11:58 AM by RNM -
Changed lines 74-75 from:

Version History for PSRFITS

to:

PSRFITS Version History

November 13, 2013, at 10:50 AM by RNM -
Added lines 74-77:

Version History for PSRFITS

PSRFITS Version History gives the date and a summary of the main changes for each version of the PSRFITS template.

October 21, 2013, at 03:43 PM by RNM -
Changed lines 50-52 from:

Click on [[http://www.atnf.csiro.au/research/pulsar/psrfits/fitsdef.html|PSRFITS template]] for the current version. The Table below describes the basic components of the template.

to:

Click on PSRFITS template for the current version. The Table below describes the basic components of the template.

Changed lines 72-73 from:

This documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.

to:

The PSRFITS Documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.

October 03, 2013, at 01:44 PM by rnm -
Changed lines 73-77 from:

This documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.

To be completed!

to:

This documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.

July 16, 2013, at 01:28 PM by RNM -
Changed lines 76-102 from:

Main Header

SIMPLE = T / file does conform to FITS standard
BITPIX = 8 / number of bits per data pixel
NAXIS = 0 / number of data axes
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format defined in Astronomy and
COMMENT Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, v73/p365.
COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.

HDRVER = '5.1 ' / Header version
FITSTYPE= 'PSRFITS ' / FITS definition for pulsar data files
DATE = ' ' / File creation date (YYYY-MM-DDThh:mm:ss)

Approximate date and UTC of file creation. aaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaa aaaaaaaaa

OBSERVER= ' ' / Observer name(s)

To be supplied.

to:

To be completed!

July 16, 2013, at 01:27 PM by RNM -
Changed lines 90-92 from:
Standard FITS preamble. Conformity with FITS standards can be
  checked using fits_verify.
to:
Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.
Changed lines 97-98 from:
Approximate date and UTC of file creation.
to:
Approximate date and UTC of file creation. aaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaa aaaaaaaaa
July 16, 2013, at 01:25 PM by RNM -
Added lines 73-101:

This documentation takes the PSRFITS template and expands on the brief comment on each line as appropriate.

Main Header

SIMPLE = T / file does conform to FITS standard
BITPIX = 8 / number of bits per data pixel
NAXIS = 0 / number of data axes
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format defined in Astronomy and
COMMENT Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, v73/p365.
COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be
  checked using fits_verify.

HDRVER = '5.1 ' / Header version
FITSTYPE= 'PSRFITS ' / FITS definition for pulsar data files
DATE = ' ' / File creation date (YYYY-MM-DDThh:mm:ss)

Approximate date and UTC of file creation.

OBSERVER= ' ' / Observer name(s)

July 16, 2013, at 01:22 PM by RNM -
Deleted lines 72-97:

Main Header

SIMPLE = T / file does conform to FITS standard
BITPIX = 8 / number of bits per data pixel
NAXIS = 0 / number of data axes
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format defined in Astronomy and
COMMENT Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, v73/p365.
COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.

HDRVER = '5.1 ' / Header version
FITSTYPE= 'PSRFITS ' / FITS definition for pulsar data files
DATE = ' ' / File creation date (YYYY-MM-DDThh:mm:ss)

Approximate date and UTC of file creation.

OBSERVER= ' ' / Observer name(s)

July 16, 2013, at 01:20 PM by RNM -
Changed lines 92-93 from:

(YYYY-MM-DDThh:mm:ss) \\

to:

(YYYY-MM-DDThh:mm:ss)

July 16, 2013, at 01:19 PM by RNM -
Changed lines 87-89 from:
Standard FITS preamble. Conformity with FITS standards can be
  checked using fits_verify.
to:
Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.
July 16, 2013, at 01:18 PM by RNM -
Added lines 73-98:

Main Header

SIMPLE = T / file does conform to FITS standard
BITPIX = 8 / number of bits per data pixel
NAXIS = 0 / number of data axes
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format defined in Astronomy and
COMMENT Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, v73/p365.
COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be
  checked using fits_verify.

HDRVER = '5.1 ' / Header version
FITSTYPE= 'PSRFITS ' / FITS definition for pulsar data files
DATE = ' ' / File creation date (YYYY-MM-DDThh:mm:ss)
-> Approximate date and UTC of file creation.

OBSERVER= ' ' / Observer name(s)

July 16, 2013, at 01:14 PM by RNM -
Added lines 73-74:

To be supplied.

July 16, 2013, at 01:13 PM by RNM -
Deleted lines 72-80:

COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.

HDRVER = '5.1 ' / Header version

July 16, 2013, at 01:12 PM by RNM -
Changed lines 78-81 from:
Standard FITS preamble. Conformity with FITS standards can be

checked using fits_verify.

to:
Standard FITS preamble. Conformity with FITS standards can be checked using fits_verify.
July 16, 2013, at 01:11 PM by RNM -
Added lines 73-83:

COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
COMMENT

Standard FITS preamble. Conformity with FITS standards can be

checked using fits_verify.

HDRVER = '5.1 ' / Header version

July 16, 2013, at 12:28 PM by RNM -
Changed lines 49-53 from:

file is maintained under sourceforge as part of PSRCHIVE. Click on [[http://www.atnf.csiro.au/research/pulsar/psrfits/fitsdef.html|PSRFITS template]] for the current version. The Table below describes the basic components of the template.

to:

file is maintained under sourceforge as part of PSRCHIVE. Click on [[http://www.atnf.csiro.au/research/pulsar/psrfits/fitsdef.html|PSRFITS template]] for the current version. The Table below describes the basic components of the template.

July 16, 2013, at 12:24 PM by 150.229.106.28 -
Changed lines 3-12 from:

There are several compelling reasons for the pulsar community to move towards a standard data storage format. For example, standardisation of file formats would significantly decrease the amount of effort required to integrate new instruments into the analysis chain. The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Centre provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging analysis software.

In accordance with FITS standards, PSRFITS consist of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, observation start time etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data itself. A feature of the standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Unused HDUs need not be written, even though they are present in the definition. This feature allows, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. While primarily designed to accommodate folded or profile data, the definition can also be used for fast-sampled or search data. A novel feature of the definition is the inclusion of HDUs containing 'history' information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.

Further details about PSRFITS and a complete description of the open pulsar data analysis system PSRCHIVE are contained in Hotan, van Straten & Manchester (PASA, 21, 302-309, 2004). Major version 2 of the definition includes provision for use of TEMPO2 for predictions (See Hobbs, Edwards & Manchester, MNRAS, 369. 655-672, 2006; astro-ph/0603381).

The PSRFITS Definition

The Table below describes the basic features of the current definition. A history table keeps a record of the file creation and processing history. Two different digitiser HDUs are included; one stores the complete histogram of count-rate data and the other stores the summary statistics for instruments where the histogram data are unavailable. Entries in the Ephemeris history HDU are based on the pulsar timing program TEMPO2 -- they form the basis for predictions of the apparent pulsar period at the time of observation which are contained in the 'polyco' (TEMPO) or 'predictor' (TEMPO2) tables. As improved parameters become available, the data may be reprocessed, leading to new rows in the ephemeris and polyco/predictor history tables. Some tables, e.g., those related to calibration, are normally only filled during off-line processing. They need not be written at file creation time. Provision is made for breaking very large (search-mode) data sets into several separate PSRFITS files, split by either frequency or time.

to:

There are compelling reasons for the pulsar community to utilise a standard data storage format. For example, standardisation of file formats significantly decreases the amount of effort required to integrate new instruments into the analysis chain. Importantly, they facilitate provision of consistent and complete file metadata, essential for archival material.

The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation. The [[http://heasarc.gsfc.nasa.gov/docs/software/|NASA High Energy Astrophysics Science Archive Research Centre]] provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging software. Accordingly, we have defined PSRFITS as a standard FITS-based format for pulsar data files. Both mean pulse profile ("fold-mode") and streamed ("search-mode") multi-channel full-polarisation data are supported. PSRFITS is fully integrated with the open-source pulsar data-analysis system PSRCHIVE.

In accordance with FITS standards, PSRFITS consist of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, receiver parameters, observation start time, etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data formatted as a series of sub-integrations. A feature of standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Furthermore, unused HDUs need not be written, even though they are present in the definition. These features allow, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. A novel feature of the definition is the inclusion of HDUs containing processing history information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.

PSRFITS and PSRCHIVE are described in Hotan, van Straten & Manchester (PASA, 21, 302-309, 2004). The primary copy of the PSRFITS definition file is maintained under sourceforge as part of PSRCHIVE. Click on [[http://www.atnf.csiro.au/research/pulsar/psrfits/fitsdef.html|PSRFITS template]] for the current version. The Table below describes the basic components of the template.

Changed line 60 from:
PSRPARAMPulsar ephemeris used to create or modify profile data
to:
PSRPARAMPulsar ephemeris used to create or modify pulse profile data
Changed lines 72-74 from:

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

to:

PSRFITS Documentation

For further information, please contact Dick.Manchester at csiro.au

December 02, 2010, at 03:51 PM by RNM -
Changed line 15 from:
to:
Changed line 31 from:

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.a

to:

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

December 02, 2010, at 03:49 PM by RNM -
Changed lines 13-15 from:

Table: PSRFITS - A summary of the current definition

to:

PSRFITS - A summary of the current definition

December 02, 2010, at 03:48 PM by RNM -
Changed line 16 from:
!HDU Name!Description
to:
HDU NameDescription
Changed line 31 from:

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

to:

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.a

December 02, 2010, at 03:47 PM by RNM -
Changed lines 1-2 from:

PSRFITS - A Standard for Pulsar Data Storage

to:

PSRFITS - A Standard for Pulsar Data Storage

Changed lines 9-10 from:

The PSRFITS Definition

to:

The PSRFITS Definition

Changed lines 13-21 from:

Table: PSRFITS - A summary of the current definition

HDU NameDescription
Main headerObserver, telescope and receiver information, source name and observation date and time
HISTORYDate, program and details of data acquisition and each subsequent processing step
PSRPARAMPulsar ephemeris used to create or modify profile data
POLYCOHistory of the TEMPO polyco files used to predict the apparent pulsar period
T2PREDICTThe TEMPO2 predictor file used to predict the apparent pulsar period
COHDDISPParameters used for coherent dedisperion of baseband data
to:

Table: PSRFITS - A summary of the current definition

!HDU Name!Description
Main headerObserver, telescope and receiver information, source name and observation date and time
HISTORYDate, program and details of data acquisition and each subsequent processing step
PSRPARAMPulsar ephemeris used to create or modify profile data
POLYCOHistory of the TEMPO polyco files used to predict the apparent pulsar period
T2PREDICTThe TEMPO2 predictor file used to predict the apparent pulsar period
COHDDISPParameters used for coherent dedisperion of baseband data
Changed lines 24-31 from:
FLUX_CALSystem temperature and injected noise calibration data as a function of frequency across the bandpass
CAL_POLNApparent polarisation of injected noise calibration signal as a function of frequency
FEEDPARParameters of feed cross-coupling as a function of frequency
SUBINTPulse profiles or streamed data as a function of time, frequency and polarisation
DIG_STATDigitiser mode, attenuator settings and count statistics
DIG_CNTSDigitiser mode and count rate distribution

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

to:
FLUX_CALSystem temperature and injected noise calibration data as a function of frequency across the bandpass
CAL_POLNApparent polarisation of injected noise calibration signal as a function of frequency
FEEDPARParameters of feed cross-coupling as a function of frequency
SUBINTPulse profiles or streamed data as a function of time, frequency and polarisation
DIG_STATDigitiser mode, attenuator settings and count statistics
DIG_CNTSDigitiser mode and count rate distribution

The PSRFITS definition file is maintained under PSRCHIVE. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

December 02, 2010, at 03:41 PM by RNM -
Changed line 15 from:
HDU NameDescription
to:
HDU NameDescription
Changed line 30 from:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

to:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au

December 02, 2010, at 03:39 PM by RNM -
Changed line 15 from:
!HDU Name!Description
to:
HDU NameDescription
Changed line 30 from:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

to:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

December 02, 2010, at 03:34 PM by RNM -
December 02, 2010, at 03:32 PM by RNM -
Changed lines 3-4 from:

There are several compelling reasons for the pulsar community to move towards a standard data storage format. For example, standardisation of file formats would significantly decrease the amount of effort required to integrate new instruments into the analysis chain. The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Centre provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging analysis software.

to:

There are several compelling reasons for the pulsar community to move towards a standard data storage format. For example, standardisation of file formats would significantly decrease the amount of effort required to integrate new instruments into the analysis chain. The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Centre provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging analysis software.

Changed line 30 from:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

to:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

December 02, 2010, at 03:23 PM by RNM -
Added line 8:
Changed lines 13-28 from:

Table: PSRFITS - A summary of the current definition HDU Title Description Main header Observer, telescope and receiver information, source name and observation date and time Processing history Date, program and details of data acquisition and each subsequent processing step Original bandpass Observed bandpass in each polarisation averaged over observation Coherent dedispersion Parameters for coherent dedisperion of baseband data Ephemeris history Pulsar parameters used to create or modify profile data Polyco history Elements of the TEMPO polyco file used to predict the apparent pulsar period TEMPO2 predictor history The TEMPO2 predictor file used to predict the apparent pulsar period Flux calibration System temperature and injected noise calibration data as a function of frequency across the bandpass Injected calibration polarisation Apparent polarisation of injected noise calibration signal as a function of frequency Feed cross-coupling Parameters of feed cross-coupling as a function of frequency Integration data Pulse profiles or fast-sampled data as a function of time, frequency and polarisation Digitiser statistics Digitiser mode and count statistics Digitiser counts Digitiser mode and count rate distribution

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

to:

Table: PSRFITS - A summary of the current definition

!HDU Name!Description
Main headerObserver, telescope and receiver information, source name and observation date and time
HISTORYDate, program and details of data acquisition and each subsequent processing step
PSRPARAMPulsar ephemeris used to create or modify profile data
POLYCOHistory of the TEMPO polyco files used to predict the apparent pulsar period
T2PREDICTThe TEMPO2 predictor file used to predict the apparent pulsar period
COHDDISPParameters used for coherent dedisperion of baseband data
BANDPASSObserved bandpass in each polarisation averaged over observation
FLUX_CALSystem temperature and injected noise calibration data as a function of frequency across the bandpass
CAL_POLNApparent polarisation of injected noise calibration signal as a function of frequency
FEEDPARParameters of feed cross-coupling as a function of frequency
SUBINTPulse profiles or streamed data as a function of time, frequency and polarisation
DIG_STATDigitiser mode, attenuator settings and count statistics
DIG_CNTSDigitiser mode and count rate distribution

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

December 02, 2010, at 02:53 PM by RNM -
Changed line 27 from:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

to:

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.

April 28, 2010, at 10:44 AM by ankur -
Added lines 1-27:

PSRFITS - A Standard for Pulsar Data Storage

There are several compelling reasons for the pulsar community to move towards a standard data storage format. For example, standardisation of file formats would significantly decrease the amount of effort required to integrate new instruments into the analysis chain. The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme. A wide array of software is available for FITS file manipulation. The NASA High Energy Astrophysics Science Archive Research Centre provides useful libraries and applications for manipulation and interrogation of FITS-based files. For example, the program fv greatly assists the process of testing and debugging analysis software.

In accordance with FITS standards, PSRFITS consist of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, observation start time etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data itself. A feature of the standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Unused HDUs need not be written, even though they are present in the definition. This feature allows, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. While primarily designed to accommodate folded or profile data, the definition can also be used for fast-sampled or search data. A novel feature of the definition is the inclusion of HDUs containing 'history' information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.

Further details about PSRFITS and a complete description of the open pulsar data analysis system PSRCHIVE are contained in Hotan, van Straten & Manchester (PASA, 21, 302-309, 2004). Major version 2 of the definition includes provision for use of TEMPO2 for predictions (See Hobbs, Edwards & Manchester, MNRAS, 369. 655-672, 2006; astro-ph/0603381). The PSRFITS Definition

The Table below describes the basic features of the current definition. A history table keeps a record of the file creation and processing history. Two different digitiser HDUs are included; one stores the complete histogram of count-rate data and the other stores the summary statistics for instruments where the histogram data are unavailable. Entries in the Ephemeris history HDU are based on the pulsar timing program TEMPO2 -- they form the basis for predictions of the apparent pulsar period at the time of observation which are contained in the 'polyco' (TEMPO) or 'predictor' (TEMPO2) tables. As improved parameters become available, the data may be reprocessed, leading to new rows in the ephemeris and polyco/predictor history tables. Some tables, e.g., those related to calibration, are normally only filled during off-line processing. They need not be written at file creation time. Provision is made for breaking very large (search-mode) data sets into several separate PSRFITS files, split by either frequency or time.

Table: PSRFITS - A summary of the current definition HDU Title Description Main header Observer, telescope and receiver information, source name and observation date and time Processing history Date, program and details of data acquisition and each subsequent processing step Original bandpass Observed bandpass in each polarisation averaged over observation Coherent dedispersion Parameters for coherent dedisperion of baseband data Ephemeris history Pulsar parameters used to create or modify profile data Polyco history Elements of the TEMPO polyco file used to predict the apparent pulsar period TEMPO2 predictor history The TEMPO2 predictor file used to predict the apparent pulsar period Flux calibration System temperature and injected noise calibration data as a function of frequency across the bandpass Injected calibration polarisation Apparent polarisation of injected noise calibration signal as a function of frequency Feed cross-coupling Parameters of feed cross-coupling as a function of frequency Integration data Pulse profiles or fast-sampled data as a function of time, frequency and polarisation Digitiser statistics Digitiser mode and count statistics Digitiser counts Digitiser mode and count rate distribution

The PSRFITS definition file is maintained under CVS control at the ATNF. The current version of the complete definition may be viewed here. For further information, please contact Dick.Manchester at csiro.au.