19th of March 2024 |
---|
SDHDF: A new file format for spectral-domain radio astronomy data |
by Toomey et al. |
Radio astronomy file formats are now required to store wide frequency
bandwidths and multiple simultaneous receiver beams and must be able
to account for versatile observing modes and numerous calibration
strategies. The need to capture and archive high-time and high
frequency-resolution data, along with the comprehensive metadata that
fully describe the data, implies that a new flexible data format and new
processing software are required.
Toomey et al.
present the Spectral-Domain Hierarchical Data Format
(SDHDF) — a new file format for radio astronomy data, in particular
for single dish or beam-formed data streams. Since 2018, SDHDF has
been the primary format for data products from the spectral-line and
continuum observing modes at Murriyang, the CSIRO Parkes 64-metre
radio telescope, and it is demonstrated that this data format can also be
used to store observations of pulsars and fast radio bursts.
The figure above is a visual representation of an example data file, showing the hierarchical association of HDF groups and datasets. An SDHDF data file (the solid green rectangle) may contain optionally none or any number of "root_ID" objects each of which is of the SDHDF_CLASS sdhdf_file and each adhering to the SDHDF definition, where each root_ID object could be a separate observation and grouped together (for example one may wish to store multiple single scans in the same file). Information about the root_ID objects are stored in a "root_parameters" dataset; this dataset and the parent metadata group are not required if the file does not contain a root_ID object. In this figure, the solid orange and solid blue rectangles represent HDF group and dataset objects respectively. |