Report from the Working Group on Astronomical Data
Ray Norris,
Overview
The management of astronomical data has become an increasingly significant issue in recent years, because of large surveys now routinely producing Terabytes of data, and because of the growing importance of www-based data centres, remote observing, data archiving, etc. There have been two dominant issues since the last GA: the Virtual Observatory, and the protection of freedom of access to large databases. The Virtual Observatory work will be describe in detail by the VO working group, but some implications have been handled by the WGAD, which will be described here. A major issue for the IAU is the proposed database protection legislation which is opposed by CODATA on behalf of the ICSU.
CODATA
CODATA is the committee on data in science and technology of the ICSU (International Council for Science), and it serves the role of coordinating data-related activities between the various scientific unions such as IAU and URSI. Given the increasing significance of data in astronomy and the threats to our freedom to use it, IAU involvement in CODATA is very important.
Ray Norris represented the IAU at the October 2000 meeting in
Database Protection Legislation
There is a worldwide shift towards increased protection for intellectual property. In particular, the World Intellectual Property Organisation (WIPO) has proposed legislation with the very reasonable aim of protecting commercial databases, but with unwanted side-effects which will cause problems for open access to scientific data (see http://www.codata.org/data_access/summary.html for a summary of issues). Various groups, particularly in the
Astronomical Data Archive Issues
The growth of databases from large surveys, and the advent of the virtual observatory, has raised a number of issues around astronomical data. One of these is the need to ensure future freedom of access to astronomical archive data from major observatories, and a resolution to this effect will be tabled at the next
Data Formats
Until about 20 years ago astronomy was split into a number of sub-disciplines corresponding to the different wavelengths at which observations were made (optical astronomy, radio astronomy, etc). Each of these tended to use its own data formats, as a result of which it was awkward to combine data from different wavelengths. In 1981 the FITS (Flexible Interchange Transport System) format was proposed (http://fits.gsfc.nasa.gov/documents.html), and adopted enthusiastically by all sub-disciplines of astronomy. This permitted easy interchange of data between the sub-disciplines, and was largely responsible for breaking down cultural barriers between these sub-disciplines. As a result, many astrophysicists today take their data at whichever wavelength is needed to solve the astrophysical problem being addressed. This is probably partly responsible for the current healthy state of astronomy and astrophysics internationally, with new discoveries about the origin and evolution of the Universe being made at a breathtaking rate.
While FITS has been enormously and demonstrably successful, astronomical data are now seen by some as outgrowing the 1980s technology on which FITS was based. The
An even more sophisticated approach has been taken by organisations in US and