| 15th of May 2024 |
|---|
|
| ATNF Colloquium |
| From Supervised to Unsupervised ML: lessons learned from learning machines |
| Kai Polsterer (HITS gGmbH, Heidelberg) |
|
Abstract:
The amount, size, and complexity of astronomical data-sets is growing
rapidly in the last decades. Now, with new technologies and dedicated
survey telescopes, the databases are even growing faster. Besides
dealing with poly-structed and complex data, sparse data has become a
field of growing scientific interest. By applying technologies from
the fields of computer sciences, mathematics, and statistics,
astronomical data can be accessed and analysed more efficiently.
A specific field of research in astroinformatics is the estimation of the redshift of extra-galactic sources, a measure of their distance, by just using sparse photometric observations. Observing the full spectroscopic information that would be necessary to directly measure the redshift, would be too time-consuming. Therefore, building accurate statistical models is a mandatory step, especially when it comes to reflecting the uncertainty of the estimates. Statistics and especially weather forecasting has introduced and utilized proper scoring rules and especially the continuous ranked probability score to characterize the calibration as well as the sharpness of predicted probability density functions. After presenting how this work led from well calibrated redshift estimates to an improvement in statistical post-processing of weather forecast simulations, an example of interdisciplinarity in data-science, we continue with unsupervised machine learning techniques. We start with the challenge of classifying morphologies of radio-galaxies, talk about star-formation history in LMC, discuss the difficulties in representing time-series, and end with a discussion on novel explorative science platforms for e.g. spectral data. In this part of the talk, we show-case how machine learning can be used as a machinery of discovery to access large data-sets. Several examples are presented to provide examples for the individual researchers in the audience. |