Data Compression and Inference in Cosmology with Self-Supervised Machine Learning

Aug 3, 2026

Upload by Hazel

Massive volumes of data from current and upcoming cosmological surveys require compression methods that summarize observations while preserving essential physical information. The work presents a self-supervised machine learning approach that builds representative summaries of large datasets using simulation-based augmentations. Experiments on hydrodynamical cosmological simulations show highly informative summaries that support precise parameter inference. The method also yields summary representations robust to systematic effects, including baryonic physics influences.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages23

Reading Time58 min

AccessFree to download

Abstract

Introduction

Motivation: large-scale cosmological surveys and dimensionality challenges

Existing summary statistics and their limitations

Fisher-information-preserving and optimal compression methods

Score-function and nuisance-hardened compression approaches

arXiv :2308 .09751v2 [ astro-ph .CO] 14 Dec 2023 Data Compression and Inference in Cosmology with Self-Supervised Machine Learning Aizhan Akhmetzhanova, 1★ Siddharth Mishra-Sharma2,3, 1†, and Cora Dvorkin1‡ 1 Department of Physics, Harvard University, Cambridge, MA 02138, USA 2 The NSF AI Institute for Artificial Intelligence and Fundamental Interactions, Cambridge, MA 02139, USA 3 Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA 18 December 2023 ABSTRACT The influx of massive amounts of data from current and upcoming cosmological surveys necessitates compression schemes that can efficiently summarize the data with minimal loss of information. We introduce a method that leverages the paradigm of self-supervised machine learning in a novel manner to construct representative summaries of massive datasets using simulationbased augmentations. Deploying the method on hydrodynamical cosmological simulations, we show that it can deliver highly informative summaries, which can be used for a variety of downstream tasks, including precise and accurate parameter inference. We demonstrate how this paradigm can be used to construct summary representations that are insensitive to prescribed systematic effects, such as the influence of baryonic physics. Our results indicate that self-supervised machine learning techniques offer a promising new approach for compression of cosmological data as well its analysis. Key words: methods: data analysis – cosmology: miscellaneous. 1 INTRODUCTION the sufficiency of manually-derived statistics (i.e., ability to compress all physically-relevant information) is the exception rather than the Over the last few decades, cosmology has undergone a phenomenal norm. transformation from a ‘data-starved’ field of research to a precision science. Current and upcoming cosmological surveys such as those In addition, given the estimated sizes of the datasets from future conducted by the Dark Energy Spectroscopic Instrument (DESI) surveys, even traditional summary statistics might be too large for (Aghamousa et al. 2016), Euclid (Laureĳs et al. 2011), the Vera scalable data analysis. For instance, Heavens et al. (2017) estimates C. Rubin Observatory (LSST Dark Energy Science Collaboration that for surveys such as Euclid or LSST, with an increased number 2012), and the Square Kilometer Array (SKA) (Weltman et al. 2020), of tomographic redshift bins, the total number of data points of among others, will provide massive amounts of data, and making full summary statistics for weak-lens∼ ing data (such as shear correlation use of these complex datasets to probe cosmology is a challenging functions) could be as high as 104, which might be prohibitively task. The necessity to create and manipulate simulations correspond- expensive when the covariance matrices for the data need to being to these observations further exacerbates this challenge. evaluated numerically from complex simulations. In order to take Analyzing the raw datasets directly is a computationally expen- advantage of the recent advances in the field of simulation-based sive procedure, so they are typically first described in terms of a inference (SBI) (e.g. Cranmer et al. (2020); Alsing et al. (2019)), theset of informative lower-dimensional data vectors or summary statis- size of the summary statistic presents an important consideration duetics, which are then used for parameter inference and other down- to the curse of dimensionality associated with the comparison of the stream tasks. These summary statistics are often motivated by in- simulated data to observations in a high-dimensional space (Alsing ductive biases drawn from the physics of the problem at hand. Some & Wandelt 2019) . widely used classes of summary statistics include power spectra and A number of methods have been proposed to construct optimal higher-order correlation functions (Chen et al. 2021b; Gualdi et al.

FAQs

Why is data compression necessary in modern cosmology surveys?

Upcoming surveys will generate extremely large datasets, making direct analysis and full high-dimensional summary statistics computationally expensive. Compression is needed to retain physically relevant information with minimal loss.

What core idea does the method use to compress cosmological data?

The method leverages self-supervised machine learning with simulation-based augmentations to construct representative summaries from massive datasets.

How does the approach handle systematic effects such as baryonic physics?

It constructs summary representations designed to be insensitive to prescribed systematic effects, including the influence of baryonic physics, improving robustness for downstream inference tasks.

Aug 3, 2026

Upload by Hazel