The Big Sobol Sequence: how many simulations do we need for simulation-based inference in cosmology?

Anirban Bairagi, Benjamin Wandelt, Francisco Villaescusa-Navarro · 2025

Read The Big Sobol Sequence: how many simulations do we need for simulation-based inference in cosmology? by Anirban Bairagi, Benjamin Wandelt, Francisco Villaescusa-Navarro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13755.txt.

Introduction

Bairagi, Wandelt, and Villaescusa-Navarro asked in twenty twenty-five how many sims it takes to train nets that read the cosmos well. Too few sims leave nets short of the best reads. They ran the first full check of that growth for dark stuff spread reads.

Stock suites of two thousand sims fell short for a plain net, they find. A growth law now maps net yield against sim count for many builds. They built the largest shared sim set yet, near thirty-three thousand runs. New suites or new train ways are a must, they say.

Catalog record

CREATORAnirban Bairagi, Benjamin Wandelt, Francisco Villaescusa-Navarro
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13755

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Big Sobol Sequence: how many simulations do we need for simulation-based inference in cosmology? Anirban Bairagi, Benjamin Wandelt, Francisco Villaescusa-Navarro Abstract How many simulations do we need to train machine learning methods to extract information available from summary statistics of the cosmological density field? Neural methods have shown the potential to extract non-linear information available from cosmological data. To achieve this depends on having appropriate network architectures and a sufficient number of simulations for training the networks. In the first detailed convergence study of neural network (NN) training to extract maximally informative summary statistics for cosmological inference, we show that currently available simulation suites, such as the Quijote Latin Hypercube(LH) with 2000 simulations, do not provide sufficient training data for a generic neural network to reach the optimal regime. We present a case study in the context of training a moment network to infer cosmological parameters from the nonlinear dark matter power spectrum, where the optimal information content can be computed through asymptotic analysis using the Cramér-Rao information bound. We find an empirical neural scaling law that predicts how much information a neural network can extract from highly informative summary statistics as a function of the number of simulations used to train the network, for a wide range of architectures and hyperparameters. Looking beyond two-point statistics, we find a similar scaling law for training neural posterior inference using wavelet scattering transform coefficients. To verify our method, we created the largest publicly released suite of cosmological simulations, the Big Sobol Sequence(BSQ), consisting of 32,768 Λ \Lambda CDM n-body simulations uniformly covering the Λ \Lambda CDM parameter space. Our method enables efficient planning of simulation campaigns for machine learning applications in cosmology, while the BSQ dataset provides an unprecedented resource for studying the convergence behavior of neural networks in cosmological parameter inference. Our results suggest that new large simulation suites or new training approaches will be necessary to achieve information-optimal parameter inference from non-linear simulations. 1 Introduction A major effort in modern cosmology is directed towards inferring cosmological parameters and testing the fundamental tenets of the standard model, \Lambda CDM ( Ostriker & Steinhardt 1995 ; Ade et al. 2014 ) , which posits that the universe’s energy budget is dominated by a cosmological constant ( \Lambda ) representing dark energy ( \sim 70%) and dark matter ( \sim 25%). This endeavor is supported both by numerical simulations and forthcoming data from a new generation of major cosmological surveys. These include large-scale structure surveys such as the Dark Energy Spectroscopic Instrument (DESI) ( Abareshi et al. 2022 ) , the Vera C. Rubin Observatory ( Vera C. Rubin Observatory LSST Solar System Science Collaboration et al. 2020 ) , Euclid ( Laureijs et al. 2011 ) , the Nancy Grace Roman Space Telescope ( Spergel et al. 2015 ) , and SPHEREx ( Doré et al. 2014 ) , as well as cosmic microwave background experiments like the Simons Observatory ( Ade et al. 2019 ) and CMB-S4 ( Abazajian et al. 2016 ) . While sophisticated methods for cosmological parameter inference have dramatically reduced our uncertainty about cosmological parameters from past surveys, e.g., WMAP ( Bennett et al. 2003 ) , Planck ( Planck Collaboration et al. 2020 ) , DES ( Abbott et al. 2022 ) , BOSS ( Dawson et al. 2013 ; Alam et al. 2017 ) , DESI ( DESI Collaboration et al. 2024 ) in recent decades, current and upcoming surveys present new methodological challenges. The traditional approach compares summary statistics ( Peebles 1993 ; Peebles 1980 ) —such as n-point correlation functions ( Peebles 2001 ; Scoccimarro 2000 ; Philcox & Slepian 2022 ) , number counts ( Artis et al. 2025 ) from various probes, i.e., galaxy clustering ( Groth et al. 1977 ; Kaiser 1987 ) and weak lensing ( Heymans et al. 2013 ; Heymans et al. 2021 ; Liu et al. 2015 ) —against theoretical predictions or numerical simulations. These methods typically assume an explicit form of the likelihood function, often Gaussian, but this assumption becomes problematic in the non-linear regime where the true likelihood form may not be known, especially in the regime of a few modes and complex modeling. For example, even when the number of modes is large, non-linear evolution can lead to strong correlations, slowing the asymptotic approach to the Gaussian form ( Sellentin & Heavens 2017 ; Hahn et al. 2019 ) . Even when good approximations do exist, standard Bayesian inference techniques like Markov Chain Monte Carlo (MCMC) can face convergence challenges ( Liddle 2009 ; Leclercq et al. 2014 ) . Implicit Inference (II), or Simulation-Based Inference (SBI), has emerged as a promising alternative that learns the joint statistics of data and parameters directly from simulated pairs, avoiding the need for explicit likelihood assumptions. SBI has demonstrated both accuracy and computational efficiency across various applications ( Alsing et al. 2018 ; Alsing et al. 2019 ; Cranmer et al. 2020 ; Jeffrey & Wandelt 2020 ; Joachimi, B. et al. 2021 ; Cole et al. 2022 ; Lemos et al. 2023 ) . Neural networks play two crucial roles in modern SBI approaches: generating or compressing informative summary statistics ( Jeffrey & Wandelt 2020 ; Lemos et al. 2024 ; Novaes et al. 2025 ) and representing statistical distributions such as the likelihood or posterior. Information Maximizing Neural Networks (IMNN) exemplify the first role, focusing on Fisher information maximization ( Charnock et al. 2018 ) , while (ensembles of) normalizing flows ( Papamakarios et al. 2019 ; Papamakarios et al. 2017 ) , neural ratio estimation ( Cole et al. 2022 ) , and diffusion models ( Song et al. 2020b ; Ho et al. 2020 ; Song et al. 2020a ) are examples of the second. While both of these tasks will require assessment of convergence, in this paper we will focus on the question of information extraction 1 1 1 For a recent paper addressing the convergence of neural density estimators in the cosmological context, see Homer et al. (2025) . , for the following reasons: 1) suboptimal information extraction will affect downstream tasks such as the inference of the likelihood or the posterior distribution; and 2) it is well known that the difficulty of fitting general high-dimensional distributions scales poorly with the number of dimensions, highlighting the utility of extracting information in a low-dimensional feature vector. Even in cases where neural density estimators are trained conditional on the feature vector, it has been observed in several of the above-cited works that providing long, uncompressed data vectors to conditional neural density estimators leads to suboptimal performance. Given the centrality of trained neural networks for the extraction of informative features, a key question emerges: How many simulations are needed to achieve near-optimal results? While one could generate simulations until the extracted information plateaus, this brute-force approach is computationally expensive or even intractable. While even suboptimal information extraction may be an improvement over traditional summary statistics, it is clearly interesting to have a simple tool based on a power law fit to assess whether the extracted information from valuable data could be improved by running more training simulations. Our work presents the first

Read the complete text