When (not) to trust Monte Carlo approximations for hierarchical Bayesian inference

Jack Heinzel, Salvatore Vitale · 2025

Read When (not) to trust Monte Carlo approximations for hierarchical Bayesian inference by Jack Heinzel, Salvatore Vitale (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07221.txt.

Introduction

Jack Heinzel and 1 co-author warn in 2025 about when Monte Carlo approximations can mislead hierarchical Bayesian inference. Hierarchical inference combines many observations into one population picture, a key task for gravitational wave astronomy, which expects thousands of new detections. The method relies on Monte Carlo sampling, random draws that stand in for exact sums, and those draws are already costly.

From first principles, the authors derive an error estimator that measures how much information the sampling approximation loses. They advise keeping that loss at or below about 0.2 bits for reliable results. They also show the common likelihood estimator is biased but can be corrected. Tests on synthetic gravitational wave populations show simple models with strong assumptions resist sampling error better. A free software package lets analysts compute the error measure on their own data.

Catalog record

CREATORJack Heinzel, Salvatore Vitale
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07221

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When (not) to trust Monte Carlo approximations for hierarchical Bayesian inference Jack Heinzel, Salvatore Vitale Abstract The coming years of gravitational wave astrophysics promises thousands of new detections, which can unlock fundamental scientific insights if the information in each observation can be properly synthesized into a coherent picture. State-of-the-art approaches often accomplish this with hierarchical Bayesian inference. However, this typically relies on Monte Carlo approximations that are already very expensive in current data, and may become prohibitively so in the future. In this paper we show how this process can be understood from a first-principles statistical approach. We derive an error estimator E ^ \hat{E} for quantifying the amount of information that is lost due to the Monte Carlo approximation and recommend that this error is limited to no more than E ^ ≲ 0.2 \hat{E}\lesssim 0.2 bits for reliable inference. We also show that the hierarchical likelihood estimator is biased but may be corrected. Finally, we show some practical examples for inference on synthetic gravitational-wave population inference, demonstrating that simple models with strong assumptions can be much more stable to Monte Carlo uncertainty than those with weaker assumptions. We also provide a pip installable package population-error with which analysts can calculate the error statistics E ^ \hat{E} . I Introduction and summary of results The initial detection of GW ( Abbott et al. 2016a ) opened a new and unique probe into fundamental questions in astrophysics, cosmology, nuclear physics and gravitational physics. Since then, the LVK ( LVK ) has confidently detected nearly 100 fully characterized observations of GW ( Abbott et al. 2019a ; Abbott et al. 2021a ; Abbott et al. 2024 ; Abbott et al. 2023a ) and promises to detect many more in the coming years ( Abbott et al. 2016b ) . Astrophysicists use these data to probe the formation of compact objects like BH and NS ( Vitale et al. 2017 ; Abbott et al. 2016c ; Zevin et al. 2017a ; Mandel and Farmer 2022 ; Mandel and Broekgaarden 2022 ; Abbott et al. 2023b , e.g.) , cosmologists use GW to constrain the structure and evolution of the universe ( Schutz 1986 ; Holz and Hughes 2005 ; Messenger and Read 2012 ; Abbott et al. 2017 ; Christensen 2019 ; Ezquiaga and Holz 2022 ; Gair et al. 2023 , e.g.) , nuclear scientists use merging NS to understand the equation of state of matter at supranuclear densities ( Baym et al. 2018 ; Capano et al. 2020 ; Dietrich et al. 2020 ; Chatziioannou 2020 ; Landry et al. 2020 , e.g.) , and gravitational physicists study the dynamic, strongly curved spacetimes around merging BH to test and place limits on the fundamental theory of gravity ( Mirshekari et al. 2012 ; Will 2014 ; Abbott et al. 2016d ; Abbott et al. 2019b ; Abbott et al. 2021b , e.g.) . In all of these disciplines, modern approaches often work by extracting information from the collection of GW observations using a powerful statistical method called hierarchical Bayesian inference ( Abbott et al. 2023b ; Del Pozzo 2012 ; Del Pozzo et al. 2013 ; Isi et al. 2019 , e.g.) . Hierarchical Bayesian inference posits an unknown underlying population from which each observation originates, and then, using the observed data, infers the underlying population by constructing a probabilistic forward model (and corresponding likelihood) for generating the observations ( Essick and Fishbach 2024 , see, e.g.) . However, the population analysis problem is technically challenging for several reasons. As the number of events included in the catalog grows, the approximations in a typical hierarchical Bayesian analysis begin to break down. For one, the noise is usually approximated as Gaussian and stationary ( Moore et al. 2015 ; Abbott et al. 2020a ) , however, the non-Gaussian and nonstationary reality can occasionally cause biases in the analysis of single events ( Pankow et al. 2018 ; Chatziioannou et al. 2021 ; Hourihane et al. 2022 ; Davis et al. 2022 ; Payne et al. 2022 ; Ghonge et al. 2024 ; Macas et al. 2024 ; Udall et al. 2025 ) , which can propagate to biases in the population level analyses. Furthermore, non-astrophysical noise transients present in real noise can imitate astrophysical GW and sometimes are significant enough to contaminate the catalog of sources included in the GW population analysis ( Cabero et al. 2019 ; Zevin et al. 2017b ; Davis et al. 2021 ; Soni et al. 2021 ; Acernese et al. 2023 ; Akutsu et al. 2021 ; Ashton et al. 2022 ; Magee et al. 2024 ) . Third, the results of a population inference may be strongly model dependent, and therefore misspecification of the model can cause additional biases or lead to incorrect interpretations of the population ( Cheng et al. 2023 ; Alvarez-Lopez et al. 2025 , e.g.) . In this paper, we focus on an additional challenge in the GW population inference problem: we do not have practical access to the exact population-level Bayesian likelihood. Instead, it is common to approximate the likelihood using a Monte Carlo approach, which estimates the many integrals appearing in the hierarchical likelihood. While this is an acceptable approximation for many simple population inference problems, it may fail for others ( Golomb and Talbot 2023 , e.g.) . This problem is well known in the GW population analysis community. However the underlying statistical phenomena is still not completely understood from a first-principles approach. In summary, our work focuses on the four following results: 1. Uncertainty quantification: We derive an error statistic \hat{E} to estimate the expected amount of information lost due to the Monte Carlo approximations. \hat{E} may be computed in post-processing, and we empirically verify the accuracy of this error estimator. 2. Unbiased likelihood estimator: We show the GW population analysis likelihood estimator is a biased estimator. The selection efficiency integral enters nonlinearly in the likelihood and causes the likelihood estimate to tend to be biased to larger values when the uncertainty is larger. We show this bias can be corrected with a simple correction term during inference. 3. Log-likelihood variance thresholds are safe…: Using the Cauchy–Schwarz inequality, we show that the log-likelihood variance threshold conjectured in Talbot and Golomb 2023 ensures the error in the posterior estimator is bounded. On the other hand, we show counterexamples that thresholds based on individual Monte Carlo integrals are not necessarily safe (as used in e.g., previous LVK population analyses Abbott et al. 2023b ). 4. …However, they can be stricter than necessary: In practice, parametric models with only a few parameters are often better behaved than the log-likelihood variance would indicate, but weaker models with many parameters tend to require the strict log-likelihood variance threshold. The analyst can check the validity of an inference using the error statistic \hat{E} . This is related to the fact that only uncertainty in the differences in the log-likelihood is important ( Farr 2019 ; Essick and Farr 2022 ; Talbot and Golomb 2023 ) . Our work is inspired by previous studies ( Farr 2019 ; Essick and Farr 2022 , and Talbot and Golomb 2023 ) to reconcile two opposing

Read the complete text