CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference
Luben M. C. Cabezas, Vagner S. Santos, Thiago R. Ramos, Pedro L. C. Rodrigues, Rafael Izbicki · 2025
Read CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference by Luben M. C. Cabezas, Vagner S. Santos, Thiago R. Ramos, Pedro L. C. Rodrigues, Rafael Izbicki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17077.txt.
Introduction
Cabezas and four co-authors recalibrated simulation-based inference in this paper from twenty twenty-five. Scientists increasingly invert complex simulators to estimate hidden parameters. The resulting credible regions often cover the truth less often than promised. Their framework, CP4SBI, wraps any estimator in a conformal calibration layer.
Two variants adjust the regions locally, one with regression trees and one with distribution functions. Both carry finite-sample guarantees for local coverage. The method works with any region style, from symmetric intervals to highest-density sets. Benchmarks showed better uncertainty quality for flow-based and diffusion-based estimators alike. The authors stress the wrapper needs no changes to the underlying model.
Catalog record
| CREATOR | Luben M. C. Cabezas, Vagner S. Santos, Thiago R. Ramos, Pedro L. C. Rodrigues, Rafael Izbicki |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-17077
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference Luben M. C. Cabezas, Vagner S. Santos, Thiago R. Ramos, Pedro L. C. Rodrigues, Rafael Izbicki Abstract Abstract Current experimental scientists have been increasingly relying on simulation-based inference (SBI) to invert complex non-linear models with intractable likelihoods. However, posterior approximations obtained with SBI are often miscalibrated, causing credible regions to undercover true parameters. We develop CP4SBI , a model-agnostic conformal calibration framework that constructs credible sets with local Bayesian coverage. Our two proposed variants, namely local calibration via regression trees and CDF-based calibration, enable finite-sample local coverage guarantees for any scoring function, including HPD, symmetric, and quantile-based regions. Experiments on widely used SBI benchmarks demonstrate that our approach improves the quality of uncertainty quantification for neural posterior estimators using both normalizing flows and score-diffusion modeling. Keywords: Simulation-based inference, Credible regions, Conformal prediction, Local coverage, Conditional coverage 1 Introduction In recent years, the machine learning research community has shown growing interest in tackling challenging open problems across a range of experimental sciences, including biology ( Chicco 2017 ; Min et al. 2017 ) , astrophysics ( Freeman et al. 2017 ; Rodríguez et al. 2022 ) , neuroscience ( Lueckmann et al. 2017 ) , and fluid dynamics ( Usman et al. 2021 ; Vinuesa and Brunton 2022 ) . A fundamental challenge common to all these domains—and the focus of this work—is the task of inferring the parameter values of statistical models that best explain observed data ( Tarantola 2005 ; Casella and Berger 2024 ) . These statistical models are typically formalized as stochastic simulators, which generate synthetic data \mathbf{x}\in\mathcal{X} given input parameters \theta\in\Theta . Although it is easy to generate data from such simulators, these models generally lack a tractable likelihood function, making classical Bayesian inference techniques for computing posterior distributions, such as Markov Chain Monte Carlo (MCMC), infeasible. Simulation-based inference (SBI) ( Cranmer et al. 2020 ) is a powerful framework for likelihood-free Bayesian inference, allowing posterior estimation from synthetic data generated by simulators. Recent advances in deep generative models enable SBI methods to approximate posterior distributions using expressive neural density estimators. Despite notable empirical successes, SBI is still maturing and faces key challenges that hinder wider adoption. These include sensitivity to model misspecification ( Wehenkel et al. 2025 ) , difficulties with hierarchical or weakly identified models ( Rodrigues et al. 2021 ) , and limited theoretical understanding of posterior consistency with finite data ( Linhart et al. 2023 ) . A meta-analysis by Hermans et al. 2021 shows that current SBI methods often produce overconfident and miscalibrated posteriors. That is, credible regions C(\mathbf{x}_{\text{obs}}) built for parameters for \theta using the approximate posterior \widehat{p}(\theta\mid\mathbf{x}_{\text{obs}}) , where \mathbf{x}_{\text{obs}} is the observed data, typically fail to achieve nominal coverage. This compromises the reliability of downstream tasks that rely on these posteriors ( Murphy 2022 ) . To mitigate the issues of miscalibrated posterior distributions, recent work draws on the success of Conformal prediction (CP) for prediction methods ( Shafer and Vovk 2008 ; Angelopoulos et al. 2023 ) to construct credible regions with finite-sample coverage guarantees in an SBI setting. However, vanilla CP yields only marginally calibrated regions ( Patel et al. 2023 ; Baragatti et al. 2024 ) , meaning that \displaystyle{\mathbb{P}}(\theta\in C(\mathbf{X}))=1-\alpha, (1) which is often refered to as marginal coverage . This is less informative than conditional coverage for a given observation \mathbf{x}_{\text{obs}} , defined as \displaystyle{\mathbb{P}}(\theta\in C(\mathbf{X})\mid\mathbf{X}=\mathbf{x}_{\text{obs}})=1-\alpha, (2) a property that is central to Bayesian inference. Moreover, current CP methods require access to a closed-form approximation to the posterior p(\theta\mid\mathbf{x}) , which is not necessarily possible for all classes of generative models currently used for SBI, such as diffusion models ( Linhart et al. 2024 ) and flow matching ( Wildberger et al. 2023 ) . We address this gap by incorporating local CP techniques ( Izbicki et al. 2022 ; Cabezas et al. 2025a ; Dheur et al. 2025 ) into the SBI framework. Our method, CP4SBI , supports a broad range of posterior approximators, including both density- and sample-based approaches. As illustrated in Figure 1 , CP4SBI yields more reliable, observation-specific credible regions, improving calibration across different \mathbf{x}_{\text{obs}} and refining standard globally calibrated methods. It only requires a calibration dataset \mathcal{D}_{\text{cal}}=\{(\theta_{i},\mathbf{x}_{i})\}_{i=1}^{B} , where each (\theta,\mathbf{X}) is either simulated from the prior and the model, or obtained by withholding a small portion (e.g., 20%) of the training data used for the posterior approximator. Figure 1: Comparison of credible regions on the Two Moons benchmark for two distinct observations \mathbf{x}_{\text{obs}} . The proposed LoCart CP4SBI method (blue) produces regions that more closely match the oracle regions (black) with correct coverage, compared to the global conformal (green) and standard self-calibration (red). This highlights the benefits of local adaptivity in improving the quality of credible region estimates. Paper Outline. Section 1.1 reviews related work. Section 1.2 presents our contributions. Background is shown in Section 2 , followed by methodology in Section 3 . Section 4 presents theoretical results, and Section 5 shows experiments on SBI benchmarks. Section 6 concludes the paper. Appendices A to C contain algorithms, proofs, and additional results and details. 1.1 Relation to Other Work Approximations of posterior distributions. Various SBI strategies for posterior estimation have been developed over recent years. These include initial approaches like Approximate Bayesian Computation (ABC) ( Marin et al. 2012 ; Sisson et al. 2018 ) , which approximates the posterior for a specific observation \mathbf{x}_{\text{obs}} via rejection sampling using a specified distance function and rejection sampling or other MCMC methods. Following this, amortized methods started being used to estimate the posterior distribution and related quantities more efficiently ( Cranmer et al. 2015 ; Gutmann and Corander 2016 ; Izbicki et al. 2019 ) . These methods include neural posterior estimation (NPE) ( Papamakarios and Murray 2016 ; Lueckmann et al. 2017 ; Greenberg et al. 2019 ; Deistler et al. 2022 ) , which directly approximates the posterior density p(\theta\mid\mathbf{x}) ; neural likelihood estimation (NLE) ( Papamakarios et al. 2019 ; Frazier et al. 2023 ) , which estimates the intractable likelihood function p(\mathbf{x}\mid\theta) ; and density ratio estimators ( Izbicki et al. 2014 ; Hermans et al. 2020 ; Durkan et al. 2020 ; Dalmasso et al. 2020 ; Dalmasso et al. 2024 ) , which estimate likelihood ratios such as p(\mathbf{x}\mid\theta)/p(\mathbf{x}) . Additionally, a subsequent development in the field has been the adaptation of more implicit generative models, such as flow-matching and diffusion models, to SBI ( Wildberger et al. 2023 ; Geffner et al. 2023 ; Linhart et al. 2024 ) . These advanced techniques aim to faithfully generate samples directly from the posterior p(\theta\mid\mathbf{x}) without requiring the estimation of a closed-form posterior density. Our approach uses such methods, but provides means to better calibrate credible sets derived from them. Calibration of approximators and recalibration techniques. Despite the breadth of SBI’s recent research and its impact on various scientific fields, Hermans et al. 2021 has revealed a prevalent challenge to all of its variants: they can suffer from statistical miscalibration and produce overconfident estimates in practical settings. Consequently, a significant body of recent work ( Hermans et al. 2021 ; Dey et al. 2022