Error analysis of a compositional score-based algorithm for simulation-based inference
Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues · 2025
Read Error analysis of a compositional score-based algorithm for simulation-based inference by Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-15817.txt.
Introduction
Camille Touron, Gabriel V. Cardoso, Julyan Arbel, and Pedro L. C. Rodrigues bound the error of a compositional score method for inference from simulations. Simulation-based inference estimates the parameters of a stochastic model that best explain observations. Combining many observations should sharpen the posterior, and score-based diffusion methods do so by adding the scores from each observation.
A natural worry is that small errors in each score add up and spoil sampling as observations pile up. That worry had not been studied in theory. The four authors analyse the GAUSS algorithm of Linhart and colleagues and prove an upper bound on its mean squared error. The bound depends on both the single-score errors and the number of observations. A Gaussian example, where every formula is available in closed form, illustrates the theory.
Catalog record
| CREATOR | Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-15817
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Error analysis of a compositional score-based algorithm for simulation-based inference Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues Abstract Simulation-based inference (SBI) has become a widely used framework in applied sciences for estimating the parameters of stochastic models that best explain experimental observations. A central question in this setting is how to effectively combine multiple observations in order to improve parameter inference and obtain sharper posterior distributions. Recent advances in score-based diffusion methods address this problem by constructing a compositional score, obtained by aggregating individual posterior scores within the diffusion process. While it is natural to suspect that the accumulation of individual errors may significantly degrade sampling quality as the number of observations grows, this important theoretical issue has so far remained unexplored. In this paper, we study the compositional score produced by the GAUSS algorithm of Linhart et al., 2024 and establish an upper bound on its mean squared error in terms of both the individual score errors and the number of observations. We illustrate our theoretical findings on a Gaussian example, where all analytical expressions can be derived in a closed form. 1 Introduction Probabilistic approaches to inverse problems ( Tarantola, 2005 ) aim at inferring parameters \bm{\theta} of a stochastic model from its outputs \bm{x} through the posterior distribution p(\bm{\theta}|\bm{x}) . Directly sampling from this posterior is often difficult in practice, since the associated likelihood p(\bm{x}|\bm{\theta}) is frequently intractable. To address this, simulation-based inference with conditional score-based modeling ( Sharrock et al., 2024 ) approximates the posterior by learning a time-dependent conditional estimate s_{\phi}(\bm{\theta},\bm{x},t) of the score of noisy versions of the posterior, \nabla_{\bm{\theta}}\log p_{t}(\bm{\theta}|\bm{x}) . Importantly, this training relies only on joint samples (\bm{\theta}_{i},\bm{x}_{i})\sim p(\bm{\theta},\bm{x}) which can be obtained sequentially as \bm{\theta}_{i}\sim\lambda(\bm{\theta}) , the prior, and then \bm{x}_{i}|\bm{\theta}_{i}\sim p(\bm{x}|\bm{\theta}_{i}) , thus circumventing the need to evaluate the likelihood (see Appendix A.1 for details). A central question in this line of work is how the denoising score matching mean squared error, denoted \epsilon_{\mathrm{DSM}}^{2} , affects the quality of samples obtained via a backward diffusion process (see Figure 1 ). Recent results by Gao et al., 2025 provide explicit bounds on the Wasserstein error of the approximate samples in terms of \epsilon_{\mathrm{DSM}}^{2} and the discretization hyperparameters (number of steps, step size) used in the backward process. These results imply that one can reach a prescribed sampling accuracy (in Wasserstein distance) either by tuning the diffusion hyperparameters or by improving the accuracy of the score estimator. We extend this setting to n IID observations, with the goal of inferring parameters from the posterior p(\bm{\theta}|\bm{x}_{1:n}) . In practice, additional observations improve inference, as the posterior concentrates with growing n (see Figure 1 ). Recent works ( Linhart et al., 2024 ; Geffner et al., 2023 ; Arruda et al., 2025 ; Gloeckler et al., 2025 ) propose a compositional score approach: instead of directly approximating \nabla_{\bm{\theta}}\log p_{t}(\bm{\theta}|\bm{x}_{1:n}) with a highly complex network, they aggregate individual posterior scores s_{\phi}(\bm{\theta},\bm{x}_{j},t)\approx\nabla_{\bm{\theta}}\log p_{t}(\bm{\theta}|\bm{x}_{j}) . The resulting compositional score can then be used in backward diffusion or annealed Langevin dynamics to sample from the multi-observation posterior. Importantly, if we can bound the error of the compositional score estimate, then we can guarantee convergence of our sampling to the target multi-observation posterior in terms of Wasserstein distance ( Gao et al., 2025 ) . However, while the error of each individual score estimate can be controlled during training time, the error of the aggregated compositional score arises from the accumulation of individual errors as n grows. To the best of our knowledge, this accumulation effect has not been theoretically analyzed so far. Here we address this gap: focusing on the compositional score ( 1 ) defined by the GAUSS algorithm of Linhart et al., 2024 , we derive in Proposition 2 an upper bound on the mean squared error (MSE) of its estimate ( 2 ) as a function of the n individual score errors. Figure 1: Contours represent the true target posterior distribution p (see Appendix A.2.1 for details), either conditioned on a single observation (top row) or eleven of them (bottom row): increasing the number of conditional observations sharpens the posterior and allows for better inference. Scattered points in red are samples from \tilde{p} obtained by a diffusion process with an inexact score estimate with mean squared error \epsilon_{\mathrm{DSM}}^{2} : larger errors tend to bias the sampling and degrade its quality. 2 Background on compositional score estimation with GAUSS Linhart et al., 2024 adopt the common assumption ( Sohl-Dickstein et al., 2015 ) that the backward kernels of the diffused prior and the diffused individual posteriors can be well approximated by Gaussian distributions for all times t\in[0,T] of a diffusion process, namely \hat{q}^{\lambda}_{0|t}(\bm{\theta}_{0}|\bm{\theta}_{t})=\mathcal{N}\big(\bm{\theta}_{0};\mu_{t,\lambda}(\bm{\theta}_{t}),\Sigma_{t,\lambda}(\bm{\theta})\big)~~\text{and}~~\hat{q}_{0|t}(\bm{\theta}_{0}|\bm{\theta}_{t},\bm{x}_{j})=\mathcal{N}\big(\bm{\theta}_{0};\mu_{t,j}(\bm{\theta}_{t}),\Sigma_{t,j}(\bm{\theta})\big), with j\in\{1,\dots,n\} . In addition, their GAUSS algorithm assumes that both the prior and each individual posterior are themselves Gaussian, \mathcal{N}(\bm{\theta};\mu_{\lambda},\Sigma_{\lambda}) and \mathcal{N}(\bm{\theta};\mu_{j},\Sigma_{j}) . Under this assumption, the diffused covariances \Sigma_{t,\lambda} and \Sigma_{t,j} are fully determined and, importantly, no longer depend on \bm{\theta} . With these simplifications, the diffused compositional score takes the form \nabla_{\bm{\theta}}\log p_{t}(\bm{\theta}|\bm{x}_{1:n})=\Lambda^{-1}\Bigg(\sum_{j=1}^{n}\Sigma_{t,j}^{-1}\nabla_{\bm{\theta}}\log p_{t}(\bm{\theta}|\bm{x}_{j})-(n-1)\Sigma_{t,\lambda}^{-1}\nabla_{\bm{\theta}}\log\lambda_{t}(\bm{\theta})\Bigg), (1) where \Lambda=\sum_{j}\Sigma^{-1}_{t,j}-(n-1)\Sigma^{-1}_{t,\lambda} . Note that the compositional score ( 1 ) does not stem from the simple addition of the n individual scores, as it aims at correctly approximating the true score of a noisy version of the multi-observation posterior for some specific noise level prescribed by a classical diffusion process. For most priors used in the SBI literature (Gaussian, Uniform, log-Normal), the diffused score \nabla_{\bm{\theta}}\log\lambda_{t}(\bm{\theta}) has a closed-form expression and therefore introduces no error ( Sharrock et al., 2024 ) . In contrast, Equation ( 1 ) also involves the scores of the individual posteriors, which are only available through the approximations s_{\phi}(\bm{\theta},\bm{x}_{j},t) and thus contribute to errors \epsilon_{\mathrm{DSM},j}^{2} . A further source of bias comes from the estimation of the covariance matrices \Sigma_{\lambda} and \Sigma_{j} . Covariance \Sigma_{\lambda} can be approximated by the empirical covariance of prior samples. However, for each posterior one must simulate a backward diffusion process using the approximate score s_{\phi}(\bm{\theta},\bm{x}_{j},t) to generate samples from \tilde{p}(\bm{\theta}|\bm{x}_{j})\approx{p}(\bm{\theta}|\bm{x}_{j}) , and then compute the empirical covariance \tilde{\Sigma}_{j} . The estimator for the compositional score ( 1 ) is then written as: \displaystyle s(\bm{\theta},\bm{x}_{1:n},t) \displaystyle=\tilde{\Lambda}^{-1}\left(\sum_{j=1}^{n}\tilde{\Sigma}_{t,j}^{-1}s_{\phi}(\bm{\theta},\bm{x}_{j},t)-(n-1)\tilde{\Sigma}_{t,\lambda}^{-1}\nabla_{\bm{\theta}}\log\lambda_{t}(\bm{\theta})\right). (2) In what follows, we derive an upper bound on the mean squared error of this estimator in terms of the individual score errors \epsilon_{\mathrm{DSM},j}^{2} , the errors incurred in estimating the precision matrices {\Sigma}^{-1}_{\lambda} and {\Sigma}^{-1}_{j} , and the number of observations n . We use \|.\| to refer either to the Euclidean norm when applied to vectors, or spectral norm when applied to matrices. 3 Contributions We first derive an upper bound on the error of the estimator for the precision matrices, noting that for the GAUSS algorithm the errors \|\Sigma^{-1}_{t,j}-\tilde{\Sigma}^{-1}_{t,j}\| at each t\in[0,T] are directly related to the initial estimation error \|\Sigma^{-1}_{j}-\tilde{\Sigma}^{-1}_{j}\| , which in turn depends on the sampling quality from each individual posterior. Since Gaussian distributions are smooth log-concave, we can use the aforementioned results from Gao et al., 2025 to tune the discretization hyperparameters of each backward diffusion process and drive the individual score error \epsilon_{\mathrm{DSM},j}^{2} sufficiently low to achieve a prescribed small Wasserstein error. The following proposition (proof in Appendix A.3.2 ) provides a bound on the