Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference
Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues · 2026
Read Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference by Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21253.txt.
Introduction
Camille Touron and co-authors give a guide for annealed Langevin dynamics in combined simulation-based inference. Score methods approximate a posterior from many observations by pooling learned scores. The combined score matches no spread along the forward path, so reverse sampling carries bias. Annealed Langevin dynamics treats it as the score of bridging densities sampled in turn, giving a bias that can be controlled when tuned well. Tuning choices cover step sizes, steps per level, and level counts.
Catalog record
| CREATOR | Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 134536 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.21253
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference Camille Touron, Gabriel V. Cardoso, Julyan Arbel, Pedro L. C. Rodrigues Abstract Compositional score-based approaches to simulation-based inference (SBI) approximate the posterior over a shared parameter given n n independent observations by aggregating individually learned posterior scores: currently, there are two main propositions of such methods ( Geffner et al., 2023 , Linhart et al., 2026 ). As the resulting composite score does not correspond to the score of any distribution along the forward diffusion path of the true multi-observation posterior, sampling from it via a reverse SDE leads to an irreducible bias. Annealed Langevin dynamics provides a principled alternative: it treats the composite score as the genuine score of a sequence of tractable bridging densities and samples from them in succession. When properly tuned, it could lead to a controllable bias. However, its hyperparameters, namely step sizes, the number of steps per level, and the number of annealing levels, have so far been chosen empirically. We derive Wasserstein bounds for annealed Langevin with approximate scores and translate them into explicit decision rules for these hyperparameters that guarantee a prescribed sampling accuracy, while highlighting different theoretical aspects of each composite score formulation. In the Gaussian setting, we obtain closed-form expressions for all relevant quantities and prove that the bridging densities of Linhart et al., 2026 consistently admit larger step sizes and require fewer total Langevin steps than those of Geffner et al., 2023 . Furthermore, we show empirically that the tuning obtained in the Gaussian setting generalizes to more complex problems, thus providing a well-understood and theoretically grounded starting point for practitioners using compositional score-based approaches. 1 Introduction In simulation-based inference (SBI), the goal is to approximate the posterior distribution p(\theta\mid x) of a stochastic model whose likelihood is intractable but which can be simulated [ 6 , 8 ] . A recurrent challenge arises when n independent observations x_{1},\ldots,x_{n} share a common unknown parameter \theta^{\star} : as n grows, the multi-observation posterior p(\theta\mid x_{1:n}) concentrates, and one wants inference to sharpen accordingly. Pooling strategies train a single network on all n observations jointly [ 19 , 20 ] , but their simulation budget scales with n and the network must be retrained for each new choice of n . Compositional strategies [ 13 , 16 ] instead combine individual posterior estimators trained once on single-observation data, amortizing over n at no additional simulation cost: a key practical advantage that motivates our focus on this paradigm. Score-based diffusion models [ 24 , 26 , 22 ] provide a natural starting point: for a single observation, one learns the individual posterior score \nabla_{\theta}\log p_{t}(\theta\mid x_{i}) along a forward diffusion path and uses it in a reverse SDE to draw samples from p(\theta\mid x_{i}) . In the multi-observation setting, the tempting shortcut is to aggregate these individual scores into a composite score and run the same reverse SDE to target p(\theta\mid x_{1:n}) . This shortcut is, however, theoretically unjustified. The reverse SDE must be driven by the exact score of the joint forward diffusion of p(\theta\mid x_{1:n}) , i.e. \nabla_{\theta}\log p_{t}(\theta\mid x_{1:n}) , which satisfies the Fokker–Planck equation of that specific process. A composite of individual scores does not satisfy this equation: it is the score of a different distribution \pi_{t}\neq p_{t}(\theta\mid x_{1:n}) that does not lie on the forward diffusion path of the target. Using such a score in a reverse SDE therefore produces samples from the wrong distribution, and no amount of additional reverse steps can correct this structural mismatch [ 9 ] . This also explains why some works [ 26 , 14 ] attempt to correct this bias by incorporating additional Langevin refinement steps into the standard reverse diffusion process. However, this comes at the cost of sacrificing the efficiency of diffusion as a fast sampling method while keeping the challenge of tuning the Langevin hyperparameters. Several works recognized this problem and proposed composite scores that, while still mismatched with the true forward diffusion, deliberately define their own tractable sequence of bridging densities [ 13 , 1 , 16 , 14 ] . In this paper, we focus on the following two main composite score formulations considered in the literature. Geffner et al., 2023 use s^{G}_{t}(\theta\mid x_{1:n})=(1-n)\,\nabla_{\theta}\log\lambda_{t}(\theta)+\textstyle\sum_{i=1}^{n}\nabla_{\theta}\log p_{t}(\theta\mid x_{i}), (1) whose associated bridging density \pi^{G}_{t} is well-defined and tractable. Linhart et al., 2026 incorporate a second-order Gaussian approximation of the backward diffusion kernels to obtain s^{L}_{t} , whose bridging density \pi^{L}_{t} is closer to the true diffused posterior p_{t}(\theta\mid x_{1:n}) , at the cost of estimating per-observation and prior covariance matrices \Sigma_{t,i} , \Sigma_{t,\lambda} . Their composite score reads s_{t}^{L}(\theta\mid x_{1:n})=\Lambda_{t}^{-1}\Big(\sum_{i=1}^{n}\Sigma_{t,i}^{-1}\nabla_{\theta}\log p_{t}(\theta\mid x_{i})+(1-n)\Sigma_{t,\lambda}^{-1}\nabla_{\theta}\log\lambda_{t}(\theta)\Big), (2) where \Lambda_{t} is a linear combination of \Sigma_{t,i} , \Sigma_{t,\lambda} and \lambda_{t}(\theta) denotes the diffused prior. In both cases, the key insight is to stop trying to target p_{t}(\theta\mid x_{1:n}) through a reverse SDE, and instead define \pi^{G}_{t} or \pi^{L}_{t} as the actual sampling target: the question becomes how to efficiently draw samples from these bridging densities given access to their scores. Annealed Langevin dynamics is one theoretically sound alternative [ 24 ] . It runs T successive Unadjusted Langevin Algorithm (ULA) chains, each with step size h_{t_{p}} and k_{t_{p}} number of steps, \theta^{(j+1)}=\theta^{(j)}+h_{t_{p}}\,\nabla_{\theta}\log\pi_{t_{p}}\!(\theta^{(j)})+\sqrt{2h_{t_{p}}}\,z^{(j)},\quad z^{(j)}\sim\mathcal{N}(0,I_{d}),\ \ j=0,\ldots,k_{t_{p}}-1 (3) on bridging densities \{\pi_{t_{p}}\}_{p=0}^{T} interpolating from \pi_{t_{T}}=\mathcal{N}(0,I_{d}) to the target \pi_{t_{0}} (Figure 1 ). It requires only the score of each \pi_{t_{p}} , which the composite scores evaluated at time t_{p} provide by construction, and its gradual annealing improves mixing for concentrated posteriors at large n . Since Song et al., 2021 established the SDE interpretation of diffusion models, the field has largely shifted to reverse-SDE and DDPM/DDIM samplers [ 15 ] , which require far fewer score evaluations. Those samplers, however, are theoretically grounded only when the score is computed or trained to match the exact forward score \nabla_{\theta}\log p_{t}(\theta\mid x_{1:n}) , a condition violated here by construction. Annealed Langevin avoids this mismatch entirely, at the cost of more score evaluations and the need to choose, at each level t_{p} , a step size h_{t_{p}} , a number of steps k_{t_{p}} , and a total number of levels T . Selecting these hyperparameters in a principled way, with guarantees on the resulting sampling quality, has remained an open problem; both Geffner et al., 2023 and Linhart et al., 2026 resort to empirical tuning. In the context of image generation, Song and Ermon, 2020 propose some techniques to choose these hyperparameters, most relying on empirical validation, but leaving theoretical guarantee on the sampling quality unaddressed. Our paper closes that gap. Figure 1: The Annealed Langevin algorithm consists in T ULA steps in succession. It allows to sample from bridging densities \{\pi_{t_{p}}\}_{p=0}^{T-1} , represented in blue: the first ULA yields samples (in red) from \pi_{t_{T-1}} , the second from \pi_{t_{T-2}} , etc… and the last one from the target density \pi_{t_{0}} . Green dashed arrows stand for the T ULA: each of them has to define its own constant step size h_{t_{p}} and number of steps k_{t_{p}} (simply denoted k_{p} for readability concerns). The first process starts from Gaussian samples; then the start point of each process corresponds to the last iterates of the previous process, which