Upper and lower bounds for local Lipschitz stability of Bayesian posteriors
Nada Cvetković, Han Cheng Lie · 2025
Read Upper and lower bounds for local Lipschitz stability of Bayesian posteriors by Nada Cvetković, Han Cheng Lie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23541.txt.
Introduction
Bayesian inference flips data into a posterior belief, and users need to know how much that belief wobbles when the inputs shift slightly. Cvetković and Lie study the maps from the misfit term and from the prior to the posterior, proving both upper and lower bounds on their local sensitivity. Earlier work had upper bounds only, in several distance measures.
The new bounds expose the role of the evidence, the overall fit of the model. As the evidence shrinks toward zero and the posterior sharpens, sensitivity not only can grow but in general does grow. The authors give conditions under which both maps are locally two-way stable, neither stretching nor squashing distances.
Catalog record
| CREATOR | Nada Cvetković, Han Cheng Lie |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 168 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23541
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Upper and lower bounds for local Lipschitz stability of Bayesian posteriors Nada Cvetković, Han Cheng Lie Abstract The work of Sprungk (Inverse Problems, 2020) established the local Lipschitz continuity of the misfit-to-posterior and prior-to-posterior maps with respect to the Kullback–Leibler divergence and the total variation, Hellinger, and 1-Wasserstein metrics, by proving certain upper bounds. The upper bounds were also used to show that if a posterior measure is more concentrated, then it can be more sensitive to perturbations in the misfit or prior. We prove upper bounds and lower bounds that emphasise the importance of the evidence. The lower bounds show that the sensitivity of posteriors to perturbations in the misfit or the prior not only can increase, but in general will increase as the posterior measure becomes more concentrated, i.e. as the evidence decreases to zero. Using the explicit dependence of our bounds on the evidence, we identify sufficient conditions for the misfit-to-posterior and prior-to-posterior maps to be locally bi-Lipschitz continuous. Keywords: Bayesian inference, posterior perturbation analysis, noninvertibility, total variation metric, Hellinger metric, Wasserstein metric, Kullback–Leibler divergence 1 Introduction In the theory of inverse problems, one is given an observed data vector y\in\mathcal{Y} , an unknown parameter \theta\in\Theta , and a function G:\Theta\to\mathcal{Y} such that y=G(\theta) , and the task is to infer \theta given y . In the frequentist Bayesian approach to inverse problems, one assumes that the observation is a realisation of a \mathcal{Y} -valued random variable Y=G(\theta^{\dagger})+\varepsilon , where \theta^{\dagger} is fixed and \varepsilon is observation noise, and one models the unknown \theta^{\dagger} using a \Theta -valued random variable with prior law \mu . The solution to the Bayesian inverse problem is given by the posterior law \mu_{\Phi} of \theta given the observed quantity y , where \mu_{\Phi} is absolutely continuous with respect to the prior \mu and uniquely defined by the normalised likelihood \ell_{\Phi,\mu} : \ell_{\Phi,\mu}(\theta;y)\coloneqq\frac{\mathrm{d}\mu_{\Phi}}{\mathrm{d}\mu}(\theta)=\frac{\exp(-\Phi(\theta;y))}{Z_{\Phi,\mu}(y)},\quad Z_{\Phi,\mu}(y)\coloneqq\int\exp(-\Phi(\theta^{\prime};y))\mathrm{d}\mu(\theta^{\prime}). (1.1) We shall refer to the negative logarithm \Phi of the unnormalised likelihood and the normalisation constant Z_{\Phi,\mu} as the ‘misfit’ and the ‘evidence’ respectively. The misfit is determined by the ‘forward model’ G that maps unknowns to observed quantities and the distribution of the observation noise \varepsilon . For example, if \mathcal{Y}=\mathbb{R}^{d} for some d\in\mathbb{N} and \varepsilon is Gaussian with mean 0 and positive-definite covariance C , then \Phi(\theta;y)=\tfrac{1}{2}\left\|C^{-1/2}(G(\theta)-y)\right\|^{2} . An important theme in the theory of Bayesian inverse problems is the stability of the posterior with respect to perturbations in one of the objects that define it. Let \mathfrak{d}_{\mathcal{P}(\Theta)} be some chosen metric or divergence on the space of probability measures on a fixed measurable space (\Theta,\Sigma) . For a fixed prior \mu , the stability of the posterior \mu_{\Phi} with respect to the misfit \Phi can be described by the local Lipschitz continuity of the misfit-to-posterior mapping L^{p}_{\mu}\ni\Phi\mapsto\mu_{\Phi} , i.e. by an upper bound of the form \mathfrak{d}_{\mathcal{P}(\Theta)}(\mu_{\Phi_{1}},\mu_{\Phi_{2}})\leq C(\mu,\Phi_{1})\left\|\Phi_{1}-\Phi_{2}\right\|_{L^{p}_{\mu}}. (1.2) Among the best-known instances of stability results in Bayesian inverse problems of the form ( 1.2 ) are those from [ 16 ] . For example, the Hellinger metric was used to measure the distance between a reference posterior measure and an approximate posterior measure associated to a perturbed misfit in [ 16 , Theorem 4.6] . In the case of Gaussian observation noise, this stability result can then be used to show stability of the posterior with respect to the data. An important consequence of the stability results in [ 16 ] is that many Bayesian inverse problems of interest are well-posed in the sense of Hadamard, i.e. the posterior depends continuously on the data y . An important reason for the importance of stability with respect to the misfit is that it implies stability with respect to the forward model, under certain assumptions on the observational noise. Stability with respect to the forward model is important in the context of numerical methods for solving Bayesian inverse problems, and more generally when accounting for the consequences of using an incorrect forward model in inference tasks. This is because such stability results ensure that more accurate approximations of the true forward model lead to more accurate approximations of the true posterior, when all other factors are held fixed. Stability with respect to the forward model establishes a link between numerical analysis and approximation theory on one hand, and robust statistical inference on the other hand, in the sense that one can use error bounds for a given numerical method or approximation in order to bound the error in the output of a statistical procedure based on that method or approximation. Although the constants in these bounds are often difficult or costly to evaluate in practice, the bounds nevertheless provide a more precise characterisation of the continuous dependence of the inference outcome on numerical or modelling approximations. An important milestone in the stability analysis of Bayesian inverse problems is given in the work [ 15 ] , which presents stability results of the posterior with respect to misfit perturbations and prior perturbations. Perturbations in the posterior are measured in the Kullback–Leibler divergence and the Hellinger, total variation, and 1-Wasserstein metrics. Stability with respect to the prior is important because the choice of prior determines the maximal support of the resulting posterior, and thus the set of statistically admissible parameter values. Furthermore, it is well-known from the mathematical-statistical theory of nonparametric Bayesian inference that the prior plays a key role in posterior consistency and contraction; see e.g. [ 7 , Chapter 6] . 1.1 Contents of this work The goal of this work is to further investigate the bounds and ideas relating to local Lipschitz stability results for the misfit-to-posterior and prior-to-posterior maps that were presented in [ 15 ] . To this end, we prove upper and lower bounds on Kullback–Leibler divergences and Hellinger, total variation, and 1-Wasserstein metrics between posterior measures associated to different misfits and priors. Recalling that \mathfrak{d}_{\mathcal{P}(\Theta)} denotes a chosen metric or divergence on the space of probability measures on \Theta , the upper bounds on posteriors that we state below are of the form \displaystyle\mathfrak{d}_{\mathcal{P}(\Theta)}(\mu_{\Phi_{1}},\mu_{\Phi_{2}})\leq \displaystyle C_{1}\mathcal{U}_{M}(\Phi_{1}-\Phi_{2},Z_{\Phi_{1},\mu},Z_{\Phi_{2},\mu}) (1.3a) \displaystyle\mathfrak{d}_{\mathcal{P}(\Theta)}((\mu_{1})_{\Phi},(\mu_{2})_{\Phi})\leq \displaystyle C_{2}\mathcal{U}_{P}(\mathfrak{d}_{\mathcal{P}(\Theta)}(\mu_{1},\mu_{2}),Z_{\Phi,\mu_{1}},Z_{\Phi,\mu_{2}}), (1.3b) where C_{1}=C_{1}(\mu,\Phi_{1}) and C_{2}=C_{2}(\mu_{1},\Phi) are positive scalars. In ( 1.3a ), we consider posteriors resulting from perturbed misfits and the same prior \mu , while in ( 1.3b ), we consider posteriors resulting from different priors \mu_{1} and \mu_{2} and the same misfit \Phi . The nonnegative functions \mathcal{U}_{M} and \mathcal{U}_{P} depend on the choice of \mathfrak{d}_{\mathcal{P}(\Theta)} and converge to zero as \Phi_{1}-\Phi_{2} converges to zero in an appropriate norm and as \mathfrak{d}_{\mathcal{P}(\Theta)}(\mu_{1},\mu_{2}) converges to zero respectively. If \mathfrak{d}_{\mathcal{P}(\Theta)} satisfies the triangle inequality, then the upper bounds above can be combined to obtain an upper bound on \mathfrak{d}_{\mathcal{P}(\Theta)}((\mu_{1})_{\Phi_{1}},(\mu_{2})_{\Phi_{2}}) , i.e. on the measure of discrepancy when both the prior and misfit are perturbed. While some of our upper bounds appear implicitly in the proofs of results in [ 15 ] , others do not. We seek upper bounds with explicit dependence on the evidence, because the evidence plays a crucial role in determining the behaviour of the posterior, by Bayes’ formula ( 1.1 ). The second main class of results that we state below are lower bounds on the above-mentioned metrics or divergences between posteriors, where perturbed misfits and