Detecting Model Misspecification in Bayesian Inverse Problems via Variational Gradient Descent
Qingyang Liu, Matthew A. Fisher, Zheyang Shen, Katy Tant, Xuebin Zhao, Andrew Curtis, Chris. J. Oates · 2025
Read Detecting Model Misspecification in Bayesian Inverse Problems via Variational Gradient Descent by Qingyang Liu, Matthew A. Fisher, Zheyang Shen, Katy Tant, Xuebin Zhao, Andrew Curtis, Chris. J. Oates (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01667.txt.
Introduction
Qingyang Liu and co-authors detect model flaws in Bayesian inverse problems. Bayesian inference is optimal when the model fits. When it does not, results can fail badly. Many post-Bayesian fixes have been offered for that risk.
Their method uses variational gradient descent to spot misspecification. Predictively oriented views lift the model itself during checks. A clear flaw alarm lets users fix the model before trusting its answers. That guard matters in imaging and geophysics, where inverse answers guide costly acts. The check runs after fitting, so it guards real workflows where a flawed model has already shaped the data analysis once.
Catalog record
| CREATOR | Qingyang Liu, Matthew A. Fisher, Zheyang Shen, Katy Tant, Xuebin Zhao, Andrew Curtis, Chris. J. Oates |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01667
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Detecting Model Misspecification in Bayesian Inverse Problems via Variational Gradient Descent Qingyang Liu, Matthew A. Fisher, Zheyang Shen, Katy Tant, Xuebin Zhao, Andrew Curtis, Chris. J. Oates Abstract Bayesian inference is optimal when the statistical model is well-specified, while outside this setting Bayesian inference can catastrophically fail; accordingly a wealth of post-Bayesian methodologies have been proposed. Predictively oriented (PrO) approaches lift the statistical model P θ P_{\theta} to an (infinite) mixture model ∫ P θ 𝑑 Q ( θ ) \int P_{\theta}\;\mathrm{d}Q(\theta) and fit this predictive distribution via minimising an entropy-regularised objective functional. In the well-specified setting one expects the mixing distribution Q Q to concentrate around the true data-generating parameter in the large data limit, while such singular concentration will typically not be observed if the model is misspecified. Our contribution is to demonstrate that one can empirically detect model misspecification by comparing the standard Bayesian posterior to the PrO ‘posterior’ Q Q . To operationalise this, we present an efficient numerical algorithm based on variational gradient descent. A simulation study, and a more detailed case study involving a Bayesian inverse problem in seismology, confirm that model misspecification can be automatically detected using this framework. 1 Introduction Detecting and mitigating model misspecification is a pertinent but difficult issue in Bayesian statistics, where one must consider the full posterior predictive distribution in lieu of e.g. plotting a simple scalar residual. The two main approaches to detecting model misspecification (also called model criticism ) are: 1. Predictive : Compare the posterior predictive distribution to held out entries from the dataset ( Gelman et al. 1996 ; Bayarri and Berger 2000 ; Walker 2013 ; Moran et al. 2024 , e.g.) . 2. Comparative : Perform model selection over a set of candidates models in which the current model is contained ( Kass and Raftery 1995 ; Wasserman 2000 ; Kamary et al. 2014 , e.g.) . In the first case, if the held out data are in some sense ‘unexpected’ under the posterior predictive distribution, this serves as evidence that either the prior or the model may be misspecified. In the second case, if the data strongly support an alternative model, this suggests that the original model may be misspecified. Of course, this is not a true dichotomy and the predictive and comparative approaches are intimately related; for instance, predictive performance is a common criteria for selecting a suitable model ( Piironen and Vehtari 2017 ; Fong and Holmes 2020 ) . For challenging applications, both of these approaches can be impractical. An example of such a challenging application is seismic travel time tomography ( Curtis and Snieder 2002 ; Zhang and Curtis 2020 ; Zhao et al. 2022 ) , a regression task where one seeks to reconstruct a subsurface seismic velocity field using measured first arrival times of seismic waves travelling between seismic source and sensor (seismometer) locations. Evaluation of the likelihood and/or its gradient here is associated with a nontrivial computational cost, since the Eikonal equation , a partial differential equation describing the high frequency approximation of the wave equation, needs to be solved to estimate the travel times of the first arriving seismic waves. It is worth pointing out that ‘misspecification’ as discussed in this work refers to the suitability of the statistical model for the dataset, rather than any errors that may be present in the physical model for seismic wave travel, though the two issues are of course closely related. Considering the predictive approach in the seismic tomography context, the spatiotemporal nature of the observations renders the data strongly dependent, representing a challenge in constructing a suitable held-out dataset. Further, while in principle there are solutions to cross-validation for dependent data ( Burman et al. 1994 ; Rabinowicz and Rosset 2022 , e.g.) , the computational cost associated with performing multiple folds of held-out prediction can render this approach impractical. Focussing instead on the comparative approach, constructing plausible alternative models requires modifying the physical assumptions underlying the seismic wave propagation. This in turn requires the implementation and optimisation of suitable numerical methods, which demands considerable effort. In the case of model misspecification, there is particular interest in exploring more sophisticated models, but without advanced physical insight, constructing such alternatives might be impractical. The aim of this paper is to propose a simple, practical and general approach to detecting model misspecification in Bayesian statistics , and we focus on the seismic travel time tomography problem to demonstrate its potential. Our solution can be considered both a predictive and a comparative approach; predictive because we explicitly assess the predictive performance of the model, and comparative because our principal innovation is to automate the generation of a candidate model set. To draw an analogy, recall the seminal paper of Kennedy and O’Hagan 2001 , where the authors propose augmenting a misspecified physics-based parametric regression function f_{\theta}(x) with a nonparametric component, i.e. f_{\theta}(x)+g(x) where g(x) and the parameter \theta are to be jointly inferred. In doing so, Kennedy and O’Hagan are in effect automatically generating a set of alternative models without requiring additional physical insight. The approach has been generalised beyond additive misspecification ( Alvarez et al. 2013 , e.g. to misspecified differential equation models;) . However, the drawbacks of this approach are twofold; (i) introducing a nonparametric component increases the data requirement, (ii) any causal predictive power of the model is lost, since the behaviour of the nonparametric component g(x) under intervention cannot be inferred. As such, an alternative approach to automatic generation of candidate models is required. Our contributions are as follows: • Automatic generation of candidate models: Following similar ideas, in particular nonparametric maximum likelihood ( Laird 1978 ) , we lift the statistical model P_{\theta} to an (infinite) mixture model P_{Q}:=\int P_{\theta}\;\mathrm{d}Q(\theta) as a mechanism to generate an infinite candidate set of alternative models (parametrised by Q ), while retaining any causal semantics present in the original statistical model. • A predictively oriented ‘posterior’: Following the prediction oriented (PrO) approaches of Masegosa 2020 ; Sheth and Khardon 2020 ; Jankowiak et al. 2020a ; Jankowiak et al. 2020b ; Morningstar et al. 2022 ; Lai and Yao 2024 ; Shen et al. 2025 ; McLatchie et al. 2025 , learning of the mixing density Q proceeds based on the performance of the predictive distribution P_{Q} (in this sense our method is a predictive approach) and is cast as an optimisation task, with the solution being a predictively oriented ‘posterior’ denoted Q_{\mathrm{PrO}} . • Detecting misspecification: We propose a direct comparison of the predictive distributions associated with Q_{\mathrm{PrO}} and the Bayesian posterior Q_{\mathrm{Bayes}} as a general strategy enabling model misspecification to be detected (in this sense our method is a comparative approach). Indeed, McLatchie et al. 2025 proved that in the well-specified setting the predictive distribution associated to Q_{\mathrm{PrO}} converges to the true data-generating distribution in the large data limit, in agreement with the predictive distribution associated to Q_{\mathrm{Bayes}} , while such agreement is typically not observed when the model is misspecified. • Computation as variational gradient descent: Since Q_{\mathrm{PrO}} is defined as the minimiser of a nonlinear variational objective, we do not have access to an unnormalised form of the target, and as