Likelihood-informed dimension reduction across tempered Bayesian posteriors
Arne Bouillon, Oliver R. A. Dunbar · 2026
Read Likelihood-informed dimension reduction across tempered Bayesian posteriors by Arne Bouillon, Oliver R. A. Dunbar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21717.txt.
Introduction
Arne Bouillon and Oliver Dunbar extend likelihood-informed subspaces to tempered Bayesian posteriors, annealed forms used when data are scarce. These subspaces keep only telling directions, but prove restrictive in practice. Their partly informed alpha-LIS spaces often reach near-optimal quality even when alpha falls below one. They also build extensions using data from the whole tempered sequence and simple gradient stand-ins for chaotic or stochastic systems. In experiments their full-sequence approach proves far more robust than the optimal alpha-equals-one choice.
Catalog record
| CREATOR | Arne Bouillon, Oliver R. A. Dunbar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106213 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.21717
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Likelihood-informed dimension reduction across tempered Bayesian posteriors Arne Bouillon, Oliver R. A. Dunbar Abstract Scientific computer simulations cannot represent all scales in realistic applications. To bridge this model-data gap, parameters are injected into models and constrained with noisy data using Bayesian inversion. To reduce the number of simulator evaluations, which can be 10 5 10^{5} or more, modern approaches employ dimension reduction in conjunction with emulation of the forward map (that contains the simulator). Due to scarcity of model evaluations and data, this dimension reduction becomes very important for posterior sampling performance. Recent work on likelihood-informed subspaces (LIS) truncates to informative directions by optimizing bounds on information loss, and though mathematically well-adapted to sampling, they are often restrictive in practice. In this work, we provably generalize this methodology to facilitate application to α \alpha -tempered (i.e., annealed , power-posterior ) distributions for α ∈ [ 0 , 1 ] \alpha\in[0,1] . We provide theory to build partially-informed spaces termed α \alpha -LIS. We show how α < 1 \alpha<1 can often produce near-optimal spaces. In addition, we focus on applying α \alpha -LIS to practical cases, where the available data is severely limited and noisy. We propose and test extensions for utilizing data from the entire sequence of distributions α 0 < ⋯ < α k \alpha_{0}<\dots<\alpha_{k} , and use simple approximations of model gradients so that our approach can be used for emulation of forward maps for chaotic or stochastic systems where derivatives are unavailable or uninformative due to noise. In experiments, our accumulated approach is much more robust to these challenging circumstances than the theoretically optimal α = 1 \alpha=1 . 1 Introduction The success of machine learning and over-parameterization has allowed new closures and data-driven representations of unresolved processes in scientific computer simulators. However, with these high-dimensional parameters, the curse of dimensionality makes calibration (finding optimal parameters to fit data) and uncertainty quantification (UQ, sampling a joint distribution for likely parameters) challenging. We must use algorithms that scale well to a high-dimensional setting. Calibration and UQ algorithms require evaluations of the simulators embedded within their loss or likelihood functions. Scientific simulators are frequently expensive black-box software, with a limited number of evaluations available. In these situations, robust dimension reduction techniques (as a way of pre-processing) play an important yet underrepresented role in algorithm performance. Posterior sampling for such models commonly requires in excess of 10^{5} model evaluations with, e.g., Markov chain Monte Carlo (MCMC) [ 32 ] . To offset this, models can be replaced with surrogates that have more amenable properties: they are cheaper, lower-dimensional, or smoother. This includes multi-fidelity or multi-level approaches [ 37 , 10 , 39 ] and data-driven emulators of either the model [ 38 , 50 , 30 , 11 ] (a focus of our investigation) or the loss/likelihood function [ 58 , 70 , 36 ] . In data-driven emulation, a design question arises: how does one select input-output pairs to train an emulator, without falling victim to the curse of dimensionality? Goal-independent strategies such as space-filling designs, Latin-hypercube sampling, and sparse grids [ 54 , 15 ] may suffer in high-dimensional problems. Meanwhile, other methods show more success by addressing a goal-oriented design problem with active or sequential approaches [ 69 , 11 , 71 , 40 ] , and some of these are themselves enhanced with emulators (e.g., Bayesian history matching [ 22 , 51 ] ). In the recent calibrate, emulate, sample approach [ 11 , 23 ] , the authors propose to address the training point design for UQ with variants of ensemble Kalman inversion (EKI) [ 8 , 45 , 67 , 7 , 43 , 6 , 26 ] . EKI is an optimization and sampling method that scales to high dimensions with relatively few evaluations and can handle complex loss functions [ 33 , 52 ] . The strict assumptions (linearity and Gaussianity) underlying the updates of EKI make it unsuitable for accurate posterior sampling, but it can efficiently place input-output training pairs in regions of high posterior mass. The heuristic is that resulting emulators have high accuracy in these regions, where they most often make predictions within a sampling algorithm. Despite this tailored sampling, training the high-dimensional emulator is still challenging: only about 10^{2} – 10^{3} training pairs are realistically available, and emulators such as Gaussian processes (GPs) are expensive in high dimensions. Motivated by this final challenge, we will investigate the most efficient use of samples generated from a sequence of approximating distributions for the purpose of dimension reduction in sampling. We consider the task of posterior sampling and, later, emulator-accelerated posterior sampling, and measure performance in terms of distances between the reduced and full posterior distributions. Our algorithms, based on optimization of information-theoretic bounds on the posterior, are known as likelihood-informed subspace (LIS) methods, with data-based (DB) and data-free (DF) variants [ 16 , 1 , 19 , 18 ] . The information-theoretic approach is quite general, and is directly linked to active subspaces for inference [ 13 , 12 , 63 ] and prior-preconditioned Hessian approaches [ 29 , 57 ] . It naturally generalizes canonical correlation analysis (CCA)—see, e.g., [ 1 ] . A conceptually related approach for dimension reduction of input and output variables—however, in linear dynamical systems instead of Bayesian inverse problems—is known as balanced truncation [ 59 , 60 ] . It was specialized to the case of Bayesian inversion in [ 65 ] . New work also directly investigates the differences in prior-to-posterior covariance [ 64 ] , which is suitable for intractable likelihoods. Motivated by sequential ensemble methodologies, we construct a flexible generalization of current LIS methods to any of the 1-parameter family of \alpha -tempered posteriors ( \alpha\in[0,1] ). We prove that this generalizes DB-LIS ( \alpha=1 , linked to the posterior distribution) and DF-LIS ( \alpha=0 , linked to the prior), and term this generalization \alpha -LIS. It can be defined for any \alpha as a linear map from the full to a reduced space, optimizing an upper bound on the Kullback–Leibler divergence between the reduced and full posterior. We demonstrate its interpolative properties over a series of numerical studies to illustrate algorithm performance, when one has draws from distributions that are neither the prior ( \alpha=0 ) nor the posterior ( \alpha=1 ). The numerical studies consider different data sample sizes, for linear and nonlinear problems, with direct sampling or by first training an emulator. The nonlinear problems include a PDE simulator (solving Darcy flow inversion) and a chaotic Lorenz ‘96 model, solving an inversion for a spatially-dependent forcing. Beyond proof-of-theory numerics, we focus on applicability to practical, highly constrained situations. We propose practical extensions for some key limitations in current LIS methodologies. • First, we propose how one can compute gradient-free approximations of the \alpha -LIS subspaces, based on available samples. This addresses the fact that many models must be treated without gradients, as these might be unavailable (e.g., in a legacy code) or uninformative (such as for chaotic or stochastic simulators). Our simple approximation is to take a global statistical linearization over the samples (as presented in, e.g., [ 7 , 72 , 6 ] ). Other gradient-free