Data-informed posterior approximation for Bayesian linear inverse problems

Haibo Li · 2026

Read Data-informed posterior approximation for Bayesian linear inverse problems by Haibo Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20770.txt.

Introduction

Haibo Li develops a data-informed design for posterior approximation in Bayesian linear inverse problems. Computing posteriors in large-scale settings is hard because the parameter space has many dimensions. His approach shifts focus from that space to data space, describing an intrinsically low-dimensional data space and its isometric embedding into that space. The prior-to-posterior update is then confined to a data-led subspace, and a quotient-space Golub-Kahan bidiagonalization method builds data-led Krylov subspaces for matrix-free use.

Catalog record

CREATORHaibo Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102207
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20770

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Data-informed posterior approximation for Bayesian linear inverse problems Haibo Li Abstract Computing posterior distributions in large-scale Bayesian linear inverse problems is challenging due to the high dimensionality of the parameter space. In this work, we develop a data-informed framework that shifts the computational focus from the parameter space to the data space. We rigorously characterize an intrinsically low-dimensional data space, establish its isometric embedding into the parameter space, and show that the prior-to-posterior update is confined to a data-informed subspace. This perspective allows posterior inference to be carried out in a reduced data-informed subspace. Based on this formulation, we propose a quotient-space Golub–Kahan bidiagonalization method to construct data-informed Krylov subspaces, and integrate empirical Bayesian inference into the iterative framework, enabling simultaneous hyperparameter estimation and posterior approximation in a matrix-free manner. Numerical experiments on representative problems support the theoretical framework and demonstrate the effectiveness of the resulting method. {keywords} Bayesian inverse problem, data-informed subspace, posterior approximation, quotient space, Krylov subspace method, Golub–Kahan bidiagonalization 1 Introduction Inverse problems arise across a diverse spectrum of scientific and engineering disciplines, including image reconstruction, computed tomography, geoscience, data assimilation and so on [ 16 , 20 , 3 , 23 , 38 ] . The goal is to recover an unknown parameter or function from indirect measurements contaminated by noise. For linear inverse problems, upon discretization of the underlying continuous forward operator, these problems are typically formulated as a high-dimensional linear system: y=Gx+\eta,\quad\eta\sim\mathcal{N}(0,\,\Gamma), (1) where x\in\mathbb{R}^{n} represents the discretized unknown, y\in\mathbb{R}^{m} is the observation data, and G\in\mathbb{R}^{m\times n} is the forward matrix. The noise \eta is assumed to follow a Gaussian distribution with a symmetric positive definite covariance \Gamma\in\mathbb{R}^{m\times m} . A fundamental challenge in solving such systems is the ill-posed nature of the inverse problem, which means that there may be multiple solutions that fit the observation data equally well, or the solution is very sensitive with respect to the observation noise. To address the ill-posedness, two predominant frameworks are widely employed: deterministic Tikhonov regularization approach and probabilistic Bayesian inference approach. The former is based on minimizing a loss function \ell(Gx,y) added by a regularizer R(x) that incorporates prior information about x [ 46 , 11 , 17 ] . In contrast, the Bayesian framework treats x and y as random variables, providing a complete characterization of the solution through the posterior distribution x\,|\,y via Bayes’ theorem [ 20 , 45 ] . This approach has gained significant traction due to its ability to provide a rigorous quantification of the underlying uncertainty. The two methodologies are connected through the maximum a posteriori (MAP) estimator, where the mean of the posterior distribution corresponds to the solution of a specific Tikhonov regularization problem [ 20 , 10 , 18 ] . To reconstruct the unknown, assume that x has a Gaussian prior \mathcal{N}(0,\lambda^{-1}\Sigma) independent of the noise \eta , where \Sigma is a symmetric positive definite matrix, and \lambda>0 is a hyperparameter governing the prior variance level. In some applications, \lambda may be known. Quite often, however, it is unknown and we assume that is the case here. According to Bayes’ rule, the prior density and the likelihood function induced by the forward model together yield the posterior probability density: \pi(x\,|\,y)\propto\pi(x)\,\pi(y\,|\,x)\propto\exp\left(-\frac{1}{2}\|Gx-y\|_{\Gamma^{-1}}^{2}-\frac{\lambda}{2}\|x\|_{\Sigma^{-1}}^{2}\right), (2) where \|x\|_{B}:=(x^{\top}Bx)^{1/2} is the B -norm of x for a positive definite matrix B . Since both the likelihood and the prior are Gaussian, the posterior distribution is also Gaussian, x\,|\,y\sim\mathcal{N}(x_{\lambda},C_{\lambda}) , with its mean and covariance given by: x_{\lambda}=C_{\lambda}G^{\top}\Gamma^{-1}y,\qquad C_{\lambda}=(G^{\top}\Gamma^{-1}G+\lambda\Sigma^{-1})^{-1}. (3) Specifically, the MAP estimator is the posterior mean, which can be obtained by minimizing the negative log-density of the Bayesian posterior: x_{\lambda}=\mathop{\mathrm{argmin}}_{x\in\mathbb{R}^{n}}\{\|Gx-y\|_{\Gamma^{-1}}^{2}+\lambda\|x\|_{\Sigma^{-1}}^{2}\}, (4) It is evident from equation 4 that the Tikhonov penalty term is explicitly determined by the prior covariance structure, bridging the probabilistic and deterministic perspectives of the inverse problem. In practical large-scale applications, the parameter dimension n is often extremely high, typically with n\gg m , as considered in this paper. Combined with the inherent ill-posedness of the inverse problem, this poses significant challenges for both stability and computational tractability. In particular, a major computational bottleneck is the inversion of the matrix G^{\top}\Gamma^{-1}G+\lambda\Sigma^{-1} to obtain the posterior mean and covariance. A variety of approaches have been proposed for dimension reduction of Bayesian inverse problems. Early work uses the Karhunen–Loève expansion [ 21 , 31 ] to represent the prior distribution in terms of its dominant eigenmodes and truncates the expansion to achieve dimension reduction [ 27 , 33 ] . However, this strategy relies entirely on the spectral properties of the prior and does not account for the influence of the forward operator or the observation data, which restricts the choice of prior models and may lead to significant truncation errors. An alternative approach [ 28 ] constructs reduced subspaces using model-constrained sampling and greedy strategy, but it may fail to capture posterior variability in directions that are not informed by the data. Another approach [ 12 ] approximates the posterior covariance via a low-rank approximation of the Hessian of the log-likelihood. All of the above approaches aim at dimension reduction in the entire parameter space, but do not fully exploit the interaction between the unknown parameter and observation data. In the common setting of inverse problems, the prior distribution encodes smoothness of the unknown parameter, and the observation data are linked to the parameter through a smoothing forward operator. As a consequence, the prior-to-posterior update is often confined to a relatively low-dimensional subspace of the parameter space. Exploiting this structure can enable efficient low-rank approximations of the posterior mean and covariance, leading to substantial computational savings [ 32 , 1 , 2 ] . In [ 9 , 44 ] , the terminology likelihood-informed subspace is proposed to identify such low-dimensional subspaces, which contain the directions along which the posterior differs significantly from the prior. All these methods approximate the posterior covariance through low-rank corrections of the prior covariance, which can provide a more balanced treatment of prior and data information. However, these approaches are still formulated in the parameter space and often rely on explicit factorizations of the prior covariance or its inverse, which can be computationally prohibitive in large-scale settings. In this paper, we shift the focus from the parameter space to the data space. Endowed with the prior-induced geometry, the parameter space is given by the finite-dimensional Hilbert space (\mathbb{R}^{n},\,\langle\cdot,\cdot\rangle_{\Sigma^{-1}}) , which can be extremely high-dimensional if n is very large. In contrast, we introduce the data space as the quotient Hilbert space (\mathbb{R}^{m}/\mathcal{N}(M),\,\langle\cdot,\cdot\rangle_{M}) , with its precise meaning and motivation detailed in Section 2 , where M=G\Sigma G^{\top} is symmetric positive semidefinite and may be rank deficient with \mathcal{N}(M) denoting its null space. The dimension of this space is \mathrm{rank}(M)=\mathrm{rank}(G) , and the smoothing nature of G typically leads to rapidly decaying singular values and hence a much smaller effective dimension. We show that the data space admits an isometric embedding into the parameter space, and that the prior-to-posterior update occurs entirely within the image of this embedding, which has the same dimension as the data space. This structure reveals that the informative directions of the posterior are entirely characterized by a low-dimensional, data-informed subspace that is independent of the ambient parameter dimension. While

Read the complete text