Hierarchical Gaussian Random Fields for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and The Karhunen-Loève Decomposition

Sohail Reddy · 2025

Read Hierarchical Gaussian Random Fields for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and The Karhunen-Loève Decomposition by Sohail Reddy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13830.txt.

Introduction

Estimating unknown fields from scattered data often leans on Gaussian random fields, but sampling them in high dimensions is costly. Sohail Reddy proposed in 2025 a layered sampling scheme that joins two classic tools to cut the cost.

One tool builds fields from a differential equation and scales well; the other compresses fields into a short expansion but can distort sampling. Reddy's method builds a hierarchy from a multigrid ladder, preserving the field's covariance at every level. Coarse-to-fine sampling then stays faithful to the target distribution. On a test problem of underground flow, the method beat existing ones in speed, accuracy, and convergence, the author reports.

Catalog record

CREATORSohail Reddy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13830

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hierarchical Gaussian Random Fields for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and The Karhunen-Loève Decomposition Sohail Reddy Abstract This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRF) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equation (SPDE), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov Chain Monte Carlo (MCMC) sampler, and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of stochastic partial differential equation (SPDE)-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable, accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Our proposed technique is more efficient, accurate, and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems. A Preprint 1 Introduction Gaussian random fields (GRFs) play a pivotal role in modeling and understanding spatially correlated phenomena across various domains, including geostatistics [ 1 ] , physics [ 2 ] , and machine learning [ 3 ] . They provide a robust framework for encoding structural information about spatial fields, including smoothness and periodicity, through covariance functions or kernels. Markov chain Monte Carlo (MCMC) [ 4 ] methods frequently intersect with GRFs in Bayesian inference and uncertainty quantification (UQ)where GRFs are often employed as priors and proposal distributions for spatial fields in inverse problems [ 5 ] . The integration of GRFs and MCMC allows for the systematic incorporation of observational data and prior knowledge into statistical models, but it also introduces computational bottlenecks. These bottlenecks are particularly pronounced in high-dimensional problems, where the cost of generating GRF samples and evaluating forward models is substantial. Efficient sampling of GRFs is a cornerstone of their utility, especially in high-dimensional settings where computational demands are significant. Traditional sampling methods, such as the Cholesky decomposition [ 6 ] and Karhunen-Loève (KL) expansions [ 7 ] , are well-suited for lower dimensional problems. For instance, the KL expansion requires the eigendecomposition of a dense covariance matrix, which can be tractable for lower dimensional sampling but often infeasible for high-dimensional, large-scale problems, thereby necessitating the need for approximations - such as low-rank, randomized matrix [ 8 ] , hierarchical matrix [ 9 ] , and Rayleigh-Ritz [ 10 ] methods. To alleviate these challenges, scalable alternatives such as circulant embedding [ 11 ] and stochastic partial differential equation (SPDE)-based sampling [ 12 , 13 ] have often been preferred for large scale problems. The SPDE approach, in particular, exploits significant advances in scalable, numerical techniques for solving PDEs, such as higher order finite elements, preconditioning strategies, scalable linear algebra, with the GRF sample being realized as the solution to the SPDE. While surrogate models, in the form of Gaussian processes [ 14 , 15 ] , polynomial chaos expansions [ 16 , 17 ] , and deep neural networks [ 18 , 19 ] , have greatly reduced the computation cost of the forward problem [ 20 ] , they introduce approximation errors that bias the sampling. This, combined with the computational expense of Bayesian inference in high-dimensional sample space, makes inverse problems particularly challenging when using MCMC techniques. Recent advancements in hierarchical GRF sampling have emerged as promising solutions to address these challenges, offering a balance between computational efficiency and accuracy. These hierarchical methods decompose the sampling process across multiple levels of resolution, enabling a coarse-to-fine representation of the field where the sampling on the less expensive, coarser resolutions is used to inform the sampling on the more expensive and accurate, finer resolution, improving the efficiency of the MCMC sampler and accelerating the convergence of the statistical moment estimation. The hierarchical MCMC sampling techniques have been extended and combined with variance-decay techniques, such as multilevel Monte Carlo (MLMC) sampling [ 21 , 22 ] , to accelerate estimation of statistical moments using multilevel samples. Naturally, such methods require a hierarchical decomposition of the GRF that is consistent across all levels, to ensure that the samples on each level are drawn from the correct distribution (i.e. the decomposition preserves the structure of the covariance). Such a hierarchical decomposition based on multigrid theory has recently been developed for SPDE-based sampling, and has shown promise in improving the acceptance rate across the levels and the multilevel estimate of the statistical moments [ 23 ] . Hierarchical methods based solely on multigrid decomposition for SPDE sampling face challenges in balancing sample space reduction with discretization accuracy. Coarsening the resolution (i.e. mesh) reduces computational costs but may introduce errors that degrade the quality of GRF realizations and forward model evaluations. Hierarchical sampling via KL expansions offers a potential solution and can achieve high discretization accuracy by utilizing a finer (nodal) resolution (i.e. mesh) for the forward problem, while reducing dimensionality of the sample space by coarsening in modal space. However, hierarchical sampling based on the KL expansion has been limited to covariance kernels - such as exponential covariance [ 24 , 25 , 26 , 27 , 28 ] - for which the analytical form of the KL expansion is known. This not only circumvents the computationally expensive eigendecomposition of the covariance matrix but also ensures the consistent ordering of the KL modes across all levels. Then, the sample space (i.e. number of terms in the expansion) can be held constant across all levels [ 28 ] or varied on each level [ 25 , 26 , 27 ] according to appropriate error estimations for the truncation [ 29 ] . While discrete approximations of the KL modes can be computed for any arbitrary discrete covariance matrix, coupling KL sampling across different levels in a hierarchy requires additional work since the coefficients of the modes on one level do not necessarily correspond to those of the modes on another level. Furthermore, ensuring ergodicity of the sampling on each level requires the set of KL modes to be complete, which is intractable to compute and store in memory for fine resolutions. Hence, a hierarchical approach that performs dimensionality reduction of the sample space (i.e. use a truncated KL expansion) without sacrificing the discretization accuracy or ergodicity of the sampler is highly sought after. We address this challenge and develop a hierarchical modal-multigrid decomposition that couples the discretization accuracy and dimensionality reduction of the KL sampling with the scalability and ergodicity-preservation of the SPDE sampler. We derive decompositions that preserve the structure of the covariance matrix and ensure the samples on each level are drawn from the correct distribution. The decomposition

Read the complete text