Semi-Implicit Approaches for Large-Scale Bayesian Spatial Interpolation

Sébastien Garneau · 2025

Read Semi-Implicit Approaches for Large-Scale Bayesian Spatial Interpolation by Sébastien Garneau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19722.txt.

Introduction

Sébastien Garneau, Carlos T. P. Zanini, and Alexandra M. Schmidt propose Co-SIVI, a faster way to do Bayesian inference for large spatial models. Standard semi-implicit variational inference approximates the posterior distribution cheaply, but it can miss dependence when spatial random effects are modeled explicitly. Their correlated version builds dependence directly into the variational distribution of those effects.

An iterative weighted least squares scheme handles both Gaussian process and nearest-neighbor Gaussian process priors, and a reparameterization helps capture how covariance parameters co-vary. In simulations with four kinds of outcome data, Co-SIVI matched the gold-standard Hamiltonian Monte Carlo results at much lower cost. Applications to large temperature and housing price datasets confirm the approach scales to real problems.

Catalog record

CREATORSébastien Garneau
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19722

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semi-Implicit Approaches for Large-Scale Bayesian Spatial Interpolation Sébastien Garneau Abstract Spatial statistics often rely on Gaussian processes (GPs) to capture dependencies across locations. However, their computational cost increases rapidly with the number of locations, potentially needing multiple hours even for moderate sample sizes. To address this, we propose using Semi-Implicit Variational Inference (SIVI), a highly flexible Bayesian approximation method, for scalable Bayesian spatial interpolation. We evaluated SIVI with a GP prior and a Nearest-Neighbour Gaussian Process (NNGP) prior compared to Automatic Differentiation Variational Inference (ADVI), Pathfinder, and Hamiltonian Monte Carlo (HMC), the reference method in spatial statistics. Methods were compared based on their predictive ability measured by the CRPS, the interval score, and the negative log-predictive density across 50 replicates for both Gaussian and Poisson outcomes. SIVI-based methods achieved similar results to HMC, while being drastically faster. On average, for the Poisson scenario with 500 training locations, SIVI reduced the computational time from roughly 6 hours for HMC to 130 seconds. Furthermore, SIVI-NNGP analyzed a simulated land surface temperature dataset of 150,000 locations while estimating all unknown model parameters in under two minutes. These results highlight the potential of SIVI as a flexible and scalable inference technique in spatial statistics. Keywords: Dimension Reduction, Exponential Family, Gaussian Process, Variational Inference 1 Introduction Easier access to satellite data and Geographic Information Systems (GIS) has led to an increase in geographically referenced data. Such data naturally occur in a variety of fields, such as epidemiology, public health, and environmental sciences ( Yu et al. 2017 , Blangiardo et al. 2016 , Rodriguez-Idiazabal et al. 2025 ) . The core principle of analyzing spatial data can be summarized by Tobler’s first law of geography: “everything is related to everything else, but near things are more related than distant things". While this law is a simplification, it captures the fundamental notion of spatial dependence, which is at the heart of spatial statistics. Gaussian processes (GPs) have been used predominantly to capture such dependence. A GP is a stochastic process for which any finite set of random variables follows a multivariate normal distribution, with mean given by the GP’s mean function and covariance given by the GP’s covariance function. The covariance function specifies how the correlation between locations decays with distance. However, the use of a GP comes with an important drawback: evaluating the likelihood of an n -dimensional GP requires \mathcal{O}(n^{3}) operations and \mathcal{O}(n^{2}) memory. Hence, models using a GP become computationally intensive even for moderately large datasets. Proposed solutions to this problem include stochastic partial differential equations ( Lindgren et al. 2011 ) , low-rank approximations ( Banerjee et al. 2008 ) , and using conditional independence structures ( Vecchia 1988 ) , among others. Variational inference (VI), a Bayesian technique that aims to approximate the posterior by a distribution in a pre-specified family of variational distributions, provides a computationally efficient alternative to Markov Chain Monte Carlo (MCMC) for large-scale models that include a GP component, such as those commonly used in spatial statistics. Among existing VI methods for spatial data, most are restricted to Gaussian outcomes ( Ren et al. 2011 , Song & Datta 2025 ) , or require specific approximations to be developed to handle specific outcome types ( Lee & Lee 2025 ) . Therefore, there remains a need for VI methods that could theoretically be flexible enough to accommodate a wide range of outcome types. In this paper, we propose the use of Semi-implicit variational inference (SIVI) ( Yin & Zhou 2018 ) for Bayesian geostatistical models. We evaluate SIVI through simulation studies where we compare it to automatic differentiation variational inference (ADVI) ( Kucukelbir et al. 2017 ) and Pathfinder ( Zhang et al. 2022 ) , two commonly used VI algorithms, and a modified Hamiltonian Monte Carlo (HMC) ( Neal 2011 ) algorithm known as the No-U-Turn sampler (NUTS) by Hoffman et al. 2014 . To further improve computational efficiency, we also consider the combination of SIVI with the nearest-neighbor Gaussian process (NNGP) ( Datta et al. 2016 ) prior. The proposed SIVI-based approaches only require the log joint distribution of the observed data and model parameters to be differentiable, so they apply to a broad range of model specifications. In this sense, we explore Gaussian and Poisson outcomes in detail to exemplify the proposed methodology. The remainder of this paper is organized as follows. In Section 2 , we review the Bayesian framework for geostatistics and the NNGP approach. Section 3 briefly introduces variational inference methods, focusing on the variational methods used in comparison to the proposed methods. Section 4 presents the proposed variational inference procedures in the context of Bayesian geostatistical models. The simulation studies are described in Section 5 . We then illustrate the proposed methods on a large-scale dataset in Section 6 . Finally, Section 7 concludes the paper with a discussion of the findings and potential areas for future research. 2 Geostatistics In geostatistics, also referred to as point-referenced data, observations are collected at a set of n fixed locations \mathcal{S}=\{s_{1},\ldots,s_{n}\} within a region of interest \mathcal{D}\subset\mathbb{R}^{d} , where typically d=2 or d=3 . At each location s\in\mathcal{S} , let y(s) denote the observed outcome and let \mathbf{x}(s) be a p\times 1 vector of covariates with associated p\times 1 vector of coefficients \bm{\beta} . Following Diggle et al. 1998 , a general model for point-referenced observations within the region \mathcal{D} can be expressed as h(\mathbb{E}[y(s)\mid w(s)])=\mathbf{x}(s)^{\top}\bm{\beta}+w(s),\quad s\in\mathcal{D}, (1) where h(\cdot) is a known link function and w(s) is a spatial random effect that captures the spatial dependence of observations at distinct locations. Observed data \mathbf{y}=(y(s_{1}),\ldots,y(s_{n}))^{\top} are assumed to be conditionally independent across locations s_{1},\ldots,s_{n} and to follow a distribution in the exponential family given \mathbf{w}=(w(s_{1}),\ldots,w(s_{n}))^{\top} . The process \{w(s):s\in\mathcal{D}\} is typically modeled as a Gaussian process with zero mean function and covariance function C(s_{i},s_{j})=Cov(w(s_{i}),w(s_{j})) , in which case we denote w(s)\sim\text{GP}(0,C) . Throughout this paper, we adopt the exponential covariance function given by C(s_{i},s_{j})=\sigma^{2}exp(-\phi\ d(s_{i},s_{j})) , where \sigma^{2}>0 is the marginal variance of the spatial process, also referred to as the partial sill, d(s_{i},s_{j}) is the Euclidean distance between locations s_{i} and s_{j} , and \phi>0 is the range parameter controlling the strength of the spatial correlation as distance between locations increases. In this paper, we focus on two specific examples in the exponential family of distributions for \mathbf{y} : the Gaussian (section 2.1 ) and the Poisson (section 2.2 ). In a Bayesian framework, a prior distribution is assigned to all the unknown parameters \bm{\theta} to complete the model specification. For instance, in the Poisson case, \bm{\theta}=\{\bm{\beta},\sigma^{2},\phi,\mathbf{w}\} . Common choices include independent Gaussian priors for the regression coefficients \bm{\beta} , inverse-gamma priors for the variance parameters \sigma^{2} and \tau^{2} (see Gelman 2006 for alternatives), and either an inverse-gamma or a uniform prior for \phi . 2.1 Gaussian Spatial Model First, we consider the Gaussian case, where the multivariate normal likelihood of observed data \mathbf{y} can be expressed in two equivalent forms. The first one, referred to as the conditional model, keeps the random effects vector \mathbf{w} in the model such that \mathbf{y}\mid\bm{\beta},\mathbf{w},\tau^{2}\sim\text{MVN}(\mathbf{X}\bm{\beta}+\mathbf{w},\tau^{2}\mathbf{I}_{n}) , where \mathbf{X} is the n\times p matrix with its i -th row representing the observed vector of covariates

Read the complete text