Dynamic Gaussian Processes and the Vanilla-SPDE Exchange
Rui-Yang Zhang, Lachlan Astfalck, Edward Cripps, David Leslie, Henry Moss · 2026
Read Dynamic Gaussian Processes and the Vanilla-SPDE Exchange by Rui-Yang Zhang, Lachlan Astfalck, Edward Cripps, David Leslie, Henry Moss (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31063.txt.
Introduction
Zhang and four colleagues speed up Gaussian process inference, a workhorse method for modeling data that varies over space and time. Exact inference scales with the cube of the number of points, which hurts badly on dense spatio-temporal grids. A reformulation based on stochastic partial differential equations makes time cheap but still cubes in space. It gets worse when observation points and prediction points do not overlap.
Their method is the Vanilla-SPDE Exchange. It exploits an equivalence between the standard and equation-based formulations. The hybrid scheme moves work to whichever side is cheaper for each part of the problem. Complexity analysis and numerical experiments confirm the savings. The result matters for uses like environmental monitoring, where predictions are needed at fixed stations but measurements arrive from elsewhere.
Catalog record
| CREATOR | Rui-Yang Zhang, Lachlan Astfalck, Edward Cripps, David Leslie, Henry Moss |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31063
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Gaussian Processes and the Vanilla-SPDE Exchange Rui-Yang Zhang, Lachlan Astfalck, Edward Cripps, David Leslie, Henry Moss Abstract Gaussian process inference is often limited by cubic computational costs, a challenge that becomes more pronounced in spatio-temporal settings where posterior inference is required over dense grids. While state-space SPDE formulations enable linear complexity in time, exact inference remains cubic in space and deteriorates further when observation locations are disjoint from the prediction locations, which inflates the number of considered spatial points. To address this, we propose the Vanilla-SPDE Exchange , which exploits an equivalence between the standard and SPDE formulations of GP inference to construct a hybrid scheme with improved computational cost. We demonstrate these gains through complexity analysis and numerical experiments. 1 Introduction Gaussian processes (GPs) are stochastic process models widely used for describing spatio-temporal phenomena [ 16 ] due to their expressiveness through flexible covariance function choices and abilities to quantify uncertainties naturally. The standard GP inferences jointly consider all the observation and prediction points for computations such as likelihood evaluations, obtaining posterior distributions, and posterior sampling [ 9 ] . The joint consideration of all relevant points implies a static view of the stochastic process, and induces the cubic scaling of computational complexity in the number of considered points. On the other hand, Stochastic Partial Differential Equation (SPDE) formulations [ 12 ] of certain classes of spatio-temporal Gaussian processes take a dynamic approach to the stochastic process by treating the GP as the stationary solution of a SPDE. Furthermore, these SPDEs admit closed-form transition densities in time, so one can conduct the exact inference sequentially. Thus, the implication of the dynamic formulation is that the computations that scale cubically in complexity under the static formulation, while still scaling cubically in the number of spatial points, only scale linearly in the number of temporal points. However, in many spatio-temporal inference problems, the observation and prediction locations are spatially disjoint. For example, predictions may be required at a fixed set of locations over time, while observations are collected elsewhere. In such settings, the SPDE formulation must include both the observation and prediction locations in the state space. Consequently, the number of spatial locations can increase substantially, and the resulting cubic dependence on the spatial dimension may outweigh the linear-in-time advantage of sequential inference. As a result, SPDE-based inference can become more computationally expensive than standard GP inference. In this work, we propose a new GP inference technique, called the Vanilla-SPDE Exchange (VaSE), that allows the full GP computation to be decomposed into components, each tackled by either the static or dynamic formulation, to obtain the best of both worlds. In particular, for the computation of posterior sampling of spatio-temporal GPs with disjoint observation-prediction locations, the hybrid inference of VaSE regresses the observations with the static method, then predicts and propagates the posterior samples using the dynamic method. This paper is organised as follows. The static Gaussian process inference is described in Section 2 . The dynamic Gaussian process framework and its sequential inference are then explored in Section 3 , which motivates the vanilla-SPDE exchange in Section 4 . Finally, the benefits of posterior sampling using VaSE for settings with disjoint observation-prediction locations are numerically investigated in Section 5 . 2 Static Gaussian Process Inference A Gaussian process (GP) \{f(x)\}_{x\in X} is a stochastic process in which every linear combination \sum_{j}a_{j}f({x_{j}}) , for real coefficients a_{j}\in\mathbb{R} and input points x_{j}\in X , is a multivariate Gaussian distribution [ 9 ] . It is standard to characterise a Gaussian process by its mean and covariance functions, i.e. f\sim\mathrm{GP}(\mu,k) where \mathbb{E}[f(x)]=\mu(x) and \mathrm{Cov}(f(x),f(x^{\prime}))=k(x,x^{\prime}) for any x,x^{\prime}\in X . The Gaussianity requirement of a GP thus demands the covariance function (also called a kernel ) k to be a positive semi-definite function [ 7 ] . In this paper, we assume the GP is spatio-temporal , so the domain of the process is the Cartesian product X=S\times T of spatial S and temporal T components. We use \boldsymbol{x}=(s,t)\in X to denote a point with location s and time t . Given data \mathcal{D}=\{(\boldsymbol{x}_{i},y_{i})\}_{i=1}^{N}=\{\underaccent{\bar}{\boldsymbol{x}},\underaccent{\bar}{y}\} as noisy observations of the truth, i.e. y_{i}=f(\boldsymbol{x}_{i})+\varepsilon_{i} , \varepsilon_{i}\sim N(0,\sigma_{\text{obs}}^{2}) , the posterior distribution of f at any vector of M test points \underaccent{\bar}{\boldsymbol{x}}^{*}\in X^{M} is given by \begin{split}f(\underaccent{\bar}{\boldsymbol{x}}^{*})|\mathcal{D}&\sim N(\mu_{\underaccent{\bar}{y}^{*}|\mathcal{D}},\Sigma_{\underaccent{\bar}{y}^{*}|\mathcal{D}}),\\ \mu_{\underaccent{\bar}{y}^{*}|\mathcal{D}}&=\mu(\underaccent{\bar}{\boldsymbol{x}}^{*})+K_{*}^{T}(K+\sigma_{\text{obs}}I_{N})^{-1}(\underaccent{\bar}{y}-\mu(\underaccent{\bar}{y}))\\ \Sigma_{\underaccent{\bar}{y}^{*}|\mathcal{D}}&=K_{**}-K_{*}^{T}(K+\sigma_{\text{obs}}I_{N})^{-1}K_{*}\end{split} (1) with N\times N identity matrix I_{N} and Gram matrices K=(k(\boldsymbol{x}_{i},\boldsymbol{x}_{j}))_{ij}\in\mathbb{R}^{N\times N} , K_{*}=(k(\boldsymbol{x}_{i},\boldsymbol{x}^{*}_{j}))_{ij}\in\mathbb{R}^{N\times M} , K_{**}=(k(\boldsymbol{x}^{*}_{i},\boldsymbol{x}^{*}_{j}))_{ij}\in\mathbb{R}^{M\times M} . Furthermore, if one wishes to draw a sample from this posterior \underaccent{\bar}{y}^{*}\sim f(\underaccent{\bar}{\boldsymbol{x}}^{*})|\mathcal{D} , it can be achieved by scaling and shifting an M -dimensional standard Gaussian vector, i.e. \underaccent{\bar}{y}^{*}=\mu_{\underaccent{\bar}{y}^{*}|\mathcal{D}}+\sqrt{\Sigma_{\underaccent{\bar}{y}^{*}|\mathcal{D}}}\xi,\qquad\xi\sim N(0,I_{M}) (2) where \sqrt{\Sigma_{\underaccent{\bar}{y}^{*}|\mathcal{D}}} is the matrix square root. Notice that the Gram matrices used in ( 1 ) represent the joint dependency between observation points and prediction points. Rather than updating predictions recursively as new data arrive, the GP constructs a covariance matrix considering every training and prediction location at once, and predictions are obtained through a marginal conditioning operation. So, test points are not treated as future states evolving through time, but simply as additional coordinates in a large correlated random vector. This thus indicates the static nature of the standard GP formulation. Formulas such as ( 1 ) and ( 2 ) indicate the cubic complexity scaling of GP regressions and sampling. Following standard matrix computation complexity analysis [ 3 ] , for matrices A\in\mathbb{R}^{N\times N} and B\in\mathbb{R}^{N\times M} , computations such as matrix inversion A^{-1} is of complexity O(N^{3}) , matrix square root (or Cholesky decomposition) \sqrt{A} is of complexity O(N^{3}) , and matrix multiplication AB is of complexity O(N^{2}M) . Therefore, the computation of the posterior covariance matrix, following ( 1 ), is of complexity O(N^{3}+N^{2}M+NM^{2}) , while drawing J samples from the posterior following ( 2 ) costs an additional O(JM^{3}) after obtaining the posterior distribution’s mean vector and covariance matrix. In addition, in the spatio-temporal setting, we would often be interested in predicting on the same set of spatial locations over time, so the M test points consist of M_{s} spatial locations and M_{t} time points such that M=M_{s}M_{t} . Therefore, the full computational complexity to draw J samples from the posterior, including regression, is O(N^{3}+N^{2}M_{s}M_{t}+NM_{s}^{2}M_{t}^{2}+JM_{s}^{3}M_{t}^{3}) , which could be formidable when any of the observation number N , prediction spatial location M_{s} , and prediction time points M_{t} is large. A common remedy for the costly GP computations is to conduct approximate inferences by reducing the effective number of considered observation locations. Such approximate methods include inducing points [ 14 ] and Vecchia approximation [ 6 ] . In this paper, however, we focus on exact GP inference, although the methods we propose here are amenable to further approximation and computational speed-up, as we will explain further in Section 4 . 3 The Dynamic approach to Gaussian Process Inference An alternative exact GP inference method with improved scalability is the dynamic approach [ 5 ; 11 ] . For a wide class of Gaussian processes, the stochastic process can be formulated as the stationary solution to a stochastic (partial) differential equation (SPDE) with closed-form transition densities across time so as to admit sequential inference that scales linearly in time. The class of such processes includes spatio-temporal GPs with Matérn temporal