Vecchia approximated Bayesian heteroskedastic Gaussian processes
Parul V. Patil, Robert B. Gramacy, Cayelan C. Carey, R. Quinn Thomas · 2025
Read Vecchia approximated Bayesian heteroskedastic Gaussian processes by Parul V. Patil, Robert B. Gramacy, Cayelan C. Carey, R. Quinn Thomas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07815.txt.
Introduction
Parul Patil and co-authors (2025) upgrade the workhorse model for computer simulations whose noise changes with the input. Gaussian processes that allow uneven noise, heteroskedastic ones, fit such simulators well, because they estimate the noise level itself as it varies. Two limits held them back: they choked on large simulation campaigns, and they plugged in point estimates for unknown variances, which understates the final uncertainty. The authors make the model fully Bayesian and scalable at once.
An elliptical slice sampler integrates over the unknown variances instead of fixing them. A Vecchia approximation, which conditions each point on a few neighbors, breaks the computing bottleneck. The upgraded model beats alternatives on a benchmark and on a real corpus of more than nine million lake temperature simulations, a nice scale for a statistics paper. Their implementation ships as an open R package, bhetGP, on CRAN.
Catalog record
| CREATOR | Parul V. Patil, Robert B. Gramacy, Cayelan C. Carey, R. Quinn Thomas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-07815
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Vecchia approximated Bayesian heteroskedastic Gaussian processes Parul V. Patil, Robert B. Gramacy, Cayelan C. Carey, R. Quinn Thomas Abstract Many computer simulations are stochastic and exhibit input dependent noise. In such situations, heteroskedastic Gaussian processes ( hetGP s) make ideal surrogates as they estimate a latent, non-constant variance. However, existing hetGP implementations are unable to deal with large simulation campaigns and use point-estimates for all unknown quantities, including latent variances. This limits applicability to small experiments and undercuts uncertainty. We propose a Bayesian hetGP using elliptical slice sampling (ESS) for posterior variance integration, and the Vecchia approximation to circumvent computational bottlenecks. We show good performance for our upgraded hetGP capability, compared to alternatives, on a benchmark example and a motivating corpus of more than 9-million lake temperature simulations. An open source implementation is provided as bhetGP on CRAN. 1 Introduction Computer simulation experiments are common in scientific fields where physical/field experimentation may be challenging ( Booker, 1998 ; Santner et al., 2018 , e.g.,) . Examples include ecology ( Johnson, 2008 ) , epidemiology ( Hu and Ludkovski, 2017 ; Fadikar et al., 2018 ) , aeronautics ( Mehta et al., 2014 ) and engineering ( Zhang et al., 2015 ) . It used to be that computer codes simulating complex phenomena were deterministic, meaning identical inputs always yield same output. Often they were solvers of systems of differential equations. Modern simulations are increasingly stochastic, utilizing pseudo-random number generators in some aspect, either via Monte Carlo (MC) for numerical quadrature ( Mehta et al., 2014 ; Herbei and Berliner, 2014 , e.g.,) , or to virtualize a system which is inherently stochastic ( Xie et al., 2012 ; Fadikar et al., 2018 , e.g.,) . Baker et al., 2022 provide a recent review of this setting, highlighting opportunities and challenges. Our interest in stochastic simulation lies in an application of the General Lake Model ( Hipsey et al., 2012 , GLM;) 1 1 1 We apologize for the acronym; there is no connection to generalized linear modeling. to reservoir temperature forecasting. The open-source GLM model has been applied to thousands of lakes around the world to simulate water quality ( Read et al., 2014 ; Bruce et al., 2018 ) . Water temperature impacts the formation of phytoplankton blooms which can adversely impact drinking water quality ( Carey et al., 2012 ) . GLM simulations are technically deterministic, but when driven by data products that summarize uncertainties in weather/climate variables, they propagate those stochastic effects. Here, we follow Holthuijzen et al., 2024 and study GLM as driven by an ensemble of forecasts generated by the National Oceanic and Atmospheric Administration (NOAA) Global Ensemble Forecast System ( Hamill et al., 2022 , NOAAGEFS;) 2 2 2 See https://www.ncei.noaa.gov/products/weather-climate-models/global-ensemble-forecast . . Going forward, we refer to NOAA-GLM. Figure 1 : NOAA-GLM simulations ahead to a horizon of 30 days, varying day of year (DOY) in 2022 and depth from the surface of the lake in meters. Example NOAA-GLM simulated temperatures ( y -axes) are provided in Figure 1 ; more detail is provided in Section 5.3 . For now, simply notice the non-linear relationship, varying noise level over inputs, and large number of outputs corresponding to temperature forecasts at just three days (there are 2730 gray circles). Holthuijzen et al., 2024 provide us with three years worth simulations for ten depths. It can be advantageous to abstract a corpus of simulations, like those in Figure 1 , with a statistical model relating inputs to outputs. Such abstractions are known as surrogates or emulators in the computer modeling literature ( Santner et al., 2018 ; Gramacy, 2020 ) . One reason to fit a surrogate is to economize on expensive simulations, keeping experiment sizes down, or to cope with limited access to simulation resources. A good surrogate can furnish predictions with appropriate uncertainty quantification (UQ) in lieu of additional simulations, tell you where to put new runs in order to accomplish some design goal like identifying safe/efficient operating conditions ( Booth et al., 2025 , e.g.,) , or help calibrate simulations to field data ( Kennedy and O’Hagan, 2001 , e.g.,) . Although many statistical models make good surrogates – any regression model would be appropriate – Gaussian processes ( Williams and Rasmussen, 2006 , GPs; e.g.,) are canonical because they are non-parametric, nonlinear and provide accurate and smooth predictions along with well-calibrated UQ. Yet for some situations GPs lack the flexibility required to capture the salient dynamics being simulated. Input-dependent noise, like in Figure 1 , is one such setting. As Baker et al., 2022 explains, stochastic processes are often heteroskedastic (exhibit input-dependent noise) not homoskedastic (constant noise for all inputs). Another downside to GPs involves computation. With N training-data input-output pairs, GP modeling involves decomposing a dense N\times N covariance matrix, incurring costs in \mathcal{O}(N^{3}) flops. This limits N to the small thousands. Noisy simulations exacerbate this bottleneck because bigger N is needed to identify the signal. Goldberg et al., 1998 were the first to propose a heteroskedastic GP, coupling two GPs: one for means and another for variances. But their Bayesian MCMC-based inferential toolkit, involving independent Metropolis proposals for N latent variance variables, one for each training data observation, was cumbersome, limiting N to the small hundreds. About a decade later, several thriftier variants arrived. Binois et al., 2018 provide a review toward appropriating two key ingredients from that line of research for their own method: 1) reduced sufficient statistics from training designs that deploy replication ( N simulations at n\ll N unique inputs), first proposed as stochastic kriging ( Ankenman et al., 2010 , SK;) ; 2) point-estimation of latent variances rather than full posterior sampling, e.g., via expectation maximization ( Kersting et al., 2007 , EM;) . Binois et al., 2018 glued the two ideas together with a well-known linear algebra trick known as the Woodbury identity, requiring only \mathcal{O}(n^{3}) flops. Their hetGP is the only method with an implementation in public software for R ( Binois and Gramacy, 2021 ) and Python ( O’Gara et al., 2025 ) . However, there are two downsides to hetGP , both broadly and for Holthuijzen et al., 2024 ’s NOAA-GLM simulations. One is that point-inference of unknown quantities, particularly variances, undercuts UQ. Another is the sheer size of modern stochastic simulation campaigns. NOAA’s thirty-one member ensemble means a 31-fold degree of replication ( n=N/31 ). If Figure 1 comprised the entire campaign, showing n=90 unique DOY and depth combinations, then hetGP would work fine. But three years of simulations, at ten depths, and a 30-day horizon means n>300{,}000 . You can’t build an n\times n matrix of that size on workstation, let alone decompose one. Holthuijzen et al., 2024 were able to circumvent the second (computational) downside but at the expense of the first (modeling and inferential fidelity). They used the scaled Vecchia approximation ( Katzfuss et al., 2022 ) to induce sparsity in the (inverse) covariance matrices involved in the two (mean and variance) GPs. The downgrade in fidelity comes from a decoupling of the two GPs in order to offload computations to a software library. They had to lean heavily independent, moment-based inference as apposed to the joint,