Scaled Block Vecchia Approximation for High-Dimensional Gaussian Process Emulation on GPUs
Qilong Pan 1,5 , Sameh Abdulah 1,5 , Mustafa Abduljabbar 1,5 , Hatem Ltaief 1,5 , Andreas Herten 2,6 , Mathis Bode 2,7 , Matthew Pratola 3,8 , Arindam Fadikar 4,9 , Marc G. Genton 1,5 , David E. Keyes 1,5 , Ying Sun 1,5 · 2025
Read Scaled Block Vecchia Approximation for High-Dimensional Gaussian Process Emulation on GPUs by Qilong Pan 1,5 , Sameh Abdulah 1,5 , Mustafa Abduljabbar 1,5 , Hatem Ltaief 1,5 , Andreas Herten 2,6 , Mathis Bode 2,7 , Matthew Pratola 3,8 , Arindam Fadikar 4,9 , Marc G. Genton 1,5 , David E. Keyes 1,5 , Ying Sun 1,5 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12004.txt.
Introduction
Qilong Pan and co-authors speed up a workhorse statistics method in a 2025 paper. Gaussian processes can stand in for costly computer simulations, letting scientists test ideas quickly. But the method bogs down on large datasets, since its core math grows fast with data size.
Their algorithm joins two known shortcuts: one that rescales inputs by direction, and one that works in blocks of near neighbors. It runs across many graphics processors at once, the first shared-memory build of its kind. Fast emulation supports tasks like risk analysis and design tuning on the huge datasets modern simulations produce.
Catalog record
| CREATOR | Qilong Pan 1,5 , Sameh Abdulah 1,5 , Mustafa Abduljabbar 1,5 , Hatem Ltaief 1,5 , Andreas Herten 2,6 , Mathis Bode 2,7 , Matthew Pratola 3,8 , Arindam Fadikar 4,9 , Marc G. Genton 1,5 , David E. Keyes 1,5 , Ying Sun 1,5 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-12004
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scaled Block Vecchia Approximation for High-Dimensional Gaussian Process Emulation on GPUs Qilong Pan 1,5 , Sameh Abdulah 1,5 , Mustafa Abduljabbar 1,5 , Hatem Ltaief 1,5 , Andreas Herten 2,6 , Mathis Bode 2,7 , Matthew Pratola 3,8 , Arindam Fadikar 4,9 , Marc G. Genton 1,5 , David E. Keyes 1,5 , Ying Sun 1,5 Abstract . Emulating computationally intensive scientific simulations is crucial for enabling uncertainty quantification, optimization, and informed decision-making at scale. Gaussian Processes (GPs) offer a flexible and data-efficient foundation for statistical emulation, but their poor scalability limits applicability to large datasets. We introduce the Scaled Block Vecchia (SBV) algorithm for distributed GPU-based systems. SBV integrates the Scaled Vecchia approach for anisotropic input scaling with the Block Vecchia (BV) method to reduce computational and memory complexity while leveraging GPU acceleration techniques for efficient linear algebra operations. To the best of our knowledge, this is the first distributed implementation of any Vecchia-based GP variant. Our implementation employs MPI for inter-node parallelism and the MAGMA library for GPU-accelerated batched matrix computations. We demonstrate the scalability and efficiency of the proposed algorithm through experiments on synthetic and real-world workloads, including a 50M point simulation from a respiratory disease model. SBV achieves near-linear scalability on up to 512 A100 and GH200 GPUs, handles 2.56B points, and reduces energy use relative to exact GP solvers, establishing SBV as a scalable and energy-efficient framework for emulating large-scale scientific models on GPU-based distributed systems. DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 1. Introduction Many modern scientific and engineering advances rely on the use of computer models that use mathematical equations to represent physical systems and are implemented as high-fidelity simulations. These models allow researchers to explore phenomena that are impractical to observe experimentally, such as climate dynamics ( Eyring et al., 2016 ; Kochkov et al., 2024 ) , materials ( Børvik et al., 2001 ; Walters et al., 2018 ) , and cosmological structure formation ( Springel, 2005 ; Spurio Mancini et al., 2022 ; Lawrence et al., 2017 ) . Computer models can generate predictions that inform policy decisions, drive technological innovation, and facilitate fundamental discoveries by solving systems of differential equations or leveraging particle-based methods ( Børvik et al., 2001 ; Eyring et al., 2016 ; Springel, 2005 ) . However, when exploring models with many input parameters, learning the relationship between inputs and outputs may require thousands of evaluations ( Fiore et al., 2018 ) . Such high computational costs often constrain their usage, motivating the adoption of statistical emulators to approximate the simulator’s behavior at a fraction of the cost ( Santner et al., 2003 ; Abdulah et al., 2024 ) . A common approach to building computer model emulators is to fit a Gaussian Process (GP) to data generated by running the simulator at a predefined set of input values (design points), thereby learning a mapping from the model input space to the model output space ( Andrianakis and Challenor, 2012 ) . GP-based emulators enhance interpretability and facilitate efficient analysis, particularly when simulating scenarios across a broad parameter space. For example, variable importance ( Saltelli et al., 2010 ) and sequential optimization ( Winkel et al., 2021 ) may only be feasible using a cheap-to-evaluate GP emulator of the underlying expensive-to-evaluate simulator. While methods such as RBF-based models ( Acosta, 1995 ; Arora et al., 2023 ) can offer flexible function approximation for such endeavors, they are fundamentally deterministic heuristics that lack marginal likelihood formulations, statistically grounded parameter estimation, and coherent uncertainty quantification. As a result, they are not well aligned with the probabilistic and inference-driven objectives of this work. In contrast, GP modeling provides a fully probabilistic framework with principled uncertainty quantification and likelihood-based inference, which are essential when studying scalable approaches to inference and made possible by covariance approximations such as Vecchia-based methods ( Vecchia, 1988 ; Katzfuss and Guinness, 2021 ) . However, GPs face significant scalability challenges due to their \mathcal{O}(n^{3}) computational cost and \mathcal{O}(n^{2}) memory requirements, where n is the number of data points. These costs arise from operations on a dense n\times n covariance matrix that captures the correlations between data inputs ( Abdulah et al., 2018a ) . For instance, modeling 200 K locations demands about 300 GB of memory and an estimated 2.6 PFLOP of computation, making standard GPs infeasible for large-scale applications without approximation methods. Thus, numerous studies have addressed the computational and memory challenges of large-scale GPs. Some efforts have pursued exact GP computation on High-Performance Computing (HPC) systems ( Abdulah et al., 2018a ; Abdulah et al., 2019 ; Abdulah et al., 2021 ) . For instance, the largest reported problem size was processed on ORNL’s Frontier system, handling 27.24 M data points in mixed-precision across approximately 9 K nodes ( \sim 36K AMD GPUs) of the system ( Abdulah et al., 2024 ) . However, most of the literature has focused on approximation to alleviate the computational burden of GP modeling, including low-rank ( Furrer et al., 2006 ; Bevilacqua et al., 2016 ; Shi et al., 2025 ) , tile low-rank ( Abdulah et al., 2018b ; Salvaña et al., 2022 ; Mondal et al., 2023 ) , covariance tapering ( Furrer et al., 2006 ) , and Vecchia approximations ( Katzfuss and Guinness, 2021 ; Pan et al., 2024 ) . Among these approximation methods, Classic Vecchia (CV) approximation demonstrates superior advantages. In ( Katzfuss et al., 2022 ) , Katzfuss et al. introduced the Vecchia approximation for GP emulation in the context of computer experiments and proposed the Scaled Vecchia (SV) method to enhance approximation accuracy in high-dimensional settings. However, their study was limited to small-scale problems and relied entirely on CPU-based computations. In contrast, efforts have been made to accelerate the classic Vecchia algorithm on GPUs, as in ( Pan et al., 2024 ) , which subsequently introduced the Block Vecchia (BV) approach ( Pan et al., 2025 ) to further enhance computational speed while preserving approximation accuracy. Nevertheless, the use of CV approximation for high-dimensional GP emulation on GPUs remains unexplored. This gap presents an opportunity to develop a scalable GPU-based method that can efficiently address the computational and memory challenges posed by high-dimensional inputs. In this work, we propose a novel algorithm, the Scaled Block Vecchia (SBV) approximation, for distributed GPU-based systems. SBV integrates the Block Vecchia (BV) algorithm from ( Pan et al., 2025 ) with the Scaled Vecchia approach introduced in ( Katzfuss et al., 2022 ) , while leveraging GPU acceleration techniques from ( Pan et al., 2024 ) . To the best of our knowledge, this work represents the first distributed implementation of any variant of the Vecchia-based GP algorithm. Leveraging the Message Passing Interface (MPI), we parallelize computations across multiple GPU-enabled nodes, achieving scalable performance well beyond that demonstrated in prior studies. To validate the accuracy and robustness of our algorithm, we perform GP simulations across a range of parameter settings and benchmark its performance on the satellite drag dataset, a standard high-dimensional testbed for evaluating GP-based models ( Sun et al., 2019 ) . As an