Advancing calibration for stochastic agent-based models in epidemiology with Stein variational inference and Gaussian process surrogates

Connor Robertson, Cosmin Safta, Jaideep Ray, Nicholson Collier, Jonathan Ozik · 2025

Read Advancing calibration for stochastic agent-based models in epidemiology with Stein variational inference and Gaussian process surrogates by Connor Robertson, Cosmin Safta, Jaideep Ray, Nicholson Collier, Jonathan Ozik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19550.txt.

Introduction

Connor Robertson and co-authors test in 2025 a faster way to calibrate computer models of epidemics. Agent-based models simulate each person in a city, which makes them rich but hard to tune. Classic tuning methods that map out likely parameter values cost too much computing for such heavy models. The team tried an inference method that moves a swarm of candidate parameter sets uphill using gradient clues.

To make the swarm method practical, they first replaced the costly model with a fast stand-in that mimics it. They then calibrated a real city-scale COVID model both ways. The swarm method matched the classic method on prediction accuracy and on how well the tuning covered the truth. The authors also report the practical catches: settings need careful tuning, and the swarm's motion needs watching as it settles.

Catalog record

CREATORConnor Robertson, Cosmin Safta, Jaideep Ray, Nicholson Collier, Jonathan Ozik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19550

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advancing calibration for stochastic agent-based models in epidemiology with Stein variational inference and Gaussian process surrogates Connor Robertson, Cosmin Safta, Jaideep Ray, Nicholson Collier, Jonathan Ozik Abstract Accurate calibration of stochastic agent-based models (ABMs) in epidemiology is crucial to make them useful in public health policy decisions and interventions. Traditional calibration methods, e.g., Markov Chain Monte Carlo (MCMC), that yield a probability density function for the parameters being calibrated, are often computationally expensive. When applied to ABMs which are highly parametrized, the calibration process becomes computationally infeasible. This paper investigates the utility of Stein Variational Inference (SVI) as an alternative calibration technique for stochastic epidemiological ABMs approximated by Gaussian process (GP) surrogates. SVI leverages gradient information to iteratively update a set of particles in the space of parameters being calibrated, offering potential advantages in scalability and efficiency for high-dimensional ABMs. The ensemble of particles yields a joint probability density function for the parameters and serves as the calibration. We compare the performance of SVI and MCMC in calibrating CityCOVID, a stochastic epidemiological ABM, focusing on predictive accuracy and calibration effectiveness. Our results demonstrate that SVI maintains predictive accuracy and calibration effectiveness comparable to MCMC, making it a viable alternative for complex epidemiological models. We also present the practical challenges of using a gradient-based calibration such as SVI which include careful tuning of hyperparameters and monitoring of the particle dynamics. 1 Introduction The calibration of agent-based models has presented a formidable challenge in enabling their widespread adoption and utility. This is particularly pressing in epidemiology, where agent-based models provide the potential for rapid simulation of interventions to manage pandemic-level diseases [ 27 , 24 , 34 , 1 , 47 ] . As the complexity of these models increases, the need for accurate calibration becomes even more pronounced. Traditional calibration techniques, such as Markov Chain Monte Carlo (MCMC), have been previously employed for uncertainty quantification tasks due to their convergence guarantees. However, these methods often carry significant computational cost, which can hinder their applicability for more complex models and for real-time decision-making scenarios. When the ABM to be calibrated has a large number of parameters, the use of MCMC becomes computationally infeasible. As a result, ABM simulations are often accelerated using surrogate or emulator model approximations and calibrations are made more affordable using techniques such as approximate Bayesian computation (ABC) [ 7 ] . In recent years, alternative calibration approaches have emerged from the field of variational inference that promise to improve the efficiency and scalability of model calibration while maintaining some convergence properties [ 48 ] . One such method is Stein Variational Inference (SVI) [ 33 ] , a technique that uses gradient information to iteratively refine a set of parameter samples or “particles.” This approach offers a compelling advantage in terms of scalability and computational efficiency, particularly for high-dimensional models, where traditional sampling methods struggle [ 57 ] . By pairing this estimation technique with surrogate modeling of ABMs with Gaussian processes (GPs), SVI presents a fast and novel pathway for calibration. This paper aims to explore the utility of SVI as an alternative calibration technique for stochastic epidemiological ABMs, specifically focusing on its application to CityCOVID. CityCOVID is a sophisticated ABM designed to simulate the spread of COVID-19 in Chicago [ 43 , 25 ] and was used to support decision making by public health officials. Specifically, we perform a comparative analysis of SVI and MCMC, evaluating their performance in terms of calibration effectiveness and predictive accuracy. Our findings reveal that SVI not only matches the predictive capabilities of MCMC but also offers a more efficient calibration process, thereby positioning it as a viable and scalable alternative to calibration for ABMs. Despite its advantages, the implementation of SVI is not without challenges. The calibration process requires careful tuning of the hyperparameters and diligent monitoring to ensure optimal performance. In this paper, we also address these practical considerations, providing insight into the nuances of employing gradient-based calibration methods in the context of agent-based modeling. The remainder of this paper is structured as follows. This introduction concludes with a literature review that outlines the use of stochastic ABMs in epidemiology, existing calibration techniques for stochastic ABMs generally, and a detailed overview of previous applications of SVI. Section 2 then outlines the calibration problem, describing the CityCOVID model, the construction of its GP surrogate, and the implementation details for both MCMC and SVI. In Section 3 , we describe the setup and results of calibration for both methods. Section 4 discusses the results of our experiment, showcasing the predictive accuracy and calibration effectiveness of SVI compared to MCMC. Finally, Section 5 concludes with a summary of the experiment results and provides directions for future research. 1.1 Literature Review ABMs have found a variety of applications across biology [ 2 , 61 , 56 ] , ecology [ 39 , 61 ] , economics [ 12 , 6 , 22 ] , and epidemiology [ 26 , 37 ] and have proven capable in providing predictions of and insight into complex dynamics [ 60 ] . These models simulate the interactions of individual agents, allowing for the exploration of heterogeneous behaviors and local interactions that can lead to emergent phenomena. Several key studies have demonstrated the utility of these models in epidemiological contexts, including the spread of infectious diseases such as influenza, COVID-19, and others. For instance, Ref. [ 32 ] developed and calibrated an ABM to simulate the transmission dynamics of influenza, highlighting the ability of ABMs to capture the impact of vaccination strategies once tuned. Similarly, Ref. [ 54 ] utilized an ABM to investigate the spread of COVID-19, providing insights into the effectiveness of social distancing measures and demonstrating practical use of ABMs for epidemiological insights. Effective calibration techniques for these ABMs are essential to ensure that model output aligns with observed data. Traditional methods, such as grid search and random sampling, have been employed [ 50 ] ; however, these approaches often struggle with high-dimensional parameter spaces. More sophisticated techniques, including Bayesian inference and MCMC sampling, have been used because of their theoretical guarantees and uncertainty quantification [ 29 , 35 , 21 ] but are limited by the usually large computational expense of repeated ABM evaluations. This is of particular importance for high-dimensional models [ 57 ] . Often, this has prompted the use of more heuristic methods such as ABC [ 62 , 38 , 43 ] , genetic algorithms [ 16 , 10 ] , or particle swarm optimization [ 49 , 32 ] . Another approach to overcoming the computational challenge of ABM calibration has been addressed via the use of surrogate or emulator models, which approximate the outputs of an ABM using information from previous runs. Foremost among these approaches are the use of GPs [ 29 , 18 , 8 ] , support vector machines [ 3 , 46 ] , bagged and/or boosted machine learning approaches [ 42 , 62 , 15 , 28 , 45 , 31 , 52 , 53 ] , and neural networks [ 13 , 11 , 4 , 28 ] . Each of these provide a flexible framework for approximating complex functions, facilitating faster

Read the complete text