Adaptive Grid-Based Thompson Sampling for Efficient Trajectory Discovery
Arindam Fadikar, Abby Stevens, Mickaël Binois, Nicholson Collier, Jonathan Ozik · 2025
Read Adaptive Grid-Based Thompson Sampling for Efficient Trajectory Discovery by Arindam Fadikar, Abby Stevens, Mickaël Binois, Nicholson Collier, Jonathan Ozik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18099.txt.
Introduction
Arindam Fadikar, with five co-authors, upgrades Bayesian optimization so it learns from single simulation runs, not just their averages. Pricey simulators are often judged by summary numbers over many runs, which throws away how each run unfolds. Their method feeds both settings and random seeds into a shared surrogate model.
That trick allows direct inference at the level of one trajectory. A common-seed scheme defines a likelihood over individual paths. For scientists tuning costly models, the gain is sharper guidance from the same runs, especially when the path itself, not the average, is what the study cares about.
Catalog record
| CREATOR | Arindam Fadikar, Abby Stevens, Mickaël Binois, Nicholson Collier, Jonathan Ozik |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18099
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adaptive Grid-Based Thompson Sampling for Efficient Trajectory Discovery Arindam Fadikar, Abby Stevens, Mickaël Binois, Nicholson Collier, Jonathan Ozik Abstract Bayesian optimization (BO) is a powerful framework for estimating parameters of computationally expensive simulation models, particularly in settings where the likelihood is intractable and evaluations are costly. In stochastic models every simulation is run with a specific parameter set and an implicit or explicit random seed, where each parameter set and random seed combination generates an individual realization, or trajectory, sampled from an underlying random process. Existing BO approaches typically rely on summary statistics over the realizations, such as means, medians, or quantiles, potentially limiting their effectiveness when trajectory-level information is desired. We propose a trajectory-oriented Bayesian optimization method that incorporates a Gaussian process (GP) surrogate using both input parameters and random seeds as inputs, enabling direct inference at the trajectory level. Using a common random number (CRN) approach, we define a surrogate-based likelihood over trajectories and introduce an adaptive Thompson Sampling algorithm that refines a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings-based densification. This approach concentrates computation on statistically promising regions of the input space while balancing exploration and exploitation. We apply the method to stochastic epidemic models, a simple compartmental and a more computationally demanding agent-based model, demonstrating improved sampling efficiency and faster identification of data-consistent trajectories relative to parameter-only inference. 1 Introduction Epidemiological simulation models are increasingly recognized as indispensable tools for supporting public health decisions during crises (e.g., Ozik2021 ; eubank2004modelling ). Their use dates back to at least the early twentieth century, when researchers began modeling how infectious diseases—including influenza and, more recently, COVID-19—propagate through contact networks. The most common types of these models, such as compartmental models and agent-based models (ABMs), each have a set of parameters governing disease dynamics. Some models are deterministic, producing identical outputs for a given set of parameter inputs. Others are stochastic, generating outputs based on both model parameters and random seeds, producing specific stochastic realizations, or trajectories, of random processes within the models. Estimating model parameters typically involves fitting simulated trajectories to observed data ( chatzilena2019contemporary ; toni2009approximate ; pooley2015posterior ) . In this paper we investigate a novel Bayesian optimization (BO) approach that seeks to align simulation outputs with empirical data at the trajectory level. This contrasts with more common approaches that rely on summary statistics over stochastic realizations. Classical calibration approaches rely on maximum likelihood estimation or Bayesian inference, often facilitated by surrogate models such as Gaussian processes (GPs) to emulate the expensive simulator and reduce computational cost. BO builds on this idea by embedding the surrogate in a sequential decision framework to prioritize promising regions of the parameter space. Other methods such as Ensemble Kalman Filtering (EnKF) ( evensen2003ensemble ; iglesias2013ensemble ) , the Partially Observed Markov Process (POMP) framework ( king2016statistical ) , Approximate Bayesian Computation (ABC) ( beaumont2002approximate ) , and simulation-based inference (SBI) ( cranmerFrontierSimulationbasedInference2020 ; dyerBlackboxBayesianInference2024a ) are also used extensively in calibrating simulation models. However, calibrating stochastic simulation models remains a fundamentally ill-posed problem ( baker2022analyzing ) . Unlike deterministic models, where each input maps to a fixed output, stochastic simulators produce outputs that are random draws from an unknown distribution. The same input parameter \mathbf{x} can yield divergent outcomes across simulation replicates, making it ambiguous what it means for parameters to match observed data. Nonetheless, in common practice, calibration involves aligning expected values or other distributional summaries of stochastic simulation trajectories with data. For the purpose of illustration, let us consider the use of ABMs for modeling infectious diseases. In an ABM, each individual is represented by an autonomous agent with predefined attributes and behaviors ( eubank2004modelling ) . Agents interact over network structures that capture heterogeneous mixing patterns, and a single ABM run produces one possible mixing and transmission trajectory ( ferguson2020report ; chang2021modelling ) . Stochasticity arises from probabilistic transmission events but also from structural variations such as dynamic contact patterns or initial conditions. This results in different outcomes even under identical model parameters. When stochasticity reflects intrinsic structural variation, knowing only the parameter values that match observations ignores specific mixing patterns underlying transmission, which may be essential for planning targeted interventions. From a statistical perspective, such aggregation over randomness induces a loss of identifiability. Hence, we argue that a principled solution should investigate the benefits of estimating both the parameters \mathbf{x} and specific stochastic realizations, denoted by random seeds r , which jointly produce trajectories consistent with observed data. Recovering this tuple (\mathbf{x},r) then enables trajectory-level inference. Individual trajectories also enable the use of trajectory continuations when modeling epidemics over time with updating surveillance data, such as with sequential data assimilation approaches ( spannaus2022inferring ; fadikar2024towards ) . Trajectory-oriented optimization (TOO) is a surrogate-based framework for calibrating stochastic simulations by jointly estimating the tuple (\mathbf{x},r) with the use of a common random number GP (CRNGP) ( fadikar2023TOO ) . The CRNGP enables the treatment of the simulation as a deterministic function over the augmented input space (\mathbf{x},r) , enabling inference at the trajectory level rather than through distributional summaries. Building on the TOO framework, this paper focuses on improving its computational efficiency to enable faster identification of simulated trajectories that are consistent with the observed data. We extend the Thompson Sampling (TS) based BO algorithm by adaptively generating the parameter-seed space for faster trajectory exploration. We illustrate this approach using a Susceptible-Exposed-Infectious-Recovered (SEIR) model of COVID-19 transmission in Chicago, highlighting the efficiency gains over existing trajectory-finding methods. We also discuss ways in which high-performance computing (HPC) workflows can extend the applicability of such complex algorithms to more computationally demanding simulations. The remainder of the paper is organized as follows. Section 2 introduces GPs as surrogate models for computationally expensive simulations and describes BO using TS as a sequential design strategy. This section also outlines the TOO framework. Section 3 details the proposed adaptive-grid TS methodology, which integrates the CRNGP surrogate with an iterative refinement strategy to efficiently identify parameter–trajectory pairs consistent with observed data. In Section 4 , we present a controlled simulation study comparing our trajectory-oriented methods with conventional parameter-focused approaches, highlighting the advantages of explicitly modeling stochastic realizations via CRNGP. Then, in Section 5 , we apply the proposed methodology to a large-scale agent-based model of COVID-19 transmission, and demonstrate its practical utility in realistic scenarios for supporting public health stakeholders. We conclude in Section 6 with potential extensions of this work. 2 Background This section outlines the key components required for trajectory-level inference in stochastic simulation models. We begin by introducing GP surrogates, followed by strategies for incorporating input-dependent variability into the surrogate framework. We then describe a BO approach based on TS, which facilitates efficient exploration of the augmented (\mathbf{x},r) input space. 2.1 Gaussian Process Surrogates GPs ( Rasmussen2005 ; gramacy2020surrogates ) are nonparametric regression models used to infer unknown functions from observed data. GPs have been used extensively as surrogates ( Sacks1989a ; higdon2004combining ) for expensive deterministic computer simulations, for tasks such as sensitivity analysis ( saltelli2008global ) , optimization ( pearce2022bayesian ) , calibration ( Kennedy2001 ) , and uncertainty quantification ( binois2015quantifying ) . Formally, a GP defines a distribution over functions f:\mathbb{R}^{d}\to\mathbb{R} , such that for any