B3O: Scalable Boltzmann Batch Bayesian Optimization

Maximilian Bloor, Liyuan Xu, Hrvoje Stojic, Victor Picheny · 2026

Read B3O: Scalable Boltzmann Batch Bayesian Optimization by Maximilian Bloor, Liyuan Xu, Hrvoje Stojic, Victor Picheny (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30228.txt.

Introduction

Maximilian Bloor, Liyuan Xu, Hrvoje Stojic, and Victor Picheny make batch Bayesian optimization scale to massive parallel simulation. Bayesian optimization chooses which expensive experiments to run next. Modern workflows can evaluate huge batches of candidates at once. Existing batch methods either cost too much or lose batch variety. Their method, B3O, reframes batch building as pure sampling.

B3O draws candidate queries directly from a Boltzmann distribution. That distribution is defined by the acquisition function itself. Sampling sidesteps the joint search bottlenecks of earlier large-batch methods. The authors prove the extra regret from sampling is negligible. Empirically, B3O beats existing batch methods on synthetic benchmarks. It also adapts robustly across complex applied tasks. Those include multi-objective electrode design for batteries. Mixed-variable race car configuration is another successful test case.

Catalog record

CREATORMaximilian Bloor, Liyuan Xu, Hrvoje Stojic, Victor Picheny
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30228

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

B3O: Scalable Boltzmann Batch Bayesian Optimization Maximilian Bloor, Liyuan Xu, Hrvoje Stojic, Victor Picheny Abstract Modern engineering workflows increasingly rely on massive parallel simulation, driving the need for scalable, large-batch Bayesian Optimization (BO). Existing batch BO methods, however, incur large computational cost or rely on approximations that erode batch diversity. We propose B3O (Boltzmann Batch Bayesian Optimization), a framework that reframes batch generation as a pure sampling problem: drawing samples directly from the Boltzmann distribution defined by the acquisition function avoids the bottlenecks of existing large-batch methods. Theoretically, we prove that queries sampled from this distribution incur only negligible additional regret. Empirically, B3O outperforms existing batch BO methods on standard synthetic benchmarks and adapts robustly across complex applied tasks, including multi-objective electrode design and mixed-variable race car configuration. 1 Introduction From simulation-driven engineering design to high-throughput molecular screening, expensive black-box objectives are increasingly evaluated in parallel, making batch Bayesian Optimization (BO), where B candidates are queried concurrently per iteration, the regime of practical interest. Fully exploiting this parallelism while preserving sample efficiency is the central challenge as B scales, and existing strategies struggle in complementary ways. Conventional multi-point acquisitions such as q -Expected Improvement [ 14 ] target the small-batch regime: they evaluate high-dimensional expectations over the joint posterior of the B candidates, with each Monte Carlo evaluation requiring an \mathcal{O}(B^{3}) Cholesky factorization and joint optimization over the B\cdot d -dimensional product space, which becomes intractable as B grows. Sequential greedy approximations such as q -UCB [ 40 ] or local penalization [ 15 ] sidestep this joint optimization at \mathcal{O}(B) inner-optimization cost, but rely on heuristics that struggle to capture joint uncertainties and produce clustered, suboptimal batches. Scalable Thompson Sampling (TS) [ 36 , 38 , 39 , 26 ] sidesteps the joint posterior by optimizing one posterior trajectory per batch element. Practical implementations rely on spectral approximations such as Random Fourier Features [ 18 ] that suffer from variance starvation [ 27 ] , collapsing the optimized trajectories onto a narrow set of modes, and the decomposition is only compatible with stationary kernels, limiting its applicability. Diversity correctives based on Determinantal Point Processes [ 21 , 20 , 28 ] reintroduce a cost cubic in B and cannot recover modes never visited. These limitations motivate a truly scalable batch BO algorithm, robust across surrogate and acquisition choices, that preserves multimodal exploration of the acquisition surface. Boltzmann exploration offers a principled alternative: queries are drawn from a density built on a utility function, with a temperature parameter trading off greediness and exploration. The mechanism has strong theoretical backing in discrete multi-armed bandits [ 4 ] , but extending it to continuous BO requires sampling from intractable, multimodal acquisition densities. 11 , 12 explored acquisition-density sampling for BO using Metropolis–Hastings and gradient-based MCMC in distributed and small-batch settings. Stein Boltzmann Sampling [ 31 ] applies Stein Variational Gradient Descent to a Boltzmann density defined over the objective itself, but requires objective gradients and is therefore inapplicable to BO’s derivative-free black-box setting. We introduce B3O, a scalable batch BO framework that recasts batch generation as sampling. Drawing B i.i.d. samples from a Boltzmann density on the marginal acquisition makes B3O scale linearly in B while preserving multimodal exploration of the acquisition landscape, sidestepping the scalability–diversity trade-off of joint-posterior and trajectory-based approaches (Table 1 ). It is also flexible: continuous, constrained multi-objective, and mixed continuous–discrete search spaces are all handled by the same algorithm, swapping the sampler or the acquisition as needed. Our contributions: 1. Method: We frame batch generation as sampling from a Boltzmann density built solely from marginal predictive distributions, achieving linear batch-size scaling. We show that both a fixed inverse temperature and a time-varying schedule yield reliable optimization. 2. Theory: We prove a finite-time cumulative regret bound for sampling from the Boltzmann distribution defined by an UCB acquisition that recovers standard GP-UCB rates up to a negligible additive term. 3. Empirical generality: We demonstrate the wide applicability of B3O, which is sampler-, surrogate-, and acquisition-agnostic, through empirical experiments across continuous synthetic benchmarks, a constrained multi-objective electrode design problem (by changing the acquisition to expected hypervolume improvement), and a mixed continuous–discrete configuration problem (by changing the sampler to a mixed-space variant of Metropolis–Hastings). Table 1: Qualitative comparison of B3O with standard Batch BO baselines. † Multi-point batch acquisitions such as q -EI provide diversity implicitly via the joint posterior, but at a B\cdot d -dimensional inner optimization cost. Method Black-box (no gradients) Large Batches Diversity Control Various BO Settings (e.g. multi-objective) Model Agnostic Multi-Point Batch Alg. (e.g. q -EI) [ 14 ] ✓ × (✓) † ✓ ✓ TS [ 38 ] ✓ ✓ × ✓ × SBS [ 31 ] × ✓ ✓ × × B3O (Proposed) ✓ ✓ ✓ ✓ ✓ Figure 1: Comparison of TS and B3O for 1D batch generation, sharing the same underlying GP posterior. Top row (TS): (left) the GP posterior; (right) query points obtained by optimizing each sampled trajectory. Bottom row (B3O): (left) the marginal Upper Confidence Bound acquisition; (right) a diverse batch of query points drawn from the corresponding Boltzmann density. 2 Problem Formulation We consider sequential optimization of an expensive function f over a compact set \mathcal{X}\subset\mathbb{R}^{d} . At each timestep t=1,\ldots,T , the algorithm selects a batch \{\mathbf{x}_{t,b}\}_{b\in[B]} that is evaluated in parallel, returning noisy rewards \{y_{t,b}=f(\mathbf{x}_{t,b})+\epsilon_{t,b}\}_{b\in[B]} with observation noise \epsilon_{t,b} . Letting \mathbf{x}^{*}\in\arg\max_{\mathbf{x}}f(\mathbf{x}) , the regret over horizon T is the cumulative gap to f(\mathbf{x}^{*}) , R(T,B;f)=\mathbb{E}\left[\sum_{t=1}^{T}\sum_{b=1}^{B}f(\mathbf{x}^{*})-f(\mathbf{x}_{t,b})\right], (1) where the expectation is over the observation noise and the algorithm’s internal sampling randomness. We assume the regularity conditions in Assumption 1 . Assumption 0 . Given a reproducing kernel Hilbert space (RKHS), the norm of the objective function is bounded: \|f\|_{H_{k}}\leq M_{\mathcal{H}} , for some M_{\mathcal{H}}>0 , and k(\mathbf{x},\mathbf{x}^{\prime})\leq 1 for all \mathbf{x},\mathbf{x}^{\prime}\in\mathcal{X} . The observation noise \epsilon_{t,b} is assumed to be zero-mean and \sigma -sub-Gaussian, independent across (t,b) . 3 Bayesian Optimization Background BO is a sequential decision-making strategy for an expensive black-box f , combining a probabilistic surrogate (e.g. Gaussian Processes [ 29 ] , Bayesian neural networks [ 23 ] , deep ensembles [ 22 ] ) with an acquisition function that, when maximized, decides where to sample next. 3.1 Probabilistic Models A surrogate provides a posterior p(f\mid\mathcal{D}_{t}) over the function space given observed data \mathcal{D}_{t}=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N_{t}} . For most acquisition functions, we only require the marginal predictives p(y\mid\mathbf{x},\mathcal{D}_{t}) , characterized by a mean \mu_{t}(\mathbf{x}) and variance v_{t}(\mathbf{x}) . The canonical choice is the Gaussian Process (GP) (Figure 1 , top-left), but any model providing marginal predictives (e.g. deep kernel learning, Bayesian neural networks, or ensembles) can be substituted. For a set of noisy observations \mathbf{y}_{t} , the GP posterior at a point \mathbf{x} is Gaussian, \mathcal{N}(\mu_{t}(\mathbf{x}),v_{t}(\mathbf{x})) , with predictive mean and variance \mu_{t}(\mathbf{x})=\mathbf{k}_{t}(\mathbf{x})^{\intercal}(\mathbf{K}_{t}+\sigma_{n}^{2}\mathbf{I})^{-1}\mathbf{y}_{t},\qquad v_{t}(\mathbf{x})=k(\mathbf{x},\mathbf{x})-\mathbf{k}_{t}(\mathbf{x})^{\intercal}(\mathbf{K}_{t}+\sigma_{n}^{2}\mathbf{I})^{-1}\mathbf{k}_{t}(\mathbf{x}). (2) Exact GPs are the standard for uncertainty quantification, but their \mathcal{O}(N_{t}^{3}) cost from covariance-matrix inversion becomes a bottleneck as data grows. We therefore use Sparse Variational Gaussian Processes (SVGPs) [ 37 ] , which use m inducing variables \mathbf{u} at locations \mathbf{Z}_{t} to reduce complexity to \mathcal{O}(N_{t}m^{2}) , scaling to the large budgets targeted by B3O without sacrificing principled uncertainty, with inducing-point allocation strategies tailored to high-throughput BO [ 26 ] .

Read the complete text