CONSTRAINED MULTI-FIDELITY BAYESIAN OPTIMIZATION WITH AUTOMATIC STOP CONDITION

Zahra Zanjani Foumani, Ramin Bostanabad · 2025

Read CONSTRAINED MULTI-FIDELITY BAYESIAN OPTIMIZATION WITH AUTOMATIC STOP CONDITION by Zahra Zanjani Foumani, Ramin Bostanabad (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01126.txt.

Introduction

Zahra Zanjani Foumani and Ramin Bostanabad build, in a 2025 paper, a smarter search method for engineering design when each test is expensive and rules must be respected. Bayesian optimization is the standard tool: it models the design space and picks the most promising next test. High fidelity tests alone still burn budget, and constrained spaces complicate the hunt.

Their framework mixes cheap low fidelity sources with the costly accurate ones, staying aware of cost at every step. Constraints may differ between sources and may themselves be black boxes. A second contribution tackles a chronic gap: when to stop. Their automatic stop condition gives a principled convergence check instead of a gut call. The code is public in a Python package. Benchmarks across several problems showed the savings hold up, which matters wherever simulation or lab time is the scarce resource.

Catalog record

CREATORZahra Zanjani Foumani, Ramin Bostanabad
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01126

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CONSTRAINED MULTI-FIDELITY BAYESIAN OPTIMIZATION WITH AUTOMATIC STOP CONDITION Zahra Zanjani Foumani, Ramin Bostanabad Abstract Bayesian optimization (BO) is increasingly employed in critical applications to find the optimal design with minimal cost. While BO is known for its sample efficiency, relying solely on costly high-fidelity data can still result in high costs. This is especially the case in constrained search spaces where BO must not only optimize but also ensure feasibility. A related issue in the BO literature is the lack of a systematic stopping criterion. To solve these challenges, we develop a constrained cost-aware multi-fidelity BO ( CMFBO ) framework whose goal is to minimize overall sampling costs by utilizing inexpensive low-fidelity sources while ensuring feasibility. In our case, the constraints can change across the data sources and may be even black-box functions. We also introduce a systematic stopping criterion that addresses the long-lasting issue associated with BO’s convergence assessment. Our framework is publicly available on GitHub through the GP+ Python package and herein we validate it’s efficacy on multiple benchmark problems. 1 Introduction Bayesian Optimization (BO) is a sequential and sample-efficient global optimization technique widely used for optimizing expensive-to-evaluate, black-box functions [ 1 ] . BO has two main components: an acquisition function (AF) and an emulator which is often a Gaussian Process (GP). Starting with an initial dataset, BO iteratively refines the emulator by adding new samples using the AF until convergence criteria are satisfied [ 2 ] . Despite its efficiency, BO can be costly when relying solely on accurate and expensive high-fidelity (HF) data. To mitigate this, multi-fidelity (MF) techniques have been developed [ 3 ] to leverage data sources of varying accuracy and cost levels to reduce overall sampling expenses by exploiting the correlations between HF and low-fidelity (LF) datasets. Compared to single-fidelity BO (SFBO), the choice of emulator in MFBO is more critical due to the heterogeneous nature of the data. In MF settings, GPs are still favored over alternatives like probabilistic neural networks which are costly to train and prone to overfit in data-scarce regimes [ 4 , 5 , 6 ] . Numerous MFBO strategies have been recently proposed but they mostly struggle to design appropriate emulators and/or AFs to handle the MF nature of the problem. For instance, many MF methods require prior knowledge about the hierarchy of LF data sources [ 7 ] . Some modeling approaches train separate emulators for each data source and hence fail to fully leverage cross-source correlations [ 8 , 9 ] . Popular bi-fidelity approaches based on Kennedy and O’Hagan’s work and its extensions [ 10 ] are limited to two fidelities and often assume simple bias forms (e.g., additive functions) for LF sources [ 11 ] . Similarly, Co-Kriging and its extensions [ 12 , 13 , 14 ] often fail to accurately model cross-fidelity relationships. The limitations on MF emulation are typically exacerbated by the AF as it may encourage over-sampling of very cheap LF sources. Such a behavior leads to inaccurate results and affects even popular packages like botorch [ 15 ] . Existing BO methods are primarily designed for unconstrained problems. While there are a few constrained strategies available, they are mostly limited to SF cases [ 16 , 17 , 18 , 19 ] . A few MFBO methods address constrained problems but they either suffer from the same emulation limitations discussed above [ 20 , 21 ] or are restricted to bi-fidelity settings [ 22 , 23 , 24 ] . Most constrained MFBO methods rely on the Probability of Feasibility (PoF) to account for feasibility in their AFs [ 25 , 23 , 26 , 27 ] . These methods require a feasibility threshold which cannot be chosen systematically and is sensitive to noisy data. Additionally, PoF requires an initially feasible solution to proceed [ 28 , 29 ] which is an assumption that is regularly violated in high-dimensional spaces. Additionally, PoF does not consider the severity of constraint violations and hence fails to direct the search towards feasible designs. Although some constrained MFBO methods don’t rely on PoF, they also have some limitations. Many assume identical constraints across fidelity levels but LF sources typically fail to provide an accurate representation of the HF source’s constraints [ 30 , 31 , 32 ] . Others adopt multi-step look-ahead frameworks to improve feasibility, but this comes at the cost of increased complexity and computational overhead [ 33 , 34 ] . The stop condition is a critical component of all BO methods including MF, constrained, and other versions. In iterative processes like BO, defining a proper stopping criterion is essential to balance cost and solution quality. Too many iterations waste resources while stopping prematurely risks suboptimal solutions. Despite its importance, this topic has received limited attention. Most approaches use simple and problem-dependent metrics such as maximum number of iterations or improvement threshold [ 35 , 36 , 37 ] . Our recent work, MFBO [ 38 ] , addresses most of the above limitations in two ways. First, it designs GPs [ 39 ] that can simultaneously fuse any number of data sources without imposing any constraint or assumption on the hierarchy of the data sources (e.g., linear correlation among sources, additive biases, noise level, etc.). Second, it quantifies the information value of LF and HF samples differently to consider the MF nature of the data while exploring the search space [ 40 ] . Specifically, the AF used in MFBO is cost-aware in that it considers the sampling cost in quantifying the value of HF and LF data points. While MFBO performs well, it shares two key limitations with other MFBO methods: (1) the inability to handle constraints, (2) the lack of systematic stopping criteria. We introduce CMFBO in this paper to address these issues. The rest of the paper is organized as follows. We introduce our approach in Section 2 and then evaluate its performance on multiple benchmark problems in Section 3 . We conclude the paper in Section 4 by summarizing our contributions and providing future research directions. 2 Methods We begin by providing background on MF modeling with GPs in Section 2.1 and then introduce our novel AF for handling constraints in Section 2.2 . Finally, in Section 2.3 we develop a stop condition that automatically adapts to the problem. Our approach is implemented via the open-source Python package GP+ [ 41 ] . 2.1 Emulation with Gaussian Processes (GPs) We first review GPs and their extension to handle mixed-input domains which seamlessly enables MF emulation. Then, we introduce a few mechanisms to improve the uncertainty quantification (UQ) accuracy of our emulator which directly benefits MFBO. 2.1.1 Mixed-Input Emulation via GPs GPs are probabilistic models that assume the training data follows a multivariate normal distribution whose mean vector and covariance matrix depend on the inputs. Assume the training dataset \mathopen{}\left\{\boldsymbol{x}^{(i)},y^{(i)}\right\}\mathclose{}_{i=1}^{n} is given where \boldsymbol{x}=[x_{1},...,x_{dx}]\in\mathbb{X}\subset\mathbb{R}^{dx} and y^{(i)}=y(\boldsymbol{x}^{(i)})\in\mathbb{R} denote the inputs and response, respectively 1 1 1 Superscript numbers enclosed in parenthesis indicate sample numbers. For instance, x^{(i)} denotes the i^{th} sample in a training dataset while x_{i} indicates the i^{th} component of the vector \boldsymbol{x}=[x_{1},\cdots,x_{dx}] . . Given \boldsymbol{y}=[y^{(1)},\cdots,y^{(n)}]^{T} and \boldsymbol{X} whose i^{th} row is

Read the complete text