Language-Based Bayesian Optimization Research Assistant (BORA)

Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, Andrew I. Cooper · 2025

Read Language-Based Bayesian Optimization Research Assistant (BORA) by Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, Andrew I. Cooper (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16224.txt.

Introduction

Abdoulatif Cisse and co-authors present in 2025 BORA, a helper that pairs a chance based search method with a large language model for lab work. Many lab tasks pair many knobs with slow, dear tests. Plain search can trap in poor zones of a bent field.

The model feeds field sense into the search to point it at rich zones. It also talks the user through each step in live notes, with its grounds laid out. Fake tests with up to fifteen knobs and four real lab tasks back the blend. Bias risk in human hints is one spur for the tool.

Catalog record

CREATORAbdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, Andrew I. Cooper
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16224

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Language-Based Bayesian Optimization Research Assistant (BORA) Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, Andrew I. Cooper Abstract Many important scientific problems involve multivariate optimization coupled with slow and laborious experimental measurements. These high-dimensional searches can be defined by complex, non-convex optimization landscapes that resemble needle-in-a-haystack surfaces, leading to entrapment in local minima. Contextualizing optimizers with human domain knowledge is a powerful approach to guide searches to localized fruitful regions. However, this approach is susceptible to human confirmation bias. It is also challenging for domain experts to keep track of the rapidly expanding scientific literature. Here, we propose the use of Large Language Models (LLMs) for contextualizing Bayesian optimization (BO) via a hybrid optimization framework that intelligently and economically blends stochastic inference with domain knowledge-based insights from the LLM, which is used to suggest new, better-performing areas of the search space for exploration. Our method fosters user engagement by offering real-time commentary on the optimization progress, explaining the reasoning behind the search strategies. We validate the effectiveness of our approach on synthetic benchmarks with up to 15 variables and demonstrate the ability of LLMs to reason in four real-world experimental tasks where context-aware suggestions boost optimization performance substantially. This work is accepted and will be part of the 34th International Joint Conference on Artificial Intelligence (IJCAI) proceedings. 1 Introduction Figure 1: The BORA framework. Icons from Flaticon (2025) . Exploring large experimental design spaces requires intelligent navigation strategies because of the costly and time-consuming function evaluations involved. Bayesian optimization has been established as an optimal experimental design methodology across disciplines spanning chemistry Häse et al. (2021) , solar energy production Andrés-Thió et al. (2024) and agronomy Zhang et al. (2024) . BO can be used to efficiently navigate combinatorially large landscapes, and to identify promising solutions in an active-learning setting. Typically, BO uses a probabilistic surrogate to approximate an expensive or unknown objective function f while iteratively searching for a maximizer, x^{*}=\operatorname*{argmax}_{x\in\mathcal{X}}f(x), (1) with f:\mathcal{X}\to\mathbb{R} defined on the search domain \mathcal{X}\subseteq\mathbb{R}^{d} . This surrogate – often a Gaussian Process (GP) Garnett (2023) – subsequently undergoes Bayesian updates as new data about the design space are acquired, according to a predefined acquisition policy or function \alpha(\cdot) , allowing for the refinement of predictions of the objective f(x) . In turn, this acquisition function suggests the next set of parameters for experiments with the highest expected utility, balancing exploration in new regions and exploitation in promising ones. Despite its successful application to a plethora of scientific tasks, BO is frequently characterized by slow initial optimization phases due to random or Latin Hypercube Poloczek et al. (2016) selection of initial samples. This can slow the search for combinatorially large spaces substantially. This highlights a fundamental challenge for standard BO; that is, the lack of inherent domain knowledge and contextual understanding of the problem at hand. Recently, BO variants have been proposed that are capable of injecting domain-specific knowledge into the search, either through structural problem characteristics Xie et al. (2023) or by using human expert knowledge Cissé et al. (2024) . The latter approach, sometimes known as ‘Human-in-the-Loop (HIL)’, has drawn considerable recent attention, and it aims to infuse domain knowledge and human reasoning into BO workflows Adachi et al. (2024) ; Huang et al. (2022) . By leveraging expert insights in the form of hypotheses Cissé et al. (2024) , preferences Kristiadi et al. (2024) or specified priors over possible optima Hvarfner et al. (2022) ; Li et al. (2020) , it is possible to enrich the optimization process and to direct searches to fruitful regions much faster. Thus, HIL methods have shown increased effectiveness and efficiency compared with data-only approaches. In particular, hypothesis-based methods have shown gains in both performance and cost. Nonetheless, these HIL approaches can be human-capital resource-intensive because they require regular human interventions. Moreover, it is easy, even for domain experts, to lose track of the state of the art in fast-moving research areas and to ignore certain promising regions of the search space Ou et al. (2022) . To address these challenges, we propose the use of Large Language Models (LLMs) Guo et al. (2024) as a facilitating framework in black-box optimization to enrich searches with domain knowledge. Specifically, we have coupled an LLM with standard BO in a hybrid optimization framework that automatically monitors and regulates the amount of domain knowledge needed when the search becomes ‘trapped’ in local minima. The algorithm capitalizes on the LLM’s inherent in-context learning (ICL) capacity and reasoning mechanism to suggest, in the form of hypotheses, promising areas in the search space from which to sample. LLMs have been employed recently to address limitations in core BO methodologies, as well as HIL variants Liu et al. (2024) ; Yin et al. (2024) . LLMs have the capacity to encode vast amounts of domain-specific and general knowledge and have demonstrated the ability to reason about relatively complex tasks through in-context learning Kroeger et al. (2024) ; Xie et al. (2022) as well as in multidisciplinary domains such as chemistry Ramos et al. (2024) . However, due to their numerically agnostic design, LLMs lag behind traditional BO methods in systematically balancing exploration versus exploitation, and have proved unreliable in many practical scenarios Huang et al. (2024) . Recent attempts have been made to integrate LLMs with BO frameworks Mahammadli (2024) ; Yin et al. (2024) but thus far, these have been limited to small problem sizes, such as hyper-parameter optimization, or situations where the optimal solution is proximal to a special value Huang et al. (2024) . Also, LLM/BO hybrids could be prohibitively costly for more complex queries, particularly if the LLM is deployed for every iteration in the optimization. Here, we propose a language-based Bayesian Optimization Research Assistant, BORA, that enriches BO with domain knowledge and contextual understanding across a range of scientific tasks. We frame BORA as a hybrid framework that augments surrogate-based optimizers with uncertainty estimates by localizing areas of interest in the search space, guided by a knowledge-enriched LLM (Figure 1 ). A heuristic policy regulates the LLM involvement in the optimization process, adaptively balancing rigorous stochastic inference with LLM-generated insights within a feasible budget of LLM computation and API usage limits. During the intervention stage, the LLM uses its domain knowledge and reasoning capabilities to comment on the optimization progress thus far, highlighting patterns observed and forming hypotheses that may yield more rewarding solutions. It then tests these hypotheses by proposing new samples that maximize the target objective. BORA is also designed to provide an effective user-optimizer interaction through its dynamic commentary on the optimization process. This promotes deeper insights from the user and, in the future, the option to intervene; for example, by either reinforcing or overriding certain insights from the LLM. To our knowledge, this is the first time that a rigorous, dynamic synergy of black-box BO with LLMs has been proposed in this context. We evaluated BORA on various synthetic functions and a pétanque gaming model, as well as real scientific tasks in chemical materials design, solar energy production, and crop production. BORA demonstrated significant improvements in search exploration, convergence speed, and optimization awareness. Compared to earlier techniques, our method shows significant efficiency gains and generalization

Read the complete text