Reasoning BO: Enhancing Bayesian Optimization with the Long-Context Reasoning Power of LLMs
Zhuo Yang, Lingli Ge, Dong Han, Tianfan Fu, Yuqiang Li · 2025
Read Reasoning BO: Enhancing Bayesian Optimization with the Long-Context Reasoning Power of LLMs by Zhuo Yang, Lingli Ge, Dong Han, Tianfan Fu, Yuqiang Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12833.txt.
Introduction
Zhuo Yang and co-authors present a 2025 arXiv paper on steering Bayesian optimization with reasoning language models. Bayesian methods hunt the peak of costly black-box functions but can stall at local highs and explain little. Reasoning BO lets a large language model guide where to sample next, backed by helper agents and a growing knowledge graph.
As runs proceed, the system offers sampling advice with the science behind it, and its hypotheses evolve. Across ten tasks, from math functions to lab chemistry, the method found better regions faster. In one reaction task it lifted yield to about sixty-one percent against twenty-five for plain Bayesian search. Smaller tuned models matched larger ones.
Catalog record
| CREATOR | Zhuo Yang, Lingli Ge, Dong Han, Tianfan Fu, Yuqiang Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-12833
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning BO: Enhancing Bayesian Optimization with the Long-Context Reasoning Power of LLMs Zhuo Yang, Lingli Ge, Dong Han, Tianfan Fu, Yuqiang Li Abstract Many real-world scientific and industrial applications require the optimization of expensive black-box functions. Bayesian Optimization (BO) provides an effective framework for such problems. However, traditional BO methods are prone to get trapped in local optima and often lack interpretable insights. To address this issue, this paper designs Reasoning BO, a novel framework that leverages reasoning models to guide the sampling process in BO while incorporating multi-agent systems and knowledge graphs for online knowledge accumulation. By integrating the reasoning and contextual understanding capabilities of Large Language Models (LLMs), we can provide strong guidance to enhance the BO process. As the optimization progresses, Reasoning BO provides real-time sampling recommendations along with critical insights grounded in plausible scientific theories, aiding in the discovery of superior solutions within the search space. We systematically evaluate our approach across 10 diverse tasks encompassing synthetic mathematical functions and complex real-world applications. The framework demonstrates its capability to progressively refine sampling strategies through real-time insights and hypothesis evolution, effectively identifying higher-performing regions of the search space for focused exploration. This process highlights the powerful reasoning and context-learning abilities of LLMs in optimization scenarios. For example, in the Direct Arylation task (a chemical reaction yield optimization problem), our method increased the yield to 60.7%, whereas traditional BO achieved only a 25.2% yield. Furthermore, our investigation reveals that smaller LLMs, when fine-tuned through reinforcement learning, can attain comparable performance to their larger counterparts. This enhanced reasoning capability paves the way for more efficient automated scientific experimentation while maintaining computational feasibility. 1 Introduction Real-world scientific optimization problems often involve complex, expensive-to-evaluate functions across diverse domains, from chemical synthesis [ 1 ] to hyperparameter tuning [ 2 , 3 ] . Bayesian Optimization (BO) provides an efficient framework for such black-box problems by constructing probabilistic surrogate models and intelligently guiding experiments through acquisition functions [ 4 ] . While BO outperforms traditional methods in sample efficiency, it faces key limitations: susceptibility to local optima [ 5 ] , sensitivity to initial sampling, and lack of interpretable scientific insights. These challenges motivate our integration of reasoning capabilities with BO. Recent research has explored integrating LLMs to create hybrid intelligent optimization frameworks that overcome Bayesian Optimization’s inherent limitations. These frameworks leverage three key LLM capabilities: First, their cross-domain knowledge from pre-training provides global heuristics to avoid local optima [ 6 ] . Second, domain priors like chemical reaction rules can be naturally injected through language to reduce sensitivity to initial sampling. Most importantly, LLMs generate scientifically plausible mechanistic hypotheses and iteratively refine them through experiments [ 5 ] . However, current LLM-BO integration approaches suffer from two critical limitations: (1) hallucinated optimization suggestions that violate domain constraints, potentially compromising safety, and (2) inability to dynamically incorporate experimental discoveries into subsequent optimization cycles. These shortcomings necessitate the development of memory-augmented closed-loop optimization systems. To address these challenges, we present the Reasoning BO framework (illustrated in Figure 2 ), which incorporates three novel mechanisms for trustworthy intelligent optimization. First, we embed a Reasoning Model within the traditional BO loop, operating through: (a) Experiment Compass interaction where users describe experiments via natural language for automatic configuration generation, (b) LLM-generated hypotheses with confidence scores based on experimental history, knowledge graphs, and BO candidate points, and (c) confidence-based filtering of hypotheses against historical data to select the most scientifically plausible candidates. Second, we construct a dynamic knowledge management system containing structured domain rules in knowledge graphs and encoded unstructured literature in vector databases. This system enables bidirectional interaction—allowing both pre-optimization expert knowledge injection and real-time assimilation of new discoveries. Third, we fine-tune lightweight models using reinforcement learning, with data sourced from previous reasoning trajectories. These components synergistically achieve a complete "hypothesis generation-experimental validation-knowledge accumulation" closed loop. For ease of exposition, we summarize our main contributions as • Incorporation of a reasoning model that leverages large language models’ inference abilities to automatically generate and evolve scientific hypotheses, reducing time costs through automated validation while ensuring scientific plausibility via confidence-based filtering, with reinforcement learning-based knowledge distillation for stable optimization. • Development of a full-cycle knowledge management system enabling seamless encoding and dynamic updating of domain expertise throughout optimization, featuring flexible knowledge architectures with customizable storage formats and open agent interfaces for extensibility. • Demonstrated superior performance in complex scientific optimization tasks requiring extensive domain knowledge, particularly in chemical yield prediction with intricate objective functions. Our framework rapidly identifies promising regions while achieving better convergence speed and final accuracy, substantially reducing research costs by minimizing validation trials and providing a trustworthy optimization solution for high-cost experiments. In our Direct Arylation benchmark, Reasoning BO demonstrates significant improvements over Vanilla BO, achieving a 23.3% higher final yield (94.39% vs 76.60%) and showing 44.6% higher initial performance (66.08% vs 21.62%), showcasing both superior initialization and continuous optimization capabilities. 2 Related Work 2.1 Bayesian Optimization Bayesian Optimization serves as a fundamental paradigm for black-box function optimization, with its core idea being the probabilistic modeling of the objective function through surrogate models that are iteratively updated based on historical observations [ 7 ] . A typical BO workflow consists of two key: (1) A probabilistic surrogate model, typically implemented as a Gaussian Process (GP) that approximates the objective function f(\mathbf{x})\sim\mathcal{N}(\mu(\mathbf{x}),\sigma^{2}(\mathbf{x})) [ 7 ] , where \mu(\mathbf{x}) and \sigma(\mathbf{x}) represent the predicted mean and uncertainty respectively; and (2) an acquisition function that guides the selection of evaluation points by balancing exploration and exploitation. Among various acquisition strategies, Expected Improvement (EI) remains widely adopted due to its theoretical foundations and practical effectiveness [ 8 , 9 ] . In our work, we employ a robust variant of Expected Improvement called qLogEI that effectively handles batch evaluations while maintaining numerical stability [ 10 ] . This acquisition function demonstrates particular advantages when combined with Monte Carlo sampling methods, offering improved performance in high-dimensional optimization scenarios. See Supplementary Material Section D.1 for detailed mathematical formulation and derivation of qLogEI. Despite BO’s theoretical sample efficiency, it faces fundamental challenges in high-cost evaluation scenarios: (1) the computational complexity of surrogate model training grows substantially with the number of evaluations, limiting its feasibility for longer optimization runs [ 11 ] , and (2) standard acquisition functions struggle to dynamically adjust the exploration-exploitation trade-off, often leading to sub-optimal local optima [ 12 ] . In contrast, our approach partially alleviates these limitations by leveraging LLMs’ global heuristic perspective and reasoning capabilities. 2.2 Language-Guided Bayesian Optimization By leveraging LLMs’ contextual understanding [ 13 , 14 ] and few-shot generation capabilities [ 15 ] , recent advances demonstrate their potential to effectively address black-box optimization challenges when integrated with BO [ 16 ] . Current mainstream work in this area can be primarily categorized into two types: (1) LLM-assisted experimental design, which incorporates reasoning capabilities into the development of candidates sampling strategies, and (2) knowledge transfer techniques, which leverage domain-specific insights or user belief to In the first category, SLLMBO [ 17 ] employs a hybrid sampling strategy combining LLMs with Tree-structured Parzen Estimator (TPE) [ 18 ] for efficient hyperparameter optimization. HypBO [ 5 ] and BORA [ 6 ] further accelerate Bayesian