LILO : Bayesian Optimization with Interactive Natural Language Feedback

Katarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy · 2025

Read LILO : Bayesian Optimization with Interactive Natural Language Feedback by Katarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17671.txt.

Introduction

Katarzyna Kobalczyk, with Zhiyuan Jerry Lin and four co-authors, lets people steer optimization with plain words in a 2025 arXiv paper. Many design problems depend on tastes that resist neat formulas. Bayesian optimization is a smart search method that builds a statistical model of the goal. Older preference versions only accept scores or pairwise picks from the decision maker.

Their framework, called LILO, adds a large language model as a translator. It turns free form feedback and prior knowledge into structured preference signals. A Gaussian process model then folds those signals into the search with honest uncertainty. The language model supports the search rather than running it. Tests on synthetic and real problems beat both older preference methods and language model only optimizers.

Catalog record

CREATORKatarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17671

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LILO : Bayesian Optimization with Interactive Natural Language Feedback Katarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy Abstract For many real-world applications, feedback is essential in translating complex, nuanced, or subjective goals into quantifiable optimization objectives. We propose a language-in-the-loop framework that uses a large language model (LLM) to convert unstructured feedback in the form of natural language into scalar utilities to conduct BO over a numeric search space. Unlike preferential BO, which only accepts restricted feedback formats and requires customized models for each domain-specific problem, our approach leverages LLMs to turn varied types of textual feedback into consistent utility signals and to easily include flexible user priors without manual kernel design. At the same time, our method maintains the sample efficiency and principled uncertainty quantification of BO. We show that this hybrid method not only provides a more natural interface to the decision maker but also outperforms conventional BO baselines and LLM-only optimizers, particularly in feedback-limited regimes. 1 Introduction Figure 1: Core steps of LILO . Bayesian optimization (BO) is a powerful strategy for optimizing expensive-to-evaluate black-box objectives ( Brochu et al. 2010 ; Shahriari et al. 2015 ; Frazier 2018 ) . Real BO deployments, however, frequently optimize for complex tradeoffs over many outcomes, where the objective is not clearly specified, and optimization must be guided by feedback from a decision maker (DM) rather than by direct access to ground-truth utility values. Existing preference BO methods address this by learning from comparisons or ratings ( González et al. 2017 ; Chu & Ghahramani 2005 ; Lin et al. 2022 ; Feng et al. 2024 ) , yet they typically require narrow feedback formats, custom likelihoods models, and careful kernel or hyper-prior engineering for the surrogate model. In contrast, black box optimization approaches driven by large language models (LLMs) end-to-end offer a more flexible interface: they can translate rich natural language input and bring in domain knowledge ( Liu et al. 2024 ; Yang et al. 2023 ; Brown et al. 2020 ) . But BO, such approaches usually lack calibrated uncertainty estimates, principled exploration–exploitation tradeoffs, and reliable convergence guarantees. In this paper, we introduce L anguage- i n-the- l oop O ptimization ( LILO ), a framework designed to combine the complementary strengths of BO and LLMs while avoiding their respective weaknesses. Rather than letting the LLM directly control the optimization, we place it in a supporting role: the LLM interprets free-form feedback and prior knowledge expressed in natural language, and translates them into structured signals for Gaussian process (GP) utility surrogates. This design leverages LLMs’ strengths at understanding language and contextual reasoning, while addressing the instability, poor sample efficiency, and weak generalization that arise when relying on LLMs’ raw in-context learning (ICL) abilities ( Brown et al. 2020 ; Zhao et al. 2024 ; Yin et al. 2024 ) . By mapping natural language feedback into quantitative utilities that GPs can model, LILO preserves BO’s calibrated uncertainty and acquisition mechanisms. Beyond showing that such a hybrid is feasible, we provide a systematic study of the design choices that make it effective and practical. The result is a principled, empirically validated framework that combines the expressivity of LLMs with the reliability of BO. In summary, our main contributions are as follows: 1. We propose LILO , a novel BO approach for settings that require eliciting utility information from a DM. LILO leverages natural language feedback, which can be more information-rich than pairwise comparisons or scalar ratings commonly used in other methods. 2. We show how to translate such natural language feedback into quantitative latent utilities that can be used effectively by a surrogate and acquisition function, systematically exploring the design choices required to render this approach both effective and practical. 3. We demonstrate the improved performance and sample efficiency of LILO compared to existing baselines on a variety of synthetic and real-world environments, outperforming both classic GP-based and LLM-based Bayesian optimization methods. 4. We show that the LLM is able to not merely serve as a “utility translator”, but also successfully leverage its domain knowledge acquired during pre-training, enhancing the optimization. 1.1 Problem Definition We consider the problem of optimizing the outputs of a black-box system with respect to preferences of a DM . Let {\mathcal{X}}~\subset~\mathbb{R}^{d} denote the search space of controllable parameters, and let each y\in{\mathcal{Y}}\subset{\mathbb{R}}^{k} define the outcomes of an experiment obtained through an expensive-to-evaluate function y=f(x) , where f:{\mathcal{X}}\to{\mathcal{Y}} maps parameters to outcomes in the outcome space {\mathcal{Y}} . A DM associates with each observed outcome y a latent utility value u=g(y)=(g\circ f)(x)\in{\mathbb{R}} , where g:{\mathcal{Y}}\to{\mathbb{R}} is an unknown utility function that reflects the DM’s preferences. In practice, the DM is often unable to directly specify the closed form of the utility function, so the exact scalar utility value is not directly observable. Instead, the DM can provide feedback, e.g., based on pairwise comparisons ( Lin et al. 2022 ) or, in this work, natural language, to inform about the shape of the utility function g . Our goal is to identify parameters x\in{\mathcal{X}} that maximize satisfaction of the DM, i.e., to solve the composite optimization problem defined as: x^{*}:=\arg\max_{x\in{\mathcal{X}}}\;g(f(x)). For example, as illustrated in Figure 1 , in hyperparameter tuning for machine learning models, x would be the collection of hyperparameters such as batch size or the learning rate and y would be the model’s evaluation metrics such as accuracy, training time, and memory usage. Given a specific set of outcomes y , the utility u reflects the DM’s overall satisfaction about the observed outcomes y . Note that this is a highly flexible setup that can accommodate a wide variety of problems. When there is no intermediate outcome that can be observed and the preference is directly defined over the parameter space {\mathcal{X}} , it is equivalent to letting y=f(x)=x , reducing this problem to the classic preferential BO (PBO) problem ( González et al. 2017 ; Astudillo et al. 2023 ) . When the y is high-dimensional (e.g., image or text) or involves hard-to-quantify outcomes (e.g., the decision maker’s subjective feelings), the proposed method is still applicable by modeling the composite structure (g\circ f)(x) directly instead of f and g separately and skipping explicit modeling of y completely. 2 Background & Related Works Preferential Bayesian Optimization. Classic BO methods ( Shahriari et al. 2015 ; Frazier 2018 ) combine a probabilistic surrogate model – typically a GP – with an acquisition function that balances exploration and exploitation. Preference learning extends this idea to settings where explicit numerical objectives are unavailable and instead only preferential feedback (e.g. in form of pairwise or comparative judgments) can be obtained from a decision maker. The seminal work of Chu & Ghahramani 2005 introduced GP preference learning with a probit likelihood to model noisy pairwise data. González et al. 2017 formalized preferential BO, providing algorithms and guarantees for optimization from pairwise comparisons. Lin et al. 2022 extended this idea with Bayesian optimization with preference exploration , where DMs provide feedback on hypothetical outcomes of experiments in order to best learn the DM’s utility function. This decision-theoretic policy, Expected Utility of the Best (EUBO), had been extended to the best-out-of-k selection in Astudillo

Read the complete text