Diffusion Large Language Models for Black-Box Optimization

Ye Yuan, Can (Sam) Chen, Zipeng Sun, Dinghuai Zhang, Christopher Pal, Xue Liu · 2026

Read Diffusion Large Language Models for Black-Box Optimization by Ye Yuan, Can (Sam) Chen, Zipeng Sun, Dinghuai Zhang, Christopher Pal, Xue Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14446.txt.

Introduction

Ye Yuan, Can Chen, Zipeng Sun, and their co-authors use diffusion language models for design search in this 2026 paper. Offline black-box optimization aims to find optimal designs from a fixed dataset of designs and labels. No new tests can be run during the search. Such cases arise in DNA sequence design and robotics, where labeled data are scarce.

The team's tool is a diffusion large language model, a model that builds text by gradual denoising. They adapt it to propose strong designs from the offline set alone. Readers get a new way to squeeze value from small, costly datasets where each label was hard to earn.

Catalog record

CREATORYe Yuan, Can (Sam) Chen, Zipeng Sun, Dinghuai Zhang, Christopher Pal, Xue Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-14446

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Diffusion Large Language Models for Black-Box Optimization Ye Yuan, Can (Sam) Chen, Zipeng Sun, Dinghuai Zhang, Christopher Pal, Xue Liu Abstract Offline black-box optimization (BBO) aims to find optimal designs based solely on an offline dataset of designs and their labels. Such scenarios frequently arise in domains like DNA sequence design and robotics, where only a few labeled data points are available. Traditional methods typically rely on task-specific proxy or generative models, overlooking the in-context learning capabilities of pre-trained large language models (LLMs). Recent efforts have adapted autoregressive LLMs to BBO by framing task descriptions and offline datasets as natural language prompts, enabling direct design generation. However, these designs often contain bidirectional dependencies, which left-to-right models struggle to capture. In this paper, we explore diffusion LLMs for BBO, leveraging their bidirectional modeling and iterative refinement capabilities. This motivates our in-context denoising module: we condition the diffusion LLM on the task description and the offline dataset, both formatted in natural language, and prompt it to denoise masked designs into improved candidates. To guide the generation toward high-performing designs, we introduce masked diffusion tree search , which casts the denoising process as a step-wise Monte Carlo Tree Search that dynamically balances exploration and exploitation. Each node represents a partially masked design, each denoising step is an action, and candidates are evaluated via expected improvement under a Gaussian Process trained on the offline dataset. Our method, dLLM , achieves state-of-the-art results in few-shot settings on design-bench. 1 Introduction Designing new objects or entities to optimize specific properties is a fundamental challenge across domains such as DNA sequence design and robotics 38 . Since querying these properties is often expensive 12 ; 2 ; 4 ; 32 , recent efforts have focused on offline settings that avoid online evaluations. This gives rise to offline black-box optimization (BBO) ( 16 ) , where the goal is to discover high-performing designs using only an offline dataset of candidates and their associated labels. The challenge is particularly pronounced when only a few labeled data points are available. Traditional methods typically either (1) train a proxy model ( 39 ; 48 ; 14 ; 47 ; 10 ) , which provides explicit gradient guidance over existing designs, or (2) train a conditional generative model ( 19 ; 18 ; 6 ) that takes a target value as input to directly generate high-scoring designs. However, such task-specific and data-limited methods are prone to out-of-distribution generalization issues and overlook the general in-context learning capabilities of pre-trained large language models (LLMs) ( 7 ) , which offer a versatile alternative. Recent efforts have adapted autoregressive LLMs for BBO by framing task descriptions and offline datasets as natural language prompts. These models generate improved candidate designs directly through in-context learning ( 43 ; 50 ; 21 ; 27 ; 40 ; 30 ) . However, many design tasks involve bidirectional dependencies—for instance, a DNA unit is influenced not only by its prefix but also by its suffix. Since autoregressive LLMs generate sequences in a left-to-right fashion, they struggle to fully model such dependencies, especially in high-dimensional and complex design spaces. In this paper, we explore diffusion LLMs for BBO. These models offer two key advantages: bidirectional modeling and iterative refinement, making them well-suited for design tasks with complex dependencies. As shown in Figure 1 , we introduce an in-context denoising module: the task description and the offline dataset are formatted as natural language prompts. These prompts, followed by an instruction to generate improved designs, are then provided to the diffusion LLM, which iteratively denoise masked designs into improved candidates. Figure 1: In-context denoising. Figure 2: Masked diffusion tree search. In-context denoising alone is often insufficient for guiding generation toward high-performing designs. To address this, we introduce a masked diffusion tree search module (Figure 2 ), which casts the denoising process as a step-wise Monte Carlo Tree Search (MCTS): each masked design is a tree node, and each denoising step is an action. The search proceeds through four stages to dynamically balance exploration and exploitation: (1) Selection : starting from the root node \bm{x}_{t=1} (fully masked design), traverse the tree using the UCT score ( 17 ) , which balances the exploitation of high-value nodes with the exploration of less-visited ones, to identify a promising node (e.g., select \bm{x}_{t=0.75,2} over \bm{x}_{t=0.75,1} due to its higher UCT score); (2) Expansion : apply diffusion LLM denoising to the selected node (e.g., \bm{x}_{t=0.75,2} ) to generate new child nodes (e.g., \bm{x}_{t=0.5,1} , \bm{x}_{t=0.5,2} ); (3) Evaluation : unroll each child node (e.g., \bm{x}_{t=0.5,1} ) into complete designs (e.g., \bm{x}_{t=0,1} ) and estimate its reward using expected improvement under a Gaussian Process; (4) Backpropagation : propagate the reward signal up the tree (e.g., \bm{x}_{t=0.5,1}\rightarrow\bm{x}_{t=0.75,2}\rightarrow\bm{x}_{t=1} ), thereby updating ancestors and informing future decisions. To summarize, our contributions are three-fold: • To the best of our knowledge, we are the first to explore diffusion LLMs for BBO, leveraging their bidirectional modeling and iterative refinement capabilities. • We propose in-context denoising , which conditions diffusion LLMs on natural language task descriptions and offline datasets to denoise masked designs into improved candidates. • We introduce masked diffusion tree search , which casts diffusion LLM denoising as a step-wise Monte Carlo Tree Search that adaptively balances exploration and exploitation. 2 Preliminaries 2.1 Offline Black-Box Optimization Offline black-box optimization (BBO) aims to find the optimal design \bm{x}^{*} that maximizes an unknown objective function f(\cdot) : \bm{x}^{*}=\arg\max_{\bm{x}}f(\bm{x}). (1) To tackle this, we assume access to an offline dataset \mathcal{D}=\{(\bm{x}_{i},y_{i})\}_{i=1}^{N} containing N data points, where each \bm{x}_{i}\in\mathbb{R}^{d} represents a d -dimensional design and y_{i} is its associated score. In many practical scenarios, only a few labeled data points are available; thus, we focus on the few-shot regime where N is small (e.g., N=10 ). A common method is to fit a deep neural network (DNN) model to approximate f(\cdot) in a supervised manner 8 , and then leverage it to guide the design optimization. However, in the few-shot setting, Gaussian Process (GP) models are often more effective than learned DNNs due to the limited data and their ability to provide principled uncertainty estimates. Specifically, we adopt a GP model ( 22 ) with a radial basis function (RBF) kernel K(\cdot,\cdot) . Given an offline dataset \bm{X}\in\mathbb{R}^{N\times d} and its corresponding labels \bm{y}\in\mathbb{R}^{N} , we model a new design \bm{x} by computing the predictive posterior distribution: \hat{y}(\bm{x})\sim\mathcal{N}(\mu(\bm{x}),\sigma^{2}(\bm{x})). (2) 2.2 Diffusion Large Language Models Large language models (LLMs) are widely used in natural language processing tasks. In contrast to conventional autoregressive models (ARMs) such as GPT 7 , diffusion LLMs 29 ; 11 ; 3 have emerged as a promising alternative, achieving competitive results with models of comparable size. Compared to ARMs, diffusion LLMs offer two key advantages: (1) the ability to leverage bidirectional context, and (2) an iterative sampling process that naturally scales at test time. A diffusion LLM defines a distribution p_{\bm{\theta}}(\bm{x}_{0}) via a two-step process: a forward corruption process and a reverse denoising process . In the forward process, tokens in the original sequence \bm{x}_{0} are progressively masked until, at t=1 , the entire sequence is fully masked. For any intermediate timestep t\in(0,1) , the sequence \bm{x}_{t} is partially masked, with each token masked

Read the complete text