Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models
Niklas Schweiger, Daniel Cremers, Karnik Ram · 2026
Read Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models by Niklas Schweiger, Daniel Cremers, Karnik Ram (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14504.txt.
Introduction
Schweiger, Cremers and Ram tune diffusion and flow models by searching over input noise. Tuning the starting noise can align outputs with a target reward. Past methods need smooth or cheap reward models and can waste memory. Their trust-region search treats both models as black boxes. It balances broad search with local refinement of the noise. Only a few settings need adjustment. Tests cover text-to-image work plus molecule and protein design. Outputs beat the base models and rival tuning methods, the authors report.
Catalog record
| CREATOR | Niklas Schweiger, Daniel Cremers, Karnik Ram |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77846 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14504
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models Niklas Schweiger, Daniel Cremers, Karnik Ram Abstract Optimizing the noise samples of diffusion and flow models is an increasingly popular approach to align these models to target rewards at inference time. However, we observe that these approaches are usually restricted to differentiable or cheap reward models, the formulation of the underlying pre-trained generative model, or are memory/compute inefficient. We instead propose a simple trust-region based search algorithm (TRS) which treats the pre-trained generative and reward models as a black-box and only optimizes the source noise. Our approach achieves a good balance between global exploration and local exploitation, and is versatile and easily adaptable to various generative settings and reward models with minimal hyperparameter tuning. We evaluate TRS across text-to-image, molecule and protein design tasks, and obtain significantly improved output samples over the base generative models and other inference-time alignment approaches which optimize the source noise sample, or even the entire reverse-time sampling noise trajectories in the case of diffusion models. Our source code is publicly available 1 1 1 Project page: https://niklasschweiger.github.io/trust-region-noise-search/ . † † footnotetext: † Equal supervision. Aesthetic Score ( \uparrow ) 6.9 7.5 7.9 7.7 8.1 8.2 Property Alignment ( \downarrow ) 0.75 0.65 0.63 0.43 0.41 0.35 Designability ( \uparrow ) 0.041 0.057 0.059 0.201 0.346 0.506 Figure 1: Progression of output samples produced by our algorithm (TRS) across iterations i\in\{0,\dots,15\} for text-to-image, molecules, and proteins. Top : Aesthetic reward alignment for the prompt “Animated movie poster…”. Middle : Molecule property alignment. Bottom : Protein designability. The horizontal arrow indicates the direction of optimization, while the arrows \uparrow/\downarrow indicate whether the objective is maximized or minimized. 1 Introduction Generative models such as diffusion and flow-based models have revolutionized diverse domains, including high-fidelity image and video synthesis, molecule generation, and language modeling, by learning complex data distributions from large-scale pretraining data ( Ho et al. 2020 ; Lipman et al. 2023 ) . While scaling model capacity and training compute has significantly improved sample quality, pretrained models often fall short of specific, fine-grained requirements. This includes generating molecules with precise binding affinities ( Guan et al. 2023 ) or images that strictly align with complex, multi-attribute prompts. This gap has motivated the emergence of inference-time alignment for diffusion and flow-based models, an exciting new paradigm where the quality of generated samples is optimized post-training using feedback from target reward models ( Bansal et al. 2024 ; Eyring et al. 2024 ) . Unlike fine-tuning, this approach requires no additional training data and instead invests additional compute at inference time to steer samples toward desired properties. Several approaches for inference-time alignment have been proposed. Gradient-based methods back-propagate through the entire (often ODE-based) iterative process of the model to adjust the initial noise sample ( Ben-Hamu et al. 2024 ; Wang et al. 2025 ) . But these are prone to high GPU-memory costs and risk drifting off the training-data manifold. Sequence-based methods, typically used for SDE samplers, range from filtering approaches that iteratively reweigh and resample ( Singhal et al. 2025 ; Kim et al. 2025 ) to tree-search based methods ( Jain et al. 2025 ; Li et al. 2025a ) . But these often require a high number of expensive reward calls or rely on accurate value estimates for terminal rewards, which are not always available or reliable. In contrast, black-box search methods which are relative underexplored, treat the generator and reward models as a black box and apply search heuristics for the source noise sample ( Ma et al. 2025 ; Tan et al. 2025 ) . While versatile and applicable to any generative architecture and reward model, we notice that existing approaches often struggle to find a good balance between global exploration and local exploration, tending toward one extreme or the other. Inspired by Bayesian optimization algorithms ( Eriksson et al. 2019 ) , which also optimize expensive black-box functions, we propose a simple and effective trust region search (TRS) approach that balances between global exploration and local exploitation in a structured manner. By controlling only the source noise, TRS is readily applicable to a wide range of generative models and reward functions without internal modifications. TRS begins by exploring multiple seed noise samples, which are then pruned and iteratively refined with local perturbations. Crucially, these perturbations are adaptively controlled in both magnitude and direction based on observed reward values, ensuring the search remains within the data manifold to produce stable and high-quality samples. The contributions of our work are as follows: 1. We introduce trust region search (TRS), a simple approach for inference-time reward alignment of black-box diffusion and flow models via adaptive source noise control. 2. We provide an extensive evaluation on text-to-image generation, demonstrating that TRS yields significantly more aligned and higher-quality samples compared to existing search heuristics and even full-noise sequence search baselines under identical compute budgets. 3. We demonstrate the versatility of our approach through an extended evaluation on small molecule and protein design tasks, where TRS proves effective even with expensive reward functions and requires minimal hyperparameter tuning. 2 Background and Related Work 2.1 Diffusion and Flow-based Models Diffusion and flow-based generative models aim to transport a simple noise distribution p_{\text{noise}}=p_{0} to a complex data distribution p_{\text{data}}=p_{1} . A sample \mathbf{x}_{0}\sim p_{0} is gradually transformed into a data sample \mathbf{x}_{1}\sim p_{1} through a continuous-time process \{\mathbf{x}_{t}\}_{t\in[0,1]} , typically discretized into T steps with t_{k}=k/T . In continuous data regimes, p_{0} is usually white Gaussian noise \mathcal{N}(\mathbf{0},\mathbf{I}) . Diffusion models are characterized by a forward noising process that gradually transforms data into noise. This process is governed by a stochastic differential equation (SDE): d\mathbf{x}_{t}=\mathbf{f}(t)\mathbf{x}_{t}dt+g(t)d\mathbf{w}_{t}, where \mathbf{f}(t) and g(t) are the drift and diffusion coefficients that determine the noising schedule ( Song et al. 2021 ) . To generate new samples, this process must be reversed. The generative (reverse) process relies on the score function \bm{\epsilon}(\mathbf{x}_{t},t)\triangleq\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x_{t}}) , which represents the gradient of the log-density of the noisy sample. Once the score is estimated via a neural network, sampling can be performed using either the deterministic or stochastic formulations shown in Table 1 . Flow matching models instead learn a continuous vector field \mathbf{v}_{\theta}(\mathbf{x}_{t},t)=d\mathbf{x}_{t}/dt ( Lipman et al. 2023 ) that defines the velocity of the samples. Unlike diffusion, the paths are not necessarily tied to a corruption process; a common choice is the optimal transport (OT) path ( Liu et al. 2023 ) , defined by linear interpolation \mathbf{x}_{t}=(1-t)\mathbf{x}_{0}+t\mathbf{x}_{1}, which yields a constant target velocity \mathbf{v}_{t}=\mathbf{x}_{1}-\mathbf{x}_{0} . Standard flow matching integrates the learned ODE for sampling, while stochastic variants ( Bose et al. 2024 ) can introduce Brownian motion and optional score corrections as detailed in Table 1 . Unified view. Both paradigms can be expressed in a general form d\mathbf{x}_{t}=\mathbf{f}(\mathbf{x}_{t},t)dt+g(t)d\mathbf{w}_{t} . As summarized in Table 1 , the two frameworks differ primarily in how the drift \mathbf{f}(\mathbf{x}_{t},t) is parameterized, either through the score function \bm{\epsilon}(\mathbf{x}_{t},t) to reverse a corruption process, or through a velocity field \mathbf{v}_{\theta}(\mathbf{x}_{t},t) to match a probability flow. Table 1: Generative sampling formulations ( t=0 as noise, t=1 as data). Note that for diffusion, \bm{\epsilon}(\mathbf{x}_{t},t) denotes the score function \nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}) . Formulation ODE