Improving Classifier-Free Guidance of Flow Matching via Manifold Projection
Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang · 2026
Read Improving Classifier-Free Guidance of Flow Matching via Manifold Projection by Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21892.txt.
Introduction
Jian-Feng Cai, Haixia Liu, Zhengyi Su, and Chao Wang revisit a standard trick for steering image generation. Classifier-free guidance lets users push a diffusion or flow model toward a prompt by extrapolating between two predictions. It works well in practice, but it rests on a heuristic linear step that is sensitive to the guidance scale.
The authors give the method a principled reading through optimization. On that basis, their 2026 preprint improves guidance for flow matching. It projects the guided path onto a manifold. That keeps the path on the surface where valid data live. Better guidance means stronger prompt control with fewer artifacts, a daily concern for everyone generating images and video with these models.
Catalog record
| CREATOR | Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21892
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Classifier-Free Guidance of Flow Matching via Manifold Projection Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang Abstract Classifier-free guidance (CFG) is a widely used technique for controllable generation in diffusion and flow-based models. Despite its empirical success, CFG relies on a heuristic linear extrapolation that is often sensitive to the guidance scale. In this work, we provide a principled interpretation of CFG through the lens of optimization. We demonstrate that the velocity field in flow matching corresponds to the gradient of a sequence of smoothed distance functions, which guides latent variables toward the scaled target image set. This perspective reveals that the standard CFG formulation is an approximation of this gradient, where the prediction gap, the discrepancy between conditional and unconditional outputs, governs guidance sensitivity. Leveraging this insight, we reformulate the CFG sampling as a homotopy optimization with a manifold constraint. This formulation necessitates a manifold projection step, which we implement via an incremental gradient descent scheme during sampling. To improve computational efficiency and stability, we further enhance this iterative process with Anderson Acceleration without requiring additional model evaluations. Our proposed methods are training-free and consistently refine generation fidelity, prompt alignment, and robustness to the guidance scale. We validate their effectiveness across diverse benchmarks, demonstrating significant improvements on large-scale models such as DiT-XL-2-256, Flux, and Stable Diffusion 3.5. Code is available at https://github.com/LeonSuZhengYi/CFG-MP . † † affiliationnotice: Authors are listed in alphabetical order. 1 Introduction Diffusion and flow methods now lead the field for producing top-tier images and videos ( 13 ; 33 ; 19 ; 21 ) . Notably, Flow Matching (FM), which trains a neural network to approximate a specified vector field that transports random noise to data samples, has become a simulation-free training strategy for flow-based generative models, delivering excellent numerical stability, lower computational overhead, and strong modeling flexibility. In conditional generation tasks, such as class-conditional or text-to-image synthesis, Classifier-Free Guidance (CFG) ( 14 ) has become the dominant mechanism for steering generation toward desired conditions. CFG operates by linearly extrapolating between unconditional and conditional model predictions, controlled by a scalar guidance scale w . Despite its simplicity and empirical success, CFG exhibits a well-known trade-off: small values of w result in weak conditional alignment, whereas large values often cause oversaturation, artifacts, or loss of diversity. In practice, the choice of w is highly sensitive and model-dependent, and is typically selected through ad-hoc tuning. This sensitivity highlights a limitation of CFG in complex sampling scenarios. To address this limitation, various refinements have been proposed. In the diffusion context, methods proposed in 22 ; 2 ; 5 ; 35 ; 28 ; 16 ; 40 focus on stabilizing the guidance process. Similarly, flow-specific variants 8 ; 30 have sought to improve sampling quality. Detailed descriptions of these related studies are provided in Related works (Section A ). Notably, 35 suggest that reducing the prediction gap, the discrepancy between unconditional and conditional predictions, can significantly enhance sampling performance. While empirically effective, existing approaches leave an important gap. On one hand, many methods are diffusion-centric, relying on discretization or noise-schedule properties that do not directly transfer to flow matching, where sampling follows a continuous-time ODE defined by a velocity field. On the other hand, the connection between prediction-gap reduction and improved sampling remains largely unexplained from a theoretical standpoint. More precisely, CFG lacks a theoretical interpretation with three key remaining questions. First, what is the objective function that the standard CFG extrapolation actually approximates when applied to flow-matching models? Second, does there exist a mathematically optimal guidance scale w , and if so, how to define and estimate? Third, why do recent empirical methods that reduce the discrepancy between conditional and unconditional predictions, often referred to as the “prediction gap”, consistently improve generation quality and robustness? Addressing these questions requires moving beyond heuristic interpretations of CFG toward an optimization-based understanding. In this work, we bridge this gap by reinterpreting conditional flow-matching sampling through the lens of optimization. We show that the ideal conditional velocity field can be expressed as the gradient of a sequence of regularized distance objectives, where each objective attracts noisy latent variables toward a scaled conditional image set. From this perspective, standard CFG emerges as an approximation to this gradient field. Crucially, this interpretation allows us to formally define an optimal guidance scale w^{\ast} as the minimizer of the approximation error. Building on this result, we decompose the approximation error as the sum of an unavoidable model error term and a scaled prediction gap. This decomposition provides a theoretical explanation that eliminating the prediction gap improves robustness and reduces sensitivity to the guidance scale. Building on this analysis, we introduce a manifold constraint to eliminate the prediction gap and propose a novel CFG-based sampling method via a homotopy optimization process under this constraint. This leads to a simple yet effective sampling scheme, termed CFG-MP, which incorporates an iterative projection toward this manifold in each step of CFG sampling. To address the potential computational overhead of iterative projection, we utilize Anderson Acceleration, yielding CFG-MP+, which significantly improves convergence speed and stability in practice. Our main contributions are summarized as follows: • Optimization-based interpretation of CFG sampling process. We show that the conditional velocity field in flow matching corresponds to the gradient of a regularized distance objective. This interpretation allows us to formally define the optimal guidance scale as the optimal approximation of the true conditional gradient. • Theoretical analysis of guidance sensitivity. We derive an error decomposition for CFG-based sampling that explicitly characterizes the interplay between guidance scale sensitivity and the prediction gap between conditional and unconditional model outputs. • CFG-MP: a manifold projection sampling method. We formulate conditional sampling as a constrained optimization problem and propose CFG-MP, an incremental gradient descent scheme that reduces the prediction gap during sampling without modifying or retraining the underlying generative model. • CFG-MP+: an acceleration of CFG-MP. We integrate Anderson Acceleration into CFG-MP to stabilize and accelerate convergence, substantially enlarging the practical stability region of iterative projection methods. Extensive experiments on class-to-image and text-to-image tasks demonstrate that our method outperforms state-of-the-art CFG variants across multiple models and datasets. 2 Preliminary Conditional and unconditional flow matching. We begin with a review of conditional flow matching (CFM). Let \mathcal{Y} denote the space of conditioning signals (e.g., text prompts), CFM aims to learn a velocity field v_{\theta}:[0,1]\times\mathbb{R}^{d}\times\mathcal{Y}\to\mathbb{R}^{d} that is represented by a neural network with parameters \theta . Given y\in\mathcal{Y} , the velocity field v_{\theta}(\cdot,\cdot,y) generates a probability density path connecting the Gaussian distribution p_{0}:=\mathcal{N}(0,I) and the conditional data distribution p_{1}^{y}:=p_{1}(\cdot|y) by an ordinary differential equation (ODE): \tfrac{d}{dt}x_{t}=v_{\theta}(t,x,y),\ x_{0}\sim p_{0},\ x_{1}^{y}\sim p_{1}^{y},\ t\in(0,1). (2.1) In the training procedure, we sample x_{t}=(1-t)x_{0}+tx_{1}^{y} with t\in(0,1) . The neural network v_{\theta}(\cdot,\cdot,y) is then trained by minimizing the following loss ( 19 ) : \mathcal{L}^{y}(\theta)=\mathbb{E}_{t,x_{0},x_{1}^{y}}\left[\|v_{\theta}(t,x_{t},y)-(x_{1}^{y}-x_{0})\|^{2}\right]. (2.2) Once trained, new samples are generated by solving the ODE ( 2.1 ) with numerical solvers ( 20 ) . Similarly, for unconditional flow matching (UFM), we aim to learn a velocity field v_{\theta}(\cdot,\cdot,\emptyset):[0,1]\times\mathbb{R}^{d}\to\mathbb{R}^{d} with the unconditional data distribution p_{1}^{\emptyset}:=p_{1}(\cdot|\emptyset) by an ODE: \tfrac{d}{dt}x_{t}=v_{\theta}(t,x,\emptyset),\ x_{0}\sim p_{0},\ x_{1}\sim p_{1}^{\emptyset},\ t\in(0,1). And the training loss is similarly defined as: \mathcal{L}^{\emptyset}(\theta)=\mathbb{E}_{t,x_{0},x_{1}^{\emptyset}}\left[\|v_{\theta}(t,x_{t},\emptyset)-(x_{1}^{\emptyset}-x_{0})\|^{2}\right]. (2.3) Classifier free guidance for flow matching. To manage the CFM and UFM