Gradual Fine-Tuning for Flow Matching Models
Gudrun Thorkelsdottir, Arindam Banerjee · 2026
Read Gradual Fine-Tuning for Flow Matching Models by Gudrun Thorkelsdottir, Arindam Banerjee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22495.txt.
Introduction
Gudrun Thorkelsdottir and Arindam Banerjee tune flow matching models in slow, gentle steps. Flow matching models learn to turn noise into data along a smooth path. Their 2026 preprint notes that free, hard tuning on small or shifting data can erase what pretraining won.
Gradual tuning moves the model in small, staged shifts instead. The aim is to keep past gains while still fitting the new task. That balance matters where data are scarce or drift over time. It also saves compute. The work offers a careful path for a fast-growing model class.
Catalog record
| CREATOR | Gudrun Thorkelsdottir, Arindam Banerjee |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22495
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gradual Fine-Tuning for Flow Matching Models Gudrun Thorkelsdottir, Arindam Banerjee Abstract Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or strict efficiency demands, where unconstrained fine-tuning can erode the accuracy and efficiency gains learned during pretraining. Prior work has produced theoretical guarantees and empirical advances for reward-based fine-tuning formulations, but these methods often impose restrictions on permissible drift structure or training techniques. In this work, we propose Gradual Fine-Tuning (GFT), a principled framework for fine-tuning flow-based generative models when samples from the target distribution are available. For stochastic flows, GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target drifts, approaching the true target as the temperature approaches zero. We prove convergence results for both marginal and conditional GFT objectives, enabling the use of suitable (e.g., optimal transport) couplings during GFT while preserving correctness. Empirically, GFT improves convergence stability and shortens probability paths, resulting in faster inference, while maintaining generation quality comparable to standard fine-tuning. Our results position GFT as a theoretically grounded and practically effective alternative for scalable adaptation of flow matching models under distribution shift. 1 Introduction Recent advances in flow-based generative modeling ( 25 ; 26 ) have achieved remarkably high generation quality, and have seen widespread use in natural image generation ( 11 ) , as well as scientific applications such as molecule generation ( 19 ; 22 ; 30 ) , biological sequence modeling ( 31 ; 9 ) , and atmospheric forecasting ( 14 ; 3 ) . In settings characterized by limited computational resources, data-scarcity, or evolving data distributions, fine-tuning a pretrained flow matching model is often the most effective approach for achieving accurate generation. Unlike training from random initialization, fine-tuning must reconcile the competing objectives of adapting to a new target distribution while preserving the structural properties (e.g. short probability paths) learned during pretraining. Such considerations are important for flow matching models, where path lengths directly influence generation efficiency, and small distributional changes can substantially alter the path lengths. As a result, effective fine-tuning requires principled mechanisms that enable controlled adaptation without sacrificing stability, accuracy, or inference efficiency. Existing work on fine-tuning flow matching models is dominated by reward-based methods. These rely on an external function that is constructed or trained to reward desirable behavior, such as human preference ( 4 ; 8 ) or physical constraints ( 33 ) . Notably, Adjoint Matching ( 10 ) casts fine-tuning as a Stochastic Optimal Control problem, and optimizes a control vector field which linearly adapts the pretrained model. Other reward-based methods ( 13 ; 12 ) extend fine-tuning to non-differentiable rewards, and demonstrate that constraining the deviation from the pretrained model can preserve generation diversity. Despite this progress, several key limitations remain. First, fine-tuning in the setting with access to target distribution samples, eliminating the reliance on an external reward function, is comparatively underexplored. Second, reward-based fine-tuning methods have been shown to require a memoryless noise schedule to provably converge to the correct target distribution ( 10 ) , meaning that training cannot leverage optimal transport (OT) couplings between source and target samples. This restriction has significant practical implications, as OT-based training can substantially improve generation efficiency ( 34 ; 28 ) . Further, reward-based optimal control methods for fine-tuning optimize an auxiliary vector field that is added linearly to the pretrained drift ( 10 ) . These approaches are therefore incompatible with many popular nonlinear fine-tuning techniques, including full fine-tuning and low-rank adaptation (LoRA) ( 18 ) . Finally, standard fine-tuning, defined as continued training after changing the target data distribution, uses the pretrained model as a warm-start, but otherwise ignores its learned dynamics. The main contributions of our work are as follows: (i) We propose a principled framework for fine-tuning flow-based generative models, called Gradual Fine-Tuning (GFT). GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target dynamics, approaching the true target as the temperature approaches zero. (ii) We generalize GFT to Conditional Flow Matching (CFM), establishing equivalent results for GFT when conditioning on arbitrary source-target couplings. Crucially, this allows for the use of training techniques that enhance inference efficiency. (iii) GFT is explicitly developed to accommodate nonlinear fine-tuning, making it compatible with widely used parameter fine-tuning techniques such as full fine-tuning and LoRA. We empirically demonstrate the strong performance of GFT on several natural distribution shift datasets. Across all settings, GFT achieves more stable convergence and consistently shorter probability paths without sacrificing generation quality relative to standard fine-tuning. This work presents the first systematic investigation of annealed gradual fine-tuning for flow matching models using a temperature parameter, and establishes a foundation for scalable and efficient adaptation of flow-based generative models. 2 Related Work Fine-Tuning for Flow Matching Flow Matching provides an efficient framework for training Continuous Normalizing Flows ( 6 ) by learning vector fields that transport probability mass between distributions ( 25 ; 26 ) . Although substantial effort has focused on improving base training objectives and inference efficiency, comparatively little work has addressed the problem of fine-tuning pretrained flow matching models under distribution shifts. Existing work is dominated by reward-based approaches of fine-tuning, which adapt the pretrained model to a tilted target distribution implicitly defined by a reward function. Adjoint Matching ( 10 ) formulates fine-tuning as a stochastic optimal control problem and introduces a control vector field that linearly modifies the pretrained dynamics. Related methods extend the reward fine-tuning paradigm to non-differentiable rewards and online settings ( 13 ; 12 ) , or perform fine-tuning through reinforcement learning methods by injecting noise into the deterministic flow ( 36 ) . While these methods are well suited to preference objectives, they fundamentally differ from our focus on fine-tuning flow matching models with access to target distribution samples. Regularization and Inference Efficiency Previous reward-based fine-tuning methods have demonstrated the use of divergence regularization to limit deviation from the pretrained model, including KL regularization ( 10 ) , or Wasserstein distance ( 13 ) . However, existing convergence guarantees require the use of a memoryless noise schedule, preventing derivative methods from conditioning on any jointly drawn source-target samples during training, including OT couplings ( 16 ; 27 ) . This restriction has critical practical consequences, as OT-based training is known to significantly enhance inference efficiency by encouraging short probability path lengths ( 34 ; 28 ) . Prior fine-tuning formulations also assume a linear relationship between the pretrained and fine-tuned vector fields ( 10 ) , rendering them incompatible with widely used nonlinear fine-tuning techniques, including full fine-tuning and low-rank adaptation (LoRA) ( 18 ) . In contrast, our approach directly admits nonlinear model updates, and establishes theoretical guarantees that remain valid under arbitrary source–target couplings. 3 Background and Preliminaries 3.1 Flow Matching Flow Matching ( 25 ; 26 ) is a training algorithm for dynamical generative models, a class of models which iteratively simulate a continuous process between a source distribution p_{0} and a data-defined target distribution p_{1} . Flow Matching defines a generative model by learning a time-varying vector field v_{\theta}(X_{t},t):\mathbb{R}^{d}\times\mathbb{R}\rightarrow\mathbb{R}^{d} which transports a family of probability densities \{p_{t}\}_{t\in[0,1]} along a deterministic path. dX_{t}=v_{\theta}(X_{t},t)dt,\qquad X_{0}\sim p_{0} (1) The trained