Training Flow Matching: The Role of Weighting and Parameterization

Anne Gagneux, Ségolène Martin, Rémi Gribonval, Mathurin Massias · 2026

Read Training Flow Matching: The Role of Weighting and Parameterization by Anne Gagneux, Ségolène Martin, Rémi Gribonval, Mathurin Massias (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06454.txt.

Introduction

Gagneux and his team study training objectives for denoising-based generative models, on loss weighting and output setup across noise, clean-image, and velocity forms. A planned numerical study asks how those choices interact with data-manifold size, architecture, and dataset size. Tests span synthetic datasets with set geometry plus image data, judging them by cleaning accuracy and output quality. Rather than a new method, the work sorts out factors that matter in flow matching training, yielding useful guidance on design choices.

Catalog record

CREATORAnne Gagneux, Ségolène Martin, Rémi Gribonval, Mathurin Massias
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40285
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06454

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Training Flow Matching: The Role of Weighting and Parameterization Anne Gagneux, Ségolène Martin, Rémi Gribonval, Mathurin Massias Abstract We study the training objectives of denoising-based generative models, with a particular focus on loss weighting and output parameterization, including noise-, clean image-, and velocity-based formulations. Through a systematic numerical study, we analyze how these training choices interact with the intrinsic dimensionality of the data manifold, model architecture, and dataset size. Our experiments span synthetic datasets with controlled geometry as well as image data, and compare training objectives using quantitative metrics for denoising accuracy (PSNR across noise levels) and generative quality (FID). Rather than proposing a new method, our goal is to disentangle the various factors that matter when training a flow matching model, in order to provide practical insights on design choices. 1 Introduction Flow matching (FM; 16 ; 2 ; 17 ) and diffusion models ( 23 ; 9 ; 24 ) are the state-of-the-art generative methods. Despite widespread adoption, many fundamental questions remain open, most notably regarding our understanding of why these models perform so well in practice. In this work, we seek to shed new light on some critical design choices: Which loss weightings and parameterizations should be preferred during training, and how do these choices depend on the architecture and data regime? To answer this, we unify different target predictions under a common weighted denoising formulation . This viewpoint allows us to directly relate denoising performance, measured at different noise levels, to generative performance. We make the following contributions: 1. For weights ( Section 4 ): We provide statistical theoretical insights into why the weights corresponding to flow-matching and signal-to-noise ratio perform robustly across a wide range of settings. To the best of our knowledge, this offers the first principled explanation for this empirical observation. 2. For parametrizations ( Section 5 ): Recent works show superior performance of denoiser parameterization (parametrization predicting the clean image), and motivate it by a manifold assumption. We find that while this behavior indeed holds in certain settings, velocity parameterization still remains more effective across many scenarios. In particular we show that, more than the manifold assumption on its own, the locality induced by the network architecture as well as the data regime have a critical influence when deciding between denoiser or velocity parametrizations. 2 Background and related works Flow matching and diffusion are equivalent in the case of Gaussian source distribution ( 6 ) : we use the flow matching conventions, but the results are easily translated for diffusion. Background on flow matching The generative process is defined over time t\in[0,1] , with an initial sample x_{0}\sim p_{0} and a target sample x_{1}\sim p_{1} . To connect with the concept of denoisers in the sequel, we further assume that the latent distribution is standard Gaussian: p_{0}=\mathcal{N}(0,\mathrm{I}_{d}) , and we work in the setting where the coupling p_{(x_{0},x_{1})} is the product coupling p_{0}\otimes p_{1} . In flow matching, generation of new samples is performed by solving on the differential equation \dot{x}(t)=v(x(t),t) from t=0 to 1 , using as initial condition x(0)=x_{0}\sim p_{0} . The function v:\mathbb{R}^{d}\times[0,1]\to\mathbb{R}^{d} is called the velocity , and the generated sample is simply the ODE solution at time t=1 , namely x(1) ; for an appropriate velocity, it should behave like a sample from p_{1} . In practice, the velocity is parametrized by a neural network v_{\theta} and learned by solving: \min_{\theta}\,\mathbb{E}_{t\sim\mathcal{U}[0,1],\,x_{0}\sim p_{0},\,x_{1}\sim p_{1}}\left[\|v_{\theta}(x_{t},t)-(x_{1}-x_{0})\|^{2}\right], (1) where x_{t}:=(1-t)x_{0}+tx_{1} is the linear interpolation between x_{0} and x_{1} . It is well-known that the solution v^{\star} to this problem (over all measurable functions) is given by the conditional expectation, v^{\star}(x,t)=\mathbb{E}[x_{1}-x_{0}\mid x_{t}=x,t] . Two important variations can be made in ( 1 ): first, it is possible to use a time-dependent weight in the loss to prioritize some noise levels, thus improving performance in some cases. Second, a quick computation shows that v^{\star}(x,t) also equals \frac{\mathbb{E}[x_{1}|x_{t}=x]-x}{1-t} and \frac{x-\mathbb{E}[x_{0}|x_{t}=x]}{t} . Therefore, having a network v_{\theta} learn v^{\star} (so-called v -prediction) is theoretically equivalent to having a network x_{\theta} learn \mathbb{E}[x_{1}|x_{t}=x] (denoising, aka x -prediction) or a network \varepsilon_{\theta} learn \mathbb{E}[x_{0}|x_{t}=x] (noise prediction, aka \varepsilon -prediction). We review these approaches below. Loss weightings Ever since the first successes of diffusion, a large body of work has studied the choice of training objectives and loss weightings across time ( 7 ; 12 ) . We briefly review this literature with the notations of the diffusion framework, which we adopt in this paragraph for simplicity . We consider the variance-preserving diffusion process, where noisy samples are generated as x_{t}=\alpha_{t}x_{0}+\sigma_{t}\varepsilon , with \varepsilon\sim\mathcal{N}(0,I_{d}) , \alpha_{t}^{2}+\sigma_{t}^{2}=1 , and time evolves from t=T (pure noise) to t=0 (clean data). Regarding loss weighting, the original diffusion formulation of 9 employs an unweighted \varepsilon -prediction loss, \mathbb{E}\big[\|\varepsilon-\varepsilon_{\theta}(x_{t},t)\|^{2}\big] . Viewing the problem as denoising, with the change of variable D_{\theta}(x,t)={({\rm Id}-\sigma_{t}\varepsilon^{\theta}(x,t))}/{\alpha_{t}} , this is equivalent to minimizing a denoising loss \|x_{0}-D_{\theta}(x_{t},t)\|^{2} but with an additional loss weighting equal to the SNR, \alpha_{t}^{2}/\sigma_{t}^{2} , prioritizing denoising of almost clean images. Several works have proposed alternative weightings to emphasize specific noise regimes. For instance, in x -prediction, 20 suggest the weighting \max(\alpha_{t}^{2}/\sigma_{t}^{2},1) , while 27 use \alpha_{t}/\sigma_{t} . Compared to the aforementioned SNR weighting, both assign increased importance to large noise levels, which are argued to be more difficult and critical for error propagation during sampling. Interpreting diffusion training as a multitask optimization problem, 7 observe gradient conflicts between noise levels and, with a similar objective, propose clipping the weighting as \min(\alpha_{t}^{2}/\sigma_{t}^{2},\gamma) to avoid over-emphasizing low-noise, easy denoising tasks. In contrast, 3 introduce the P2 weighting, which emphasizes intermediate noise levels under the hypothesis that perceptually relevant features emerge during this “content” phase. We emphasize that so far, these design choices rely on empirical observation and heuristics, and a consensus has not been reached. Unifying perspectives Several works have proposed unifying views on loss weightings in diffusion and flow-matching models. In particular, 12 reinterpret a wide range of objectives and weightings, including flow matching, as differently weighted ELBO formulations, while 14 ; 6 systematically relate common parameterizations (noise, score, velocity) to the loss weightings they induce. While these works provide valuable theoretical unification, they do not investigate why and when different weighting choices lead to substantially different empirical performance. In Section 4 , we address this gap through a denoising-based analysis that directly connects loss weighting to both denoising and generation performance. Data-prediction versus velocity-prediction In contrast with flow matching, which traditionally uses v -prediction, most diffusion implementations followed 9 and train networks to predict the noise \varepsilon . Nevertheless, v -prediction or x -prediction have also been explored ( 20 ; 7 ) . In particular, a recent work by 15 advocates that, real data lying on a low dimensional manifold ( 4 ) , x -prediction should be preferred, as the network’s task is thus made simpler. Subsequent work by 11 provably show on toy linear models the benefits of x -prediction when the data lives in a low dimensional subset of a large space. In Section 5 , we revisit this take and show the importance of other factors beyond the manifold assumption. 3 Generation as denoising Any choice of target between the clean image, the noise and the velocity naturally

Read the complete text