Understanding, Accelerating, and Improving MeanFlow Training

Jin-Young Kim Hyojun Go Lea Bogensperger Julius Erbach, Nikolai Kalischek Federico Tombari Konrad Schindler Dominik Narnhofer · 2025

Read Understanding, Accelerating, and Improving MeanFlow Training by Jin-Young Kim Hyojun Go Lea Bogensperger Julius Erbach, Nikolai Kalischek Federico Tombari Konrad Schindler Dominik Narnhofer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19065.txt.

Introduction

Jin-Young Kim and seven co-authors dissect and then speed the training of MeanFlow, a generative model that promises fine samples in few steps. MeanFlow learns two velocity fields at once: the instant field and the average field over a time span. How the two interact in training was unclear, which left schedules to guesswork.

Three findings guide their fix. A sound instant field must form before the average field can be learned. The instant field gains from the average field at small time gaps and loses at large ones. Smooth learning of large gap averages, the key to one step generation, rests on prior sound instant and small gap fields. Their scheme builds the instant field fast, then shifts weight from short to long spans.

Catalog record

CREATORJin-Young Kim Hyojun Go Lea Bogensperger Julius Erbach, Nikolai Kalischek Federico Tombari Konrad Schindler Dominik Narnhofer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-19065

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Understanding, Accelerating, and Improving MeanFlow Training Jin-Young Kim Hyojun Go Lea Bogensperger Julius Erbach, Nikolai Kalischek Federico Tombari Konrad Schindler Dominik Narnhofer Abstract MeanFlow promises high-quality generative modeling in few steps, by jointly learning instantaneous and average velocity fields. Yet, the underlying training dynamics remain unclear. We analyze the interaction between the two velocities and find: (i) well-established instantaneous velocity is a prerequisite for learning average velocity; (ii) learning of instantaneous velocity benefits from average velocity when the temporal gap is small, but degrades as the gap increases; and (iii) task-affinity analysis indicates that smooth learning of large-gap average velocities, essential for one-step generation, depends on the prior formation of accurate instantaneous and small-gap average velocities. Guided by these observations, we design an effective training scheme that accelerates the formation of instantaneous velocity, then shifts emphasis from short- to long-interval average velocity. Our enhanced MeanFlow training yields faster convergence and significantly better few-step generation: With the same DiT-XL backbone, our method reaches an impressive FID of 2.87 on 1-NFE ImageNet 256 × \times 256, compared to 3.43 for the conventional MeanFlow baseline. Alternatively, our method matches the performance of the MeanFlow baseline with 2.5 × \times shorter training time, or with a smaller DiT-L backbone. Our code is available at https://github.com/seahl0119/ImprovedMeanFlow . 1 Introduction Diffusion models 61 ; 65 ; 28 and Flow Matching 39 ; 2 ; 40 have achieved state-of-the-art results across image 75 ; 4 ; 48 , video 70 ; 17 ; 1 , and 3D generation 21 ; 22 . However, it remains a persistent weakness that the denoising relies on many, small iteration steps, making sampling computationally expensive 25 . Higher-order samplers 62 ; 43 ; 44 ; 13 ; 80 ; 32 ; 54 partially alleviate this, though achieving high fidelity with fewer than 10 steps remains a challenge. Consequently, recent work has focused on models that enable inference in a few steps, or even a single step. Figure 1 : Our novel, enhanced training strategy reaches the performance of MeanFlow-XL in \approx 2.5 \times fewer training epochs, and converges to a final model with superior performance ( \approx 16% lower FID). Early approaches distill few-step generative models from pretrained multi-step diffusion models, using direct 45 ; 81 ; 56 , adversarial 59 ; 76 ; 58 , or score-based supervision 87 ; 47 ; 77 . The two-stage design increases complexity, requires two distinct training processes, and often depends on large-scale synthetic data generation 45 ; 40 , or on propagation through teacher–student cascades 49 ; 56 . Consistency models 63 represent a step towards one-time, end-to-end training, by enforcing consistent outputs for all samples drawn along the same denoising trajectory. Despite various improvements 42 ; 71 ; 64 ; 19 ; 34 ; 72 ; 26 , a substantial performance gap remains between few-step consistency models and multi-step diffusion models. More recent research 34 ; 16 ; 6 ; 85 ; 7 ; 55 ; 74 has proposed to characterize diffusion/flow quantities along two distinct time indices. Among these attempts, MeanFlow 18 stands out as a stable end-to-end training scheme that markedly narrows the gap between one-step and multi-step generation. The key to MeanFlow’s success is the idea of exploiting the intrinsic relationship between the instantaneous velocity (at a single time point) and the average velocity (integrated over a time interval), such that a single network learns both simultaneously. However, MeanFlow training is computationally expensive, and has only been analyzed rather superficially. In particular, it remains poorly understood how the two coupled velocity fields interact during learning and how their interplay can be coordinated to achieve high-quality one-step generation. Here, we investigate these learning dynamics and develop a training strategy that greatly improves both generation quality and efficiency. Through controlled experiments, we determine that: (i) instantaneous velocity must be established early in the training process, because it provides the foundation for learning average velocity: if instantaneous velocities are poorly formed or corrupted, then learning average velocities fails altogether; (ii) in the opposite direction, the time interval over which average velocities are computed (the “temporal gap”) critically determines how they impact the learning of instantaneous velocities: small gaps facilitate instantaneous velocity formation and refinement, while large gaps destabilize it; (iii) task affinity analysis reveals that one should initially focus on small-gap average velocities, which lay the foundation for learning the large-gap average velocities that are required for one-step generation. Standard MeanFlow training ignores these subtle, but impactful dynamics. Throughout the training process, it applies the same, fixed loss function and sampling scheme, disregarding the complex dependencies between the two velocity fields. This naive training objective interferes with the early formation of reasonable instantaneous velocities, which in turn delays the learning of average velocities. Ultimately, the current training practice significantly degrades overall performance compared to what would be achievable with a given model and dataset, and also slows down the training. To remedy these issues, we propose a simple yet effective extension of MeanFlow training. To quickly establish reasonable instantaneous velocities, we adopt acceleration techniques from diffusion training 10 ; 25 ; 20 ; 73 ; 35 . To support the learning of correct average velocities, we design a progressive weighting scheme. In early training stages, the weighting prioritizes small gaps, which reinforces instantaneous velocity formation and prepares the ground for large-gap learning. As training progresses, the weighting gradually transitions to equal weighting across all gap lengths, ensuring accurate average velocities over large gaps, which are the vital ingredient for few-step inference. Empirically, the enhanced training protocol substantially improves the generation results and also accelerates convergence. On the standard 1-NFE ImageNet 11 256\times 256 benchmark, we improve the FID of MeanFlow-XL from FID 3.43 to 2.87, see Fig. 1 . To reach the performance of conventional training, our improved training scheme needs 2.5 \times fewer iterations. Remarkably, it is even capable of matching that same performance with a smaller DiT-L backbone. In a broader context, our work shows that there is still a lot of untapped potential to accelerate recent few-step generative models. With a better understanding of their internal dynamics and numerical properties, high-quality real-time generation may well be achievable. 2 Related Work Acceleration of diffusion and flow matching training. Diffusion models 61 ; 65 ; 28 gradually perturb data with noise and train a network to reconstruct the clean signal. This noise-injection and denoising process can be described via stochastic differential equations (SDEs) or, equivalently, as a probability-flow ordinary differential equation (ODE) 32 ; 66 . Flow Matching 39 ; 2 ; 40 extends this with velocity fields, enabling the model to learn transport paths connecting data and reference distributions. Training these models is computationally expensive 52 . To accelerate it, prior work mostly focuses on critical timesteps using two strategies: (1) timestep-dependent loss weighting based on SNR 25 , perceptual quality 10 , or uncertainty 20 ; 33 , and (2) modified sampling distributions 82 ; 83 ; 84 . Furthermore, there are hybrid approaches that combine the two 73 , as well as adaptive schedulers 35 . We will leverage some of the existing acceleration techniques to speed up the formation of instantaneous velocity, which is important to speed

Read the complete text