Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities
Pramook Khungurn, Pratch Piyawongwisal, Sira Sriswasdi, Supasorn Suwajanakorn · 2025
Read Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities by Pramook Khungurn, Pratch Piyawongwisal, Sira Sriswasdi, Supasorn Suwajanakorn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01169.txt.
Introduction
Pramook Khungurn, Pratch Piyawongwisal, Sira Sriswasdi, and Supasorn Suwajanakorn improve how big flow models are shrunk into fast ones. Flow models learn a moving field that turns noise into data over time. A distilled version can jump between any two times in one step. Training that jumper well is the hard part.
Their new loss matches speeds at both ends of the jump. It adds terms for the starting speed that earlier losses left out. It also cleans the target for the ending speed, dropping a derivative that added noise. A slowly averaged copy of the model supplies steady targets. Early tests across data sets and designs show better few-step generation than prior losses. The work aims at image tools that answer in a handful of steps.
Catalog record
| CREATOR | Pramook Khungurn, Pratch Piyawongwisal, Sira Sriswasdi, Supasorn Suwajanakorn |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01169
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities Pramook Khungurn, Pratch Piyawongwisal, Sira Sriswasdi, Supasorn Suwajanakorn Abstract A flow matching model learns a time-dependent vector field v t ( x ) v_{t}(x) that generates a probability path { p t } 0 ≤ t ≤ 1 \{p_{t}\}_{0\leq t\leq 1} that interpolates between a well-known noise distribution ( p 0 p_{0} ) and the data distribution ( p 1 p_{1} ). It can be distilled into a two-timed flow model (TTFM) ϕ s , x ( t ) \phi_{s,x}(t) that can transform a sample belonging to the distribution at an initial time s s to another belonging to the distribution at a terminal time t t in one function evaluation. We present a new loss function for TTFM distillation called the initial/terminal velocity matching (ITVM) loss that extends the Lagrangian Flow Map Distillation (LFMD) loss proposed by Boffi et al. ( 2024 ) by adding redundant terms to match the initial velocities at time s s , removing the derivative from the terminal velocity term at time t t , and using a version of the model under training, stabilized by exponential moving averaging (EMA), to compute the target terminal average velocity. Preliminary experiments show that our loss leads to better few-step generation performance on multiple types of datasets and model architectures over baselines. 1 Introduction Diffusion models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ) model a continuum of probability distributions \{p_{t}\}_{0\leq t\leq 1} such that p_{0} is a well-known noise distribution, and p_{1} is the data distribution p_{\mathrm{data}} or a slightly noisy version of it. 1 1 1 Lipman et al. call this a “probability path” ( 2023 ) . A data item can be sampled by first sampling a noise vector from p_{0} and then gradually transforming it into samples from p_{t_{1}} , p_{t_{2}} , \dotsc , p_{t_{K}} where 0<t_{1}<t_{2}<\dotsb<t_{K}=1 . Compared to the previous reigning champion, GANs ( Goodfellow et al., 2014 ) , diffusion models yield better sample quality ( Dhariwal & Nichol, 2021 ) and are more stable to train. Nevertheless, because the sampling process involves applying the model multiple times, diffusion models are significantly slower than other approaches, such as GANs, VAEs ( Kingma & Welling, 2014 ) , and normalizing flows ( Rezende & Mohamed, 2016 ) that can generate a sample with a single model evaluation. As a result, speeding up diffusion models has received much interest. One model acceleration approach is to distill ( Hinton et al., 2015 ) the knowledge of a pre-trained diffusion model into a new network that can generate samples in one or a few evaluations. We are interested in distilling a flow matching model ( Lipman et al., 2023 ) , a variant of diffusion models that predicts a time-dependent velocity field that continuously transforms p_{0} to p_{1} . Specifically, our objective is to construct a two-timed flow model (TTFM) 2 2 2 Different authors have different names for this type of function. Lipman et al. call it a “flow” ( 2023 ) , Boffi et al. a “flow map” ( Boffi et al., 2024 ) , and Kim et al. a “consistency trajectory model” ( 2023 ) . We think that our naming better captures its characteristics and distinguishes it from other similar functions such as a normalizing flow, which has no time input, and a consistency model ( Song et al., 2023 ) , which has one time input. \phi_{s,t}(x) , which is a deterministic function such that \phi_{s,t}(x)\sim p_{t} given that x\sim p_{s} for any 0\leq s<t\leq 1 . TTFM allows a data item to be generated in a single evaluation, by computing \phi_{0,1}(x_{0}) where x_{0}\sim p_{0} , or in multiple steps, which may increase the sample quality. Furthermore, TTFM also supports sample manipulation, such as with SDEdit ( Meng et al., 2022 ) . While several approaches train the student models with no time input ( Luhman & Luhman, 2021 ; Zheng et al., 2023 ; Yin et al., 2024b ; Yin et al., 2024a ; Xie et al., 2024 ; Kang et al., 2024 ; Nguyen & Tran, 2024 ) or one time input ( Salimans & Ho, 2022 ; Sauer et al., 2023 ; Song et al., 2023 ; Berthelot et al., 2023 ; Tee et al., 2024 ) , only a few have considered two time inputs ( Kim et al., 2023 ; Boffi et al., 2024 ) . In this paper, we explore TTFM distillation and propose a new loss function for it, called the initial/terminal velocity matching (ITVM) loss. It is based on the recent Lagrangian flow matching distillation (LFMD) loss ( Boffi et al., 2024 ) that forces the trained network to obey a partial differential equation (PDE) that governs a flow matching model’s trajectory. We make a key observation that, rather than enforcing the PDE directly, we can enforce special cases of the PDE along with a property called “consistency” ( Song et al., 2023 ) to achieve the same effect. This formulation has an advantage that, during training, the input to the teacher model is always in its training distribution, resulting in better supervising signals. More concretely, we modify the LFMD loss by first adding mathematically redundant terms that enforce the special cases. We then convert the original formula by replacing the partial derivative with a finite difference, which simplifies the implementation and speeds up computation. We then change the matching target of the finite difference from the teacher model to the exponentially moving averaged version of the model under training. This last step encourages the student to be consistent with itself rather than reproduce the teacher’s out-of-distribution outputs. We extensively evaluated ITVM under multiple datasets and model architectures and found that it yields better few-step generation performance on many datasets than those of the LFMD loss and other baselines. 2 Previous Works Diffusion models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ) have demonstrated remarkable effectiveness in generative modeling across diverse domains such as image, audio, and video. Their approach learns the score of a noise-adding process governed by a stochastic differential equation (SDE), which has an equivalent ordinary differential equation (ODE), and reverses the process by simulating one of the differential equations backward in time ( Song et al., 2021 ) . Recent advancement like rectified flow ( Liu et al., 2023 ) , flow matching ( Lipman et al., 2023 ) , and stochastic interpolants ( Albergo & Vanden-Eijnden, 2023 ) extend this framework by introducing different mathematical formulations and more flexibility in the modeling of the noise-adding process. However, the iterative sampling process of diffusion model is computationally intensive. To address this limitation, two main strategies have emerged: (1) reducing the simulation cost of ODEs or SDEs ( Lu et al., 2022 ; Shaul et al., 2023 ) , and (2) distillation, which trains a new model (the student) that compresses the sampling process of a diffusion model (the teacher) into fewer steps. Distillation. A straightforward distillation approach is to match the student’s outputs to those of the teacher. Luhman and Luhman proposed a simple implementation ( 2021 )