Stochastic Transition-Map Distillation for Fast Probabilistic Inference
George Rapakoulias, Peter Garud, Lingjiong Zhu, Panagiotis Tsiotras · 2026
Read Stochastic Transition-Map Distillation for Fast Probabilistic Inference by George Rapakoulias, Peter Garud, Lingjiong Zhu, Panagiotis Tsiotras (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07661.txt.
Introduction
George Rapakoulias and three co-authors speed up diffusion model sampling while keeping it genuinely random. Most fast samplers collapse the process into something more deterministic, which hurts downstream tasks like posterior sampling and inverse problems that need true stochastic draws. Their method instead distills the full transition map of the sampling equation.
Called Stochastic Transition-Map Distillation, the approach parameterizes those transitions with a conditional mean-flow model, giving a one-step or few-step stochastic sampler. It needs no pretrained teacher, no two-level optimization, and no stored trajectories, which keeps training efficient. The authors prove convergence bounds in the Wasserstein distance and validate the sampler on MNIST, CIFAR-10, and CelebA image generation.
Catalog record
| CREATOR | George Rapakoulias, Peter Garud, Lingjiong Zhu, Panagiotis Tsiotras |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07661
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stochastic Transition-Map Distillation for Fast Probabilistic Inference George Rapakoulias, Peter Garud, Lingjiong Zhu, Panagiotis Tsiotras Abstract Diffusion models achieve strong generation quality, diversity, and distribution coverage, but their performance often comes with expensive inference. In this work, we propose Stochastic Transition-Map Distillation (STMD), a teacher-free framework for accelerating diffusion model inference while preserving probabilistic sample generation. In contrast to score-based diffusion models, whose denoising parametrization models the mean of the posterior distribution, STMD distills the full transition map associated with the sampling stochastic differential equation (SDE). We parameterize these SDE transitions with a conditional Mean Flow model, yielding a one- or few-step stochastic sampler that retains the transition structure of the underlying diffusion process. This perspective is especially useful for downstream tasks that require stochastic inference, such as diffusion posterior sampling, inverse problems, and energy-based fine-tuning. Compared to recent distillation methods, STMD requires no pretrained teacher, bi-level optimization, or trajectory simulation and caching, enabling efficient and scalable training. We derive convergence bounds for our method in the Wasserstein distance, providing a strong theoretical foundation for our approach, and validate STMD on various image generation examples on the MNIST, CIFAR-10, and CelebA datasets. † † footnotetext: Corresponding authors: [email protected], [email protected], [email protected] 1 Introduction and Motivation Continuous-time diffusion models excel in generation quality and sample diversity compared to their predecessors, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) ( 20 ; 21 ; 3 ; 29 ; 30 ) ; however, this performance increase comes at the expense of inference complexity, stemming from the need to integrate the underlying continuous-time ordinary differential or stochastic differential equations. Although many techniques have been proposed to accelerate inference, such as distillation ( 38 ; 48 ) and Optimal Transport (OT) based approaches ( 35 ; 47 ) , to name a few, achieving the optimal tradeoff between generation quality and sample diversity versus inference complexity is still unresolved ( 13 ) . Flow Matching (FM) and its variants ( 35 ; 34 ; 1 ) have emerged as a powerful tool for training continuous-time flow models. They work by approximating the drift term (also referred to as the policy) of an Ordinary Differential Equation (ODE) or Stochastic Differential Equation (SDE) as a mixture of elementary point-to-point conditional drift terms. Implementing Flow Matching in practice, however, requires a temporal discretization of the learned continuous-time ODE/SDE with a very small time step size, slowing down inference ( 47 ; 34 ) . Distillation techniques offer a set of tools to accelerate inference by learning a flow map of the corresponding ODE directly ( 48 ; 5 ; 6 ; 17 ; 18 ; 44 ) . At large, these methods leverage different techniques to “learn to integrate” an underlying continuous-time ODE. Although they achieve high-quality results, one disadvantage is that they are usually used only with deterministic inference, that is, given an initial noise sample, the generated sample is unique. Many downstream tasks, however, benefit from stochastic inference to boost generation quality, improve model alignment with user queries ( 27 ) , or perform diffusion posterior inference based on noisy or partial sample observations ( 10 ) . In this direction, recent works have explored training one- or few-step generators to match the distribution of a diffusion model ( 32 , Section 10.2) ( 56 ; 23 ; 42 ) . In this work, we take a different approach and propose a method for learning the transition probabilities of a given SDE, thereby generalizing ODE-based distillation approaches to the stochastic regime. We achieve this by using a conditional Mean Flow, which learns a single or few-step generator for the transition probability of a diffusion model. The contributions of this paper are as follows: • We propose STMD, a distillation framework based on conditional Mean Flows, enabling us to learn the transition maps of high-dimensional diffusion SDEs using a single-step or a few-step generator through a regression objective, and without using any pretrained model. • We provide a theoretical convergence analysis in the 2-Wasserstein distance for the Mean Flow algorithm, and then extend it to study the convergence of our learned transition map to that of the underlying diffusion process. To the best of our knowledge, this is the first Mean Flow convergence analysis in the 2-Wasserstein distance. • We illustrate this method in various scenarios related to image generation and inpainting problems, showing strong empirical performance. 2 Preliminaries 2.1 Flow Matching with Straight-Line Interpolants We first recall the basic Flow Matching formulation ( 35 ; 34 ; 1 ) with straight-line interpolants. All distributions are supported in the d -dimensional Euclidean space, denoted \mathbb{R}^{d} . To align with the recently proposed Mean Flow model ( 17 ) , we will use z_{0}\sim\rho_{0} to denote the data distribution, and z_{1}\sim\rho_{1} to denote a sample from a simple prior distribution, e.g., a Gaussian distribution. We denote the densities of flow-related variables with \rho , and use the variable s to denote time. Unless stated otherwise, all differential equations run backward in time, that is, from s=1 to s=0 . For s\in[0,1] , define the straight-line interpolant z_{s}=(1-s)z_{0}+sz_{1}. (1) The corresponding conditional velocity is constant along the path and is given by v_{s}(z_{s}|z_{0},z_{1})=z_{1}-z_{0}. (2) A velocity field that transports \rho_{0} to \rho_{1} and vice versa can be obtained by averaging this conditional velocity over all pairs (z_{0},z_{1})\sim\rho_{0}\times\rho_{1} that could have produced the same intermediate point z_{s} : v_{s}(z)=\mathbb{E}\left[z_{1}-z_{0}\mid z_{s}=z\right]. (3) The Flow Matching objective trains a neural network v^{\theta}_{s}(z) to approximate ( 3 ) using a finite-sample approximation of the conditional flow matching objective ( 34 ; 35 ) : \mathcal{L}_{\mathrm{CFM}}=\mathbb{E}\left[\left\|v^{\theta}_{s}(z_{s})-(z_{1}-z_{0})\right\|^{2}\right]. (4) This conditional objective is equivalent, in terms of its optimizer, to regression against the marginal velocity v_{s}(z) defined in ( 3 ). Indeed, by expanding ( 4 ) and using the tower property of conditional expectation, \mathbb{E}\left[\left\|v^{\theta}_{s}(z_{s})-(z_{1}-z_{0})\right\|^{2}\right]=\mathbb{E}\left[\left\|v^{\theta}_{s}(z_{s})-v_{s}(z_{s})\right\|^{2}\right]+C, (5) where C is independent of \theta , motivating the use of CFM objective ( 4 ). At inference time, samples are generated by integrating the learned ODE backwards from s=1 to s=0 : \dot{z}_{s}=v^{\theta}_{s}(z_{s}). (6) 2.2 Consistency Models and Mean Flows (a) (b) Figure 1 : (a): Consistency training learns the flow map of an ODE without requiring sampling of simulated trajectories during training. Figure taken from 48 . (b): Mean Flows parametrize the flow map using the average velocity along a trajectory, allowing for taking exact, finite-time steps during inference. Figure taken from 17 . A limitation of flow models is that during inference, the learned flow ( 6 ) needs to be integrated. To accelerate inference, distillation approaches aim at learning the flow map of the inference ODE: \dot{z}_{s}=v_{s}(z_{s}). (7) Seminal works in this direction include 48 ; 28 , which are based on the probability flow ODE to model v_{s} in ( 7 ), and 5 ; 17 , that leverage a flow-matching ODE and use a drift of the form ( 3 ). A recent work is the Mean Flow (MF) model ( 17 ) , which learns the flow map of ( 6 ) without requiring a pretrained reference model v^{\theta}_{s}(z_{s}) . Consider an ODE ( 7 ) with drift as in ( 3 ). Let 0\leq r<s\leq 1 . Consider a trajectory