Multi-step Consistency Models: Fast Generation with Theoretical Guarantees

Nishant Jain, Xunpeng Huang, Yian Ma, Tong Zhang · 2025

Read Multi-step Consistency Models: Fast Generation with Theoretical Guarantees by Nishant Jain, Xunpeng Huang, Yian Ma, Tong Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01049.txt.

Introduction

Nishant Jain, Xunpeng Huang, Yian Ma, and Tong Zhang wrote this paper in two thousand twenty-five. It puts multi-step consistency models on firm theory ground, a class of fast generators that rival diffusion models.

Consistency models map a noisy input to any point on the reverse path, so samples come in a few steps. The authors prove a tight bound on the gap to the true spread, reached in a step count that grows only with the log of size over error. A like bound holds when the data are rough, with a step count that grows with dimension. They also show the models can be learned well with small steps, in both smooth and rough cases. The rough-case rates lead the field.

Catalog record

CREATORNishant Jain, Xunpeng Huang, Yian Ma, Tong Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01049

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-step Consistency Models: Fast Generation with Theoretical Guarantees Nishant Jain, Xunpeng Huang, Yian Ma, Tong Zhang Abstract Consistency models have recently emerged as a compelling alternative to traditional SDE-based diffusion models. They offer a significant acceleration in generation by producing high-quality samples in very few steps. Despite their empirical success, a proper theoretic justification for their speed-up is still lacking. In this work, we address the gap by providing a theoretical analysis of consistency models capable of mapping inputs at a given time to arbitrary points along the reverse trajectory. We show that one can achieve a KL divergence of order O ⁡ ( ε 2 ) O(\varepsilon^{2}) using only O ⁡ ( log ⁡ ( d / ε ) ) O\left(\log\left(d/\varepsilon\right)\right) iterations with a constant step size. Additionally, under minimal assumptions on the data distribution (non-smooth case)—an increasingly common setting in recent diffusion model analyses—we show that a similar KL convergence guarantee can be obtained, with the number of steps scaling as O ⁡ ( d ​ log ⁡ ( d / ε ) ) O\left(d\log\left(d/\varepsilon\right)\right) . Going further, we also provide a theoretical analysis for estimation of such consistency models, concluding that accurate learning is feasible using small discretization steps, both in smooth and non-smooth settings. Notably, our results for the non-smooth case yield best-in-class convergence rates compared to existing SDE/ODE-based analyses under minimal assumptions. 1 Introduction Lately, diffusion models Sohl-Dickstein et al., 2015 have become an important topic of research in computer vision and generative modelling Song and Ermon, 2019 ; Croitoru et al., 2023 ; Lugmayr et al., 2022 ; Song et al., 2020a ; Nichol et al., 2021 ; Song et al., 2021 ; Ho et al., 2020 , with applications ranging from generating images or videos Epstein et al., 2023 ; Chen et al., 2023d controllably to other areas like drug/protein design Gruver et al., 2023 ; Guo et al., 2024 . These models comprise a forward process that gradually adds noise to the data, and then the generation is done by the corresponding denoising process, which is sometimes referred to as the reverse/generation process. These forward/reverse processes can either be seen as transition kernels Huang et al., 2024 ; Song et al., 2020a ; Ho et al., 2020 or instead modeled as stochastic differential equations (SDEs) Song et al., 2020c ; Song et al., 2020b ; Song et al., 2021 ; Chen et al., 2023c . This is popularly referred as score based generative modeling due to its reliance on the neural-network parameterized score function , which at a given time instant is the gradient of log probability of the marginal distribution corresponding that time. Following from the particle-density transition property, Song et al., 2020c firstly argued that score-based diffusion method can be implemented by an ordinary differential equation (ODE) version. That means, under the unbiased score estimation, the ODE-based generation will share the same marginal distribution of the particles as those in the SDE-based generations. From a practical perspective, ODE-based methods can often bring the underlying distribution of particles closer to the data distribution faster than SDE-based methods while maintaining comparable generation quality Lu et al., 2022 ; Zhou et al., 2024 ; Zhang and Chen, 2022 . Moreover, researchers appreciate the deterministic update property in ODE-based methods since all the randomness is left in the particle initialization, which inspires the proposal of consistency models Song et al., 2023 . Motivated by distilling the deterministic mapping from ODE-based diffusion models, the original consistency model paper Song et al., 2023 takes any point along the probability flow ODE to the start i.e. the true data distribution with single-step generation function. Following the attention growth, a recent work Kim et al., 2023 attempted to improve the consistency model by extending the consistency function to any timestamp pair along the reverse ODE trajectory and consequently proposed a multi-step training scheme to achieve this, showing empirical effectiveness. From a theoretical perspective, Dou et al., 2024 ; Li et al., 2024a investigates the sample efficiency or the number of iterations to train such consistency models. However, the essential advantage of the consistency model in the inference process remains unknown. Although Lyu et al., 2024 provides some initial exploration for the inference efficiency of the consistency model, for achieving the convergence, an \tilde{O}(\varepsilon^{2}) step size is required that shares the same order as that in typical SDE or ODE-based inference algorithm. That result can neither show the advantages of introducing the deterministic update distillation nor match the real practice experience. Therefore, a natural question is raised: Can the consistency model achieve convergence with a larger step size, matching the real practice experience, and what kind of convergence will it achieve? In this work, we argue that the inference of consistency models via an adapted version of the multi-step updates allows a constant-level step size, which leads to a linear KL convergence toward the original data distribution under minimal smooth assumptions. Specifically, with this setup, we show that at the inference time, one can achieve O(\varepsilon^{2}) error with a constant step size. We provide this analysis for two scenarios: a) having popular assumptions used in the diffusion model analysis Chen et al., 2023c ; Song et al., 2020c ; Xu and Chi, 2024 ; Lyu et al., 2024 , which includes Lipschitzness of the score function, small score estimation error, finite second moment, and b) without assuming that the score function is Lipschitz Chen et al., 2023a ; Benton et al., 2024 a scenario that is recently considered and deemed closer to the to real-world applications. We are able to achieve this convergence by using the multi-step generation where after every application of the consistency model, there is a noising step during inference. Intuitively, this noise is effective in cancelling the accumulative score approximation error. Along, with this, another major ingredient is the modified formulation of the original consistency model that can map a sample from a given time instant to any arbitrary instant along the reverse ODE. We thereby analyse a modified multi-step sampling (version adapted to this formulation) in the KL divergence. We summarize the major contributions of this work as follows: • We provide an inference time analysis to achieve the O(\varepsilon^{2}) KL divergence in O\left(\log(\frac{d}{\varepsilon})\right) and a constant step size, utilizing the consistency function corresponding to the reverse probability flow ODE. • We further relax the smoothness assumption and provide the first analysis for consistency models under this scenario to adapt them more general data distributions, showing that the number of steps scales linearly in dimension as O\left(d\log(\frac{d}{\varepsilon})\right) . • We finally provide a theoretical analysis for estimating such consistency functions (under both smooth and non-smooth scenarios) and conclude that under fine-grained discretization at the training time, they can be estimated with very high accuracy. 1.1 Related Work SDE-Based Analysis of Diffusion Models. The foundational work establishing the effectiveness of diffusion models for generative tasks is the Denoising Diffusion Probabilistic Models (DDPM) framework introduced by Ho et al., 2020 . Building on this, Song et al., 2020c demonstrated that the forward noising process in DDPMs can be interpreted as a stochastic differential equation (SDE), laying the groundwork for continuous-time formulations of diffusion models. Subsequent works

Read the complete text