A Simple Analysis of Discretization Error in Diffusion Models

Juhyeok Choi, Chenglin Fan · 2025

Read A Simple Analysis of Discretization Error in Diffusion Models by Juhyeok Choi, Chenglin Fan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08337.txt.

Introduction

Juhyeok Choi and Chenglin Fan give a leaner proof for how image-breeding AI models lose skill in set steps. Such models run as math flows that add noise and then learn to shed it. Code must chop the flow into steps, and each chop adds error that past proofs bound with heavy work. Their frame, under the head title about other noise choices, trims that proof with a classic growth bound.

The step error rate they find matches past work at a clean half-power pace. A swap claim follows: cheap coin-flip noise can stand in for the standard bell noise with no loss of that rate. The one need is a matched spread for the swap noise. Lab tests back both claims, and wrong spread breaks the gain as the math warns.

Catalog record

CREATORJuhyeok Choi, Chenglin Fan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08337

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Simple Analysis of Discretization Error in Diffusion Models Juhyeok Choi, Chenglin Fan Abstract Diffusion models, formulated as discretizations of stochastic differential equations (SDEs), achieve state-of-the-art generative performance. However, existing analyses of their discretization error often rely on complex probabilistic tools. In this work, we present a simplified theoretical framework for analyzing the Euler–Maruyama discretization of variance-preserving SDEs (VP-SDEs) in Denoising Diffusion Probabilistic Models (DDPMs), where T T denotes the number of denoising steps in the diffusion process. Our approach leverages Grönwall’s inequality to derive a convergence rate of 𝒪 ⁡ ( 1 / T 1 / 2 ) \mathcal{O}(1/T^{1/2}) under Lipschitz assumptions, significantly streamlining prior proofs. Furthermore, we demonstrate that the Gaussian noise in the discretization can be replaced by a discrete random variable (e.g., Rademacher or uniform noise) without sacrificing convergence guarantees—an insight with practical implications for efficient sampling. Experiments validate our theory, showing that (1) the error scales as predicted, (2) discrete noise achieves comparable sample quality to Gaussian noise, and (3) incorrect noise scaling degrades performance. By unifying simplified analysis and discrete noise substitution, our work bridges theoretical rigor with practical efficiency in diffusion-based generative modeling. 1 Introduction Diffusion models are a class of generative models that have recently achieved state-of-the-art performance in tasks such as image synthesis, audio generation, and molecular design. Inspired by non-equilibrium thermodynamics, these models learn to reverse a gradual noising process, transforming pure noise into structured data through a series of denoising steps, where T denotes the number of denoising steps in the diffusion process.. The core idea behind diffusion models is to model the data distribution by simulating a Markovian forward process that incrementally adds Gaussian noise to the data, eventually destroying all structure. A neural network is then trained to approximate the reverse process — learning to recover clean data from noisy observations. Once trained, the model can generate new samples by starting from random noise and iteratively applying the learned denoising steps sohl2015deep ; ho2020denoising . Compared to other generative approaches such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), diffusion models are often more stable to train and produce higher-quality samples, albeit with slower generation speed due to their iterative nature. Recent advances, including Denoising Diffusion Implicit Models (DDIM) and consistency models, have addressed some of these limitations by improving sample efficiency and generation speed song2021denoising ; saharia2022photorealistic ; ho2022classifier . Diffusion models have quickly become a central tool in generative AI research, offering both strong theoretical grounding and impressive empirical results across a wide range of applications, including text-to-image generation ramesh2022hierarchical ; saharia2022photorealistic , audio synthesis kong2021diffwave , molecular and protein design trippe2022diffusion ; watson2023novo , video generation ho2022imagenvideo , and medical imaging wolleb2022diffusion . We present a simplified and rigorous analysis of the Euler–Maruyama discretization in Denoising Diffusion Probabilistic Models (DDPMs), with the following key contributions: (1) Simplified Theoretical Framework : We analyze the reverse-time variance-preserving SDE (VP-SDE) using a time-rescaled formulation and Grönwall’s inequality, yielding a clean and accessible proof structure. (2) Convergence Guarantee : We provide a simplified proof of the strong convergence rate \mathcal{O}(1/\sqrt{T}) under standard Lipschitz assumptions, avoiding the complex probabilistic arguments used in prior analyses. (3) Discrete Noise Replacement : We show that Gaussian noise in the discretization can be substituted with discrete alternatives (e.g., Rademacher or uniform) without affecting the convergence rate. (4) Empirical Support : Controlled experiments on synthetic and image datasets confirm that (i) the error scales as predicted, (ii) discrete noise performs comparably to Gaussian noise, and (iii) improper noise scaling leads to degradation. 2 Related Work The connection between diffusion models and SDEs has been formalized in prior work song2021score . Our work builds the convergence rate of the sampling of diffusion models by applying change of variables and analyzing the discretization error. Quantized and low-precision noise has also been explored xiao2022discrete , but without rigorous convergence guarantees. More recently, several studies have established polynomial convergence rates for diffusion models chen2023sampling ; lee2023convergence ; Li2023 ; chen2023improved . For general input distributions, Benton et al. benton2023 provided the first complexity bound that scales linearly with the dimension d , requiring \mathcal{O}(d/\varepsilon^{2}) iterations. In the special case where T/\log^{5}T=\Omega(d^{2}) , Li et al. li2024c refined this result, reducing the iteration complexity to \mathcal{O}(d/\varepsilon) . Under additional smoothness assumptions, the iteration complexity has been further improved to \min\{d,\,d^{2/3}L^{1/3},\,d^{1/3}L\}\cdot\varepsilon^{-2/3} jiao2025instance , where L denotes a relaxed Lipschitz constant of the score function. For an overview of recent empirical and theoretical progress, we refer readers to the surveys by Tang and Zhao yang2023diffusion ; tang2024tutorial . 3 Preliminary 3.1 Grönwall’s Inequality Grönwall’s inequality is one of the strong tools for establishing computation bounds, proven by Grönwall in 1919 Gronwall . It can be stated as below: Theorem 3.1 (Continuous Grönwall’s Inequality) . Let a\geq 0 , b(t)\geq 0 and u:[0,T]\rightarrow\mathbb{R}_{+} be a continuous function. If \displaystyle u(t)\leq a+\int_{0}^{t}b(s)u(s)ds, then \displaystyle u(T)\leq a\exp\left(\int_{0}^{T}b(t)dt\right). We can also obtain a discrete version corollary as below. Corollary 3.2 (Discrete Grönwall’s Inequality) . Let a\geq 0 and \{u_{n}\} , {b_{n}} be non-negative sequences. If \displaystyle u_{n}\leq a+\sum_{k=0}^{n-1}b_{k}u_{k}, then \displaystyle u_{n}\leq a\left(1+\sum_{k=0}^{n-1}b_{k}\prod_{j=k+1}^{n-1}(1+b_{j})\right). 3.2 Overview of Diffusion Models Diffusion models are generative models that transform a simple noise distribution into a complex data distribution by simulating a time-reversed stochastic process. The forward process (also called the diffusion process) gradually adds noise to data, while the reverse process (also called the generative or sampling process) aims to recover data from pure noise. Forward Process. Let X_{0}\sim p_{\text{data}} be a sample from the data distribution. The forward process \{X_{t}\}_{t\in[0,1]} evolves according to the SDE: \displaystyle dX_{t}=-\frac{1}{2}\beta(t)X_{t}\,dt+\sigma(t)\,dW_{t}, where W_{t} is a standard Wiener process, \beta(t):[0,1]\to\mathbb{R}_{+} is a non-decreasing noise schedule, and we define \sigma(t):=\sqrt{\beta(t)} following the Variance-Preserving (VP) SDE formulation in song2021score . This process transforms the data distribution into a tractable noise distribution (typically \mathcal{N}(0,I) ) as t\to 1 . Reverse Process. By results from stochastic calculus (e.g., Anderson’s time-reversal of diffusions), the reverse-time process \{\overline{X}_{t}\}_{t\in[0,1]} also follows an SDE: \displaystyle d\overline{X}_{t}=\left(-\frac{1}{2}\beta(t)\overline{X}_{t}-\beta(t)\nabla_{x}\log p_{t}(\overline{X}_{t})\right)dt+\sqrt{\beta(t)}\,d\overline{W}_{t}, where \overline{W}_{t} is a reverse-time Wiener process and p_{t} is the marginal distribution of X_{t} in the forward process. In practice, the score function \nabla_{x}\log p_{t}(x) is intractable, so it is approximated by a neural network trained to predict either the score or the noise added during the forward process. In particular, Denoising Diffusion Probabilistic Models (DDPM) ho2020denoising use a parameterized model \epsilon_{\theta}(x,t) to estimate the noise in the forward process. Plugging this into the reverse SDE leads to the following approximate generative process: \displaystyle d\overline{X}_{t}=\left(\frac{\beta(t)}{\sigma_{t}}\epsilon_{\theta}(\overline{X}_{t},t)-\frac{\beta(t)}{2}\overline{X}_{t}\right)dt+\sqrt{\beta(t)}\,d\overline{W}_{t}, which is the SDE we analyze in the following section. Our goal is to quantify the error when simulating this reverse-time process using the Euler–Maruyama scheme. 4 Time change and Euler-Maruyama discretization DDPM can be considered as a discretization of the reverse-time VP-SDE \displaystyle d\overline{X}_{t}=\left(\frac{\beta(t)}{\sigma_{t}}\epsilon_{\theta}\left(\overline{X}_{t},t\right)-\frac{\beta(t)}{2}\overline{X}_{t}\right)dt+\sqrt{\beta(t)}d\overline{W}_{t}. To analyze the error of the Euler-Maruyama discretization, one should bound the time interval. However, with the original time scheme, the time interval of one step is fixed to 1 . Hence we use the variable transformation \tau=t/T , where T is the number of timestep of DDPM. We obtain \displaystyle d\overline{X}_{\tau} \displaystyle=T\left(\frac{\beta(t)}{\sigma_{t}}\epsilon_{\theta}\left(\overline{X}_{\tau},t\right)-\frac{\beta(t)}{2}\overline{X}_{\tau}\right)d\tau+\sqrt{T}\sqrt{\beta(t)}d\overline{W}_{\tau} \displaystyle=b\left(\overline{X}_{\tau},\tau\right)d\tau+\sigma(\tau)d\overline{W}_{\tau}. The Euler-Maruyama discretization can be written as \displaystyle\overline{X}_{\tau}^{(h)}=\begin{cases}\overline{X}_{sh}^{(h)}+b\left(\overline{X}_{sh},sh\right)(\tau-sh)+\sigma(sh)(\overline{W}_{\tau}-\overline{W}_{sh}),&\tau\in[(s-1)h,sh],\\ \overline{X}_{1}^{(h)},&\tau=1\end{cases} Here, h denotes the time interval, which equals 1/T .

Read the complete text