A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models

Jiayi Fu, Yuxia Wang, [email protected], [email protected] · 2026

Read A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models by Jiayi Fu, Yuxia Wang, [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22586.txt.

Introduction

Fu and Wang write a unified tutorial on diffusion theory, joining views apart: diffusion probabilistic models, score based modelling, stochastic differential equations and numerical sampling. Starting from a conditional Gaussian noising process, they derive ODE and SDE forms and reach the reverse time equations that make generation possible. They explain the marginal score as the central unknown in reverse sampling and show score matching as the standard denoising aim. DDPM and DDIM, flow matching, and score based SDEs all land in one framework.

Catalog record

CREATORJiayi Fu, Yuxia Wang, [email protected], [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE138575
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22586

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models Jiayi Fu, Yuxia Wang, [email protected], [email protected] Abstract Diffusion models have emerged as a dominant framework for generative modeling, but their mathematical foundations are often presented separately through diffusion probabilistic models, score-based modeling, stochastic differential equations, and numerical sampling methods. We write this tutorial to provide a unified and self-contained account of these viewpoints from the perspective of differential equations. Starting from a conditional Gaussian noising process, we derive ordinary differential equation (ODE) and stochastic differential equation (SDE) representations, pass to the corresponding marginal forward dynamics, and then obtain the reverse-time SDE and probability-flow ODE that make generation possible. We show that the central unknown quantity in reverse sampling is the marginal score, explain how score matching becomes the standard denoising objective under a noise-prediction parameterization, and discuss practical reverse-time sampling and guidance. We further place DDPM, DDIM, flow matching, and score-based SDEs in a common framework, and conclude with diffusion language models in continuous embedding space together with a brief discussion of discrete masked-token diffusion. The tutorial is intended as a bridge between the analytical foundations of diffusion processes and the modern generative algorithms built upon them. Introduction Diffusion models now occupy a central position in modern generative modeling. Their contemporary development combines the diffusion-probabilistic line initiated by nonequilibrium thermodynamics and latent-variable learning [ 23 , 6 , 18 , 11 ] with the score-based line built on score matching and denoising score matching [ 9 , 29 , 25 , 26 , 27 ] . These ideas have supported a broad range of influential generative systems, including methods for large-scale image synthesis, guidance, editing, and latent-space generation [ 4 , 7 , 17 , 16 , 20 , 21 ] . Despite this success, the literature remains technically fragmented. Reverse-time diffusion theory is often discussed through stochastic calculus and Fokker–Planck equations [ 1 , 30 , 19 ] ; deterministic probability-flow viewpoints are frequently introduced through DDIM and modern ODE solvers [ 24 , 15 , 14 , 10 , 22 ] ; and recent unifying perspectives connect diffusion to neural ODEs and flow matching [ 3 , 13 , 8 ] . As a result, readers often encounter DDPM, score-based SDEs, reverse ODEs, flow matching, and noise-prediction losses as distinct constructions, even though they are closely related mathematically. This tutorial is written to make those relationships explicit within a single, coherent narrative. The main body of the tutorial starts from the conditional Gaussian forward process and derives its ODE and SDE representations before passing to the corresponding marginal dynamics. We then show how reverse-time sampling arises from the reverse SDE and the probability-flow ODE, and why the marginal score \nabla\log p_{t}(x) is the central unknown quantity that must be learned. This viewpoint leads naturally to score matching, to the denoising objective used in practical diffusion models, and to a unified interpretation of DDPM and DDIM as discretizations of reverse-time continuous dynamics. We also discuss guided generation and fast sampling from the reverse equations. A second goal is to position diffusion models within neighboring generative frameworks. Accordingly, we compare the reverse-time presentation used in the main text with the generative-time formulations of flow matching and score-based SDE modeling [ 13 , 27 ] . We then show how the same continuous-state formalism extends to diffusion language models in embedding space [ 12 , 28 , 5 ] , while briefly situating discrete masked-token diffusion models [ 2 ] outside the main scope of the tutorial. The appendices collect the analytical tools used throughout, including differential operators, Brownian motion under time reversal, continuity and Fokker–Planck equations, Fisher’s identity, and the orthogonality argument behind the denoising loss. Setup Let X_{0}\in\mathbb{R}^{d} be a data-valued random variable with density p_{0}(x_{0}) , and let t\in[0,1] denote forward time. Throughout the main development, we assume that the noising schedules \alpha_{t} and \sigma_{t} are differentiable functions of t satisfying \alpha_{0}=1,\quad\alpha_{1}=0,\quad\sigma_{0}=0,\quad\sigma_{1}=1,\qquad\alpha_{t}\geq 0,\ \sigma_{t}\geq 0\ \text{for all }t\in[0,1], and that the induced diffusion coefficient is nonnegative: \frac{\,\mathrm{d}}{\,\mathrm{d}t}\sigma_{t}^{2}-2\frac{\dot{\alpha}_{t}}{\alpha_{t}}\sigma_{t}^{2}\geq 0. Typically, \alpha_{t} decreases while \sigma_{t} increases, so that the forward process progressively corrupts the data and terminates at Gaussian noise. We then define f_{t}:=\frac{\dot{\alpha}_{t}}{\alpha_{t}},\qquad g_{t}^{2}:=\frac{\,\mathrm{d}}{\,\mathrm{d}t}\sigma_{t}^{2}-2\frac{\dot{\alpha}_{t}}{\alpha_{t}}\sigma_{t}^{2}. (1) The conditional Gaussian forward kernel is p_{t}(x\mid x_{0}):=\mathcal{N}(x;\alpha_{t}x_{0},\sigma_{t}^{2}I), (2) and the corresponding marginal forward density is p_{t}(x):=\int_{\mathbb{R}^{d}}p_{t}(x\mid x_{0})p_{0}(x_{0})\,\mathrm{d}x_{0}. (3) To describe reverse-time dynamics, we introduce the reverse-time variable \tau:=1-t, (4) and define the reverse process by Y_{\tau}:=X_{1-\tau}. (5) Its density is q_{\tau}(x):=p_{1-\tau}(x). (6) Unless explicitly stated otherwise, all gradients, divergences, and Laplacians are taken with respect to the state variable. We write W_{t} for a standard Brownian motion when an SDE is expressed in forward time t . When the reverse SDE is written in the reverse-time variable \tau , its driving Brownian motion is denoted by W^{\mathrm{rev}}_{\tau} . When the same reverse SDE is written using the original label t:1\to 0 , we denote the corresponding reverse-time Brownian motion by \bar{W}_{t} , where \bar{W}_{t}:=W^{\mathrm{rev}}_{1-t}. Thus W^{\mathrm{rev}}_{\tau} and \bar{W}_{t} represent the same reverse-time noise under two different time parametrizations. Appendix B gives a detailed discussion. 1 Conditional Forward Process 1.1 Conditional Gaussian forward path The conditional forward process is the family of random variables X_{t}\mid X_{0}=x_{0},\qquad 0\leq t\leq 1, whose density path is prescribed by the Gaussian kernel ( 2 ). Equivalently, if \varepsilon\sim\mathcal{N}(0,I) , then X_{t}=\alpha_{t}X_{0}+\sigma_{t}\varepsilon (7) has conditional law X_{t}\mid X_{0}=x_{0}\sim p_{t}(\cdot\mid x_{0}). Figure 1: Conditional forward process. Given a time grid 0=t_{0}<\cdots<t_{n}<t_{n+1}<\cdots<t_{N}=1 and an initial clean sample X_{0} , the forward Gaussian path progressively corrupts the sample. The same path may be viewed either as repeated Gaussian perturbation on the grid or as the solution of the conditional forward ODE/SDE, where z\sim\mathcal{N}(0,(t_{n+1}-t_{n})I) . The terminal state is approximately pure Gaussian noise. Proposition 1.1 (Conditional score) . The score of the conditional Gaussian path is \nabla\log p_{t}(x\mid x_{0})=-\frac{x-\alpha_{t}x_{0}}{\sigma_{t}^{2}}. (8) Proof. Since \log p_{t}(x\mid x_{0})=C_{t}-\frac{1}{2\sigma_{t}^{2}}\|x-\alpha_{t}x_{0}\|_{2}^{2}, where C_{t} is independent of x , differentiating with respect to x gives ( 8 ). ∎ 1.2 Conditional ODE that generates the conditional forward process Theorem 1.2 (Conditional forward ODE) . For every fixed x_{0} , the conditional Gaussian path ( 2 ) is generated by the ODE \frac{\,\mathrm{d}X_{t}}{\,\mathrm{d}t}=u_{t}(X_{t}\mid x_{0}), (9) with velocity field u_{t}(x\mid x_{0}):=\left(\dot{\alpha}_{t}-\frac{\dot{\sigma}_{t}}{\sigma_{t}}\alpha_{t}\right)x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}}x. (10) Equivalently, the conditional density satisfies the conditional continuity equation \partial_{t}p_{t}(x\mid x_{0})=-\nabla\!\cdot\bigl(p_{t}(x\mid x_{0})u_{t}(x\mid x_{0})\bigr). (11) Proof. Fix x_{0} and define r_{t}(x):=x-\alpha_{t}x_{0}. From ( 7 ), X_{t}=\alpha_{t}x_{0}+\sigma_{t}\varepsilon. Differentiating with respect to t gives \frac{\,\mathrm{d}X_{t}}{\,\mathrm{d}t}=\dot{\alpha}_{t}x_{0}+\dot{\sigma}_{t}\varepsilon=\dot{\alpha}_{t}x_{0}+\dot{\sigma}_{t}\frac{X_{t}-\alpha_{t}x_{0}}{\sigma_{t}}, which simplifies to ( 10 ). Thus the ODE ( 9 ) indeed generates the conditional Gaussian path. We now verify ( 11 ) by exact algebra. Since p_{t}(x\mid x_{0})=\frac{1}{(2\pi\sigma_{t}^{2})^{d/2}}\exp\!\left(-\frac{\|r_{t}(x)\|_{2}^{2}}{2\sigma_{t}^{2}}\right), we have \log p_{t}(x\mid x_{0})=-\frac{d}{2}\log(2\pi\sigma_{t}^{2})-\frac{\|r_{t}(x)\|_{2}^{2}}{2\sigma_{t}^{2}}. Because \partial_{t}r_{t}(x)=-\dot{\alpha}_{t}x_{0},\qquad\partial_{t}\|r_{t}(x)\|_{2}^{2}=-2\dot{\alpha}_{t}r_{t}(x)^{\top}x_{0}, we obtain \displaystyle\partial_{t}\log p_{t}(x\mid x_{0}) \displaystyle=-d\frac{\dot{\sigma}_{t}}{\sigma_{t}}-\partial_{t}\!\left(\frac{\|r_{t}(x)\|_{2}^{2}}{2\sigma_{t}^{2}}\right) \displaystyle=-d\frac{\dot{\sigma}_{t}}{\sigma_{t}}+\frac{\dot{\alpha}_{t}}{\sigma_{t}^{2}}r_{t}(x)^{\top}x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}^{3}}\|r_{t}(x)\|_{2}^{2}. Therefore \partial_{t}p_{t}(x\mid x_{0})=p_{t}(x\mid x_{0})\left[-d\frac{\dot{\sigma}_{t}}{\sigma_{t}}+\frac{\dot{\alpha}_{t}}{\sigma_{t}^{2}}r_{t}(x)^{\top}x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}^{3}}\|r_{t}(x)\|_{2}^{2}\right]. (12) Next, \nabla p_{t}(x\mid x_{0})=p_{t}(x\mid x_{0})\nabla\log p_{t}(x\mid x_{0})=-p_{t}(x\mid x_{0})\frac{r_{t}(x)}{\sigma_{t}^{2}}. Since u_{t}(x\mid x_{0})=\dot{\alpha}_{t}x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}}r_{t}(x), we get \displaystyle u_{t}(x\mid x_{0})^{\top}\nabla p_{t}(x\mid x_{0}) \displaystyle=-p_{t}(x\mid x_{0})\left[\frac{\dot{\alpha}_{t}}{\sigma_{t}^{2}}r_{t}(x)^{\top}x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}^{3}}\|r_{t}(x)\|_{2}^{2}\right]. Also, \nabla\!\cdot u_{t}(x\mid x_{0})=\frac{\dot{\sigma}_{t}}{\sigma_{t}}\nabla\!\cdot r_{t}(x)=d\frac{\dot{\sigma}_{t}}{\sigma_{t}}, because r_{t}(x)=x-\alpha_{t}x_{0} and \nabla\!\cdot x=d . Hence \displaystyle\nabla\!\cdot\bigl(p_{t}(x\mid x_{0})u_{t}(x\mid x_{0})\bigr) \displaystyle=u_{t}(x\mid x_{0})^{\top}\nabla p_{t}(x\mid x_{0})+p_{t}(x\mid x_{0})\nabla\!\cdot u_{t}(x\mid x_{0}) \displaystyle=p_{t}(x\mid x_{0})\left[-\frac{\dot{\alpha}_{t}}{\sigma_{t}^{2}}r_{t}(x)^{\top}x_{0}-\frac{\dot{\sigma}_{t}}{\sigma_{t}^{3}}\|r_{t}(x)\|_{2}^{2}+d\frac{\dot{\sigma}_{t}}{\sigma_{t}}\right]. Therefore -\nabla\!\cdot\bigl(p_{t}(x\mid x_{0})u_{t}(x\mid x_{0})\bigr)=p_{t}(x\mid x_{0})\left[-d\frac{\dot{\sigma}_{t}}{\sigma_{t}}+\frac{\dot{\alpha}_{t}}{\sigma_{t}^{2}}r_{t}(x)^{\top}x_{0}+\frac{\dot{\sigma}_{t}}{\sigma_{t}^{3}}\|r_{t}(x)\|_{2}^{2}\right]. (13) Comparing ( 12 ) and ( 13 ) proves ( 11 ). ∎ Lemma 1.3 (Score form of the conditional forward velocity) . The velocity field ( 10 ) can be rewritten as u_{t}(x\mid x_{0})=f_{t}x-\frac{1}{2}g_{t}^{2}\nabla\log p_{t}(x\mid x_{0}). (14)

Read the complete text