A Malliavin calculus approach to score functions in diffusion generative models

Ehsan Mirafzali 1∗ , Frank Proske 2 , Utkarsh Gupta 1 , Daniele Venturi 3 , Razvan Marinescu 1 1 Department of Computer Science, University of California Santa Cruz, Santa Cruz, California, USA. 2 Department of Mathematics, University of Oslo, Oslo, Norway. 3 Department of Applied Mathematics, Unive · 2025

Read A Malliavin calculus approach to score functions in diffusion generative models by Ehsan Mirafzali 1∗ , Frank Proske 2 , Utkarsh Gupta 1 , Daniele Venturi 3 , Razvan Marinescu 1 1 Department of Computer Science, University of California Santa Cruz, Santa Cruz, California, USA. 2 Department of Mathematics, University of Oslo, Oslo, Norway. 3 Department of Applied Mathematics, Unive (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05550.txt.

Introduction

Ehsan Mirafzali and co-authors derive an exact formula for the score function in diffusion generative models. Their 2025 paper works with the models that build images and other data by reversing a gradual noising process. The score, a gradient that steers that reversal, is usually only estimated from data by rough methods.

Their route joins Malliavin calculus, a tool for random processes, with a new Bismut-type formula. The final expression uses only the first and second variation processes of the system, with all Malliavin terms removed. That cleaner form should be easier to use in practice. The authors offer the result as a firm base for new sampling methods for complex distributions.

Catalog record

CREATOREhsan Mirafzali 1∗ , Frank Proske 2 , Utkarsh Gupta 1 , Daniele Venturi 3 , Razvan Marinescu 1 1 Department of Computer Science, University of California Santa Cruz, Santa Cruz, California, USA. 2 Department of Mathematics, University of Oslo, Oslo, Norway. 3 Department of Applied Mathematics, Unive
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05550

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Malliavin calculus approach to score functions in diffusion generative models Ehsan Mirafzali 1∗ , Frank Proske 2 , Utkarsh Gupta 1 , Daniele Venturi 3 , Razvan Marinescu 1 1 Department of Computer Science, University of California Santa Cruz, Santa Cruz, California, USA. 2 Department of Mathematics, University of Oslo, Oslo, Norway. 3 Department of Applied Mathematics, University of California Santa Cruz, Santa Cruz, California, USA. *Corresponding author(s). E-mail(s): [email protected] ; Contributing authors: [email protected] ; [email protected] ; [email protected] ; [email protected] ; Abstract Score-based diffusion generative models have recently emerged as a powerful tool for modelling complex data distributions. These models aim at learning the score function, which defines a map from a known probability distribution to the target data distribution via deterministic or stochastic differential equations (SDEs). The score function is typically estimated from data using a variety of approximation techniques, such as denoising or sliced score matching, Hyvärien’s method, or Schrödinger bridges. In this paper, we derive an exact, closed form, expression for the score function for a broad class of nonlinear diffusion generative models. Our approach combines modern stochastic analysis tools such as Malliavin derivatives and their adjoint operators (Skorokhod integrals or Malliavin Divergence) with a new Bismut-type formula. The resulting expression for the score function can be written entirely in terms of the first and second variation processes, with all Malliavin derivatives systematically eliminated, thereby enhancing its practical applicability. The theoretical framework presented in this work offers a principled foundation for advancing score estimation methods in generative modelling, enabling the design of new sampling algorithms for complex probability distributions. Our results can be extended to broader classes of stochastic differential equations, opening new directions for the development of score-based diffusion generative models. keywords Score function, diffusion models, Bismut formula, Malliavin calculus, stochastic differential equations, generative modelling. 1 Introduction Diffusion generative models ( Sohl-Dickstein et al. 2015 ) have recently emerged as a powerful tool for modelling complex data distributions in a variety of applications ranging from image synthesis ( Dhariwal and Nichol 2021 ; Rombach et al. 2022 ) to finance ( Sattarov et al. 2023 ) . Score-based diffusion models ( Song et al. 2021 ) , which study diffusion processes through score functions and SDEs, provide an equivalent formulation of these generative models. Rather than learning the data distribution directly, these models estimate the so-called score function \nabla_{y}\log p_{t}(y) , which governs the transformation of a simple reference distribution into the target distribution via a time-reversed nonlinear diffusion process. Historically, score-based generative models have been built upon diffusion processes described by linear Itô’s stochastic differential equations, i.e., SDEs with drift and diffusion coefficients that are linear in the state variables. Such a linearity assumption ensures analytical tractability and comes with other desirable properties. For example, the Fokker–Planck equation ( Fokker 1914 ; Planck 1917 ) associated with the diffusion process admits a Gaussian stationary distribution and Gaussian transition densities, which have been leveraged in the development of techniques such as Denoising Diffusion Probabilistic Models (DDPM) ( Ho et al. 2020 ) . While the linearity assumption in diffusion processes has been instrumental in developing the first generation of score-based generative models, it also represents an overly restrictive simplification. Many real-world systems exhibit nonlinear dynamics, where both the drift and diffusion coefficients depend nonlinearly on the state variables. Extending diffusion models to such settings poses substantial challenges. Specifically, unlike linear SDEs, nonlinear SDEs typically lack closed-form expressions for their transition probability densities, making the computation of the score function \nabla_{y}\log p_{t}(y) extremely challenging. The purpose of this paper is to bridge this gap by establishing a new rigorous link between score-based diffusion models and Malliavin calculus, a stochastic calculus of variations introduced by Paul Malliavin in the 1970s ( Malliavin 1978 ; Malliavin 1997 ) . Originally devised to investigate hypoelliptic operators and stochastic partial differential equations, Malliavin calculus provides a robust framework for analysing the smoothness of SDE solutions and their densities. By leveraging Malliavin derivatives and a novel Bismut-type formula we derive in this paper, which is distinct from the classical Bismut–Elworthy–Li formula ( Bismut 1984 ; Elworthy and Li 1994 ; Elworthy 1982 ) tailored to heat kernels, we derive an explicit representation of the score function for nonlinear diffusion processes. The resulting formula is expressed entirely in terms of first and second variation processes ( Kunita 1997 ; Bismut 1981 ) , with all Malliavin derivatives systematically eliminated. In particular, our framework generalises the state‐independent results of ( Mirafzali et al. 2025 ) to fully nonlinear SDEs with state‐dependent diffusion coefficients. For the numerical experiments of this framework, see the same work ( Mirafzali et al. 2025 ) . Very recently, other studies have applied Malliavin calculus to diffusion generative models for different ends: one focuses on robust numerical treatment of singular diffusion bridges and conditional control, and additionally derives an explicit yet distinct conditional-score formula via a generalised Tweedie identity ( Pidstrigach et al. 2025 ) , and another proposes an extension of diffusion models to infinite‐dimensional diffusion settings using Malliavin calculus ( Greco 2025 ) . The implications of this new theoretical framework are profound. First, Malliavin calculus provides a powerful tool for overcoming the analytical challenges posed by nonlinear diffusion processes. Second, its rigorous foundations enable the development and analysis of new classes of generative models beyond linear diffusion. From a practical standpoint, this can potentially yield more accurate generative models for applications such as high-fidelity image synthesis, fluid dynamics Venturi et al. 2013 , and protein folding simulations in molecular dynamics ( Abramson et al. 2024 ) . This paper is organised as follows. In Section 2 we state our main theorem expressing the score function \nabla_{y}\log p_{t}(y) in terms of first and second variation processes via a Bismut-type formula. Unlike the Bismut-Elworthy-Li formula, our formula directly targets the score function, leveraging the first and second variation processes to yield a practical closed form expression. In Section 3 we briefly review basic notions of Malliavin calculus and and prove various theorems on covering vector fields. In Section 4 we discuss first and second variation processes in detail. In Section 5 we prove our main theorem, i.e., Theorem 1 . The proof includes the full derivation of the score function for nonlinear SDEs, along with several useful lemmas that assist in the process. In Section 6 we provide a simplified expression of the score function for nonlinear SDEs with state-independent diffusion coefficients. 2 Problem statement and main result Consider the m -dimensional stochastic differential equation dX_{t}=b(t,X_{t})\,dt+\sigma(t,X_{t})\,dB_{t},\qquad X_{0}=x,\qquad 0\leq t\leq T, (1) where X_{t}\in\mathbb{R}^{m} , B_{t}\in\mathbb{R}^{d} is a standard Brownian motion, b:[0,T]\times\mathbb{R}^{m}\to\mathbb{R}^{m} is the drift coefficient, and \sigma:[0,T]\times\mathbb{R}^{m}\to\mathbb{R}^{m\times d} is the diffusion coefficient, with \sigma^{l}(t,X_{t}) denoting its l -th column for l=1,\ldots,d . The initial condition x\in\mathbb{R}^{m} is deterministic, and T>0 is a fixed terminal time. We assume that b and \sigma are \mathcal{C}^{2} functions with bounded derivatives, which ensures the existence of a unique strong solution to ( 1 ). Let \gamma_{X_{T}}:=\bigl(\,\langle DX_{T}^{i},DX_{T}^{j}\rangle_{L^{2}([0,T])}\bigr)_{i,j=1}^{m} denote the Malliavin covariance matrix of X_{T} . If X_{T}\in\mathbb{D}^{\infty} and \gamma_{X_{T}} is almost surely invertible with \mathbb{E}\!\bigl[\|\gamma_{X_{T}}^{-1}\|^{p}\bigr]<\infty for every p>0 , then the terminal value X_{T} admits a smooth probability density p(y) with respect

Read the complete text