Probability-Flow ODE in Infinite-Dimensional Function Spaces

Kunwoo Na, Junghyun Lee, Se-Young Yun, Sungbin Lim · 2025

Read Probability-Flow ODE in Infinite-Dimensional Function Spaces by Kunwoo Na, Junghyun Lee, Se-Young Yun, Sungbin Lim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10219.txt.

Introduction

Kunwoo Na, Junghyun Lee, Se-Young Yun, and Sungbin Lim speed up a class of AI generators in a 2025 paper. The generators build whole functions at once, such as the answer curves of physics equations, rather than single values. Drawing a sample takes many steps of a noisy process, and each step costs time on big problems.

They derive, for the first time, a flow equation that mirrors a known shortcut from flat spaces, now in the endless dimensional space of functions. The shortcut lets the generator take far fewer steps while keeping sample quality high. They show the gain on function making tasks and on work tied to partial differential equations.

Catalog record

CREATORKunwoo Na, Junghyun Lee, Se-Young Yun, Sungbin Lim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10219

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Probability-Flow ODE in Infinite-Dimensional Function Spaces Kunwoo Na, Junghyun Lee, Se-Young Yun, Sungbin Lim Abstract Recent advances in infinite-dimensional diffusion models have demonstrated their effectiveness and scalability in function generation tasks where the underlying structure is inherently infinite-dimensional. To accelerate inference in such models, we derive, for the first time, an analog of the probability-flow ODE (PF-ODE) in infinite-dimensional function spaces. Leveraging this newly formulated PF-ODE, we reduce the number of function evaluations while maintaining sample quality in function generation tasks, including applications to PDEs. 1 Introduction Diffusion model ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ; Song et al., 2021b ; Kingma et al., 2021 ) is a class of generative model that adds noise to real data to train the score network and sequentially approximate the time-reversed process ( Föllmer and Wakolbinger, 1986 ; Anderson, 1982 ) to generate samples from the true data distribution. This model has shown remarkable empirical success in numerous domains such as image generation ( Song et al., 2021b ; Song et al., 2021a ) , video generation ( Luo et al., 2023 ) , medical data processing ( Song et al., 2022 ; Chung and Ye, 2022 ; Akrout et al., 2023 ) , and audio generation ( Kong et al., 2020 ) . However, ’’classical” diffusion models formulated on finite-dimensional Euclidean spaces limit their applicability to function generation problems as they can only generate function values realized on a fixed discretization of the function‘s domain ( Li et al., 2020 ) and cannot capture functional properties of a data such as integrability or smoothness ( Kerrigan et al., 2023 ) . Motivated by such a limitation of finite-dimensional models, there has been a line of works extending the finite-dimensional diffusion model to infinite-dimensional Hilbert spaces; for instance, Hagemann et al. (2023) ; Kerrigan et al. (2023) ; Lim et al. (2023a) ; Lim et al. (2023b) ; Pidstrigach et al. (2023) ; Phillips et al. (2022) ; Baldassari et al. (2023) . Kerrigan et al. (2023) proposes a discrete-time model that serves as an analog of Ho et al. (2020) in infinite-dimensional space, and Hagemann et al. (2023) introduces a finite-dimensional approximation of an infinite-dimensional SDEs and utilizes the time-reversal formula in finite-dimensional spaces. Lim et al. (2023a) ; Franzese et al. (2023) ; Pidstrigach et al. (2023) propose continuous-time models by extending the SDE framework of Song et al. (2021b) to infinite dimensions based on semigroup theory (ref. Da Prato and Zabczyk (2014) ); however, their consideration is limited to a relatively simple class of SDEs, such as Langevin type SDE or SDEs with constant-time diffusion coefficients. Later, Lim et al. (2023b) proved a general form of time-reversal formula which encompasses various choices of SDEs such as VPSDE, VESDE, sub-VPSDE ( Song et al., 2021b ) and variance scheduling ( Nichol and Dhariwal, 2021 ) , by exploiting more advanced mathematical machinery, e.g., variational approach and functional derivatives (ref. Krylov and Rozovskii (2007) ; Bogachev and Mayer-Wolf (1999) ). Research works mentioned above are primarily focused on the training of diffusion models, i.e., they aim to implement a mathematical framework in which the score-matching objective ( Sohl-Dickstein et al., 2015 ; Vincent, 2011 ) and time reversal ( Föllmer and Wakolbinger, 1986 ; Millet et al., 1989 ) of the noising process are possible. Although the ’’SDE‘‘ component and the ’’score-matching‘‘ component of the finite-dimensional diffusion model have been transferred to infinite dimensions, the existence of an infinite dimensional analog of probability-flow ODE (PF-ODE; Song et al. (2021b) ) is still open. Indeed, PF-ODE has been crucial in the sampling process of diffusion models as it allows for fast sampling ( Chen et al., 2023 ; Lu et al., 2022b ) and consistency modeling ( Song et al., 2023 ) . In this work, we aim to accelerate the inference process of infinite-dimensional diffusion models by extending the probability-flow ODE ( Song et al., 2021b ) to infinite-dimensional spaces. Contributions. Our contributions are as follows: • We derive in a mathematically rigorous manner the notion of probability-flow ODE (Theorem 3.1 ) associated with a general class of stochastic differential equations (SDEs) in infinite-dimensional spaces, including VPSDE, VESDE, sub-VPSDE ( Song et al., 2021b ) and variance scheduling ( Nichol and Dhariwal, 2021 ) . We note that our infinite-dimensional probability-flow ODE is widely applicable regardless of the specific formulation of the infinite-dimensional diffusion model. • We empirically demonstrate that sampling with PF-ODE achieves comparable or superior generation quality to the previous SDE-based approach while requiring significantly fewer number of function evaluations (NFEs) in both toy and real-world PDE problems. 2 Preliminaries 2.1 Probability-flow ODE in \mathbf{R}^{n} Let us consider the following stochastic differential equation in \mathbf{R}^{n} ( n<\infty ) over t\in[0,T] : dX_{t}=f(t,X_{t})dt+\sigma(t)dB_{t},\quad X_{0}\sim p_{0}=p_{\mathrm{data}}, (1) where (B_{t})_{t\geq 0} is a standard Brownian motion in \mathbf{R}^{n} , f:[0,T]\times\mathbf{R}^{n}\rightarrow\mathbf{R}^{n} is the drift term, \sigma:[0,T]\rightarrow\operatorname{Mat}_{n}(\mathbf{R}) is the diffusion term, and p_{0}=p_{\mathrm{data}} is the probability density of the target data distribution. Closely related to this SDE is the so-called probability-flow ODE (PF-ODE; Song et al. (2021b) ): \displaystyle dY_{t}=\left[f(t,Y_{t})-\frac{1}{2}A(t)\nabla\log p_{t}(Y_{t})\right]dt,\quad Y_{0}\sim p_{0}, where A(t)=\sigma(t)\sigma(t)^{\intercal} , and p_{t} is the density of X_{t} . It is well-known that the solution for the PF-ODE has the same density as X_{t} for each t ( Song et al., 2021b , Appendix D.1) . The derivation of the PF-ODE heavily relies on the Fokker-Planck equation (ref. Øksendal (2003) for example), a well-studied second-order PDE whose solution is (t,x)\mapsto p_{t}(x) . In infinite-dimensional spaces, however, one cannot utilize the probability density function in the analysis due to the lack of reference measure (ref. Lunardi et al. (2015) , Proposition 2.2.1). Hence, a more careful treatment is required for infinite-dimensional cases. 2.2 Infinite-Dimensional Analysis Let \mathcal{H} denote a real separable Hilbert space, and (W_{t})_{t\geq 0} be a Q -Wiener process on \mathcal{H} . Denote by \mathcal{H}_{Q} the Cameron-Martin space (ref. Da Prato and Zabczyk (2014) ) of \mathcal{N}(0,Q) . Let \mathscr{L}_{2}(\mathcal{H}) be the set of Hilbert-Schmidt operators on \mathcal{H} , and let \{\varphi_{i}\} be an orthonormal basis of \mathcal{H} that consists of eigenvectors of Q corresponding to \lambda_{i} . We assume \mathcal{H} is a function space over some set \Omega\subseteq\mathbf{R}^{d} ( d<\infty ); for example, \mathcal{H}=L^{2}(\Omega) or \mathcal{H}=W^{1,2}(\Omega) . Due to the lack of reference measure in \mathcal{H} , we shall express the time evolution of a family of probability measures in a weak sense ; that is, we express the evolution of the dual pairings of a probability measure and test functions. Below, we introduce the minimal background required for this work; we refer the readers to Appendix A for a more detailed overview. Test functions. The class of cylindrical functions \mathcal{FC}_{b}^{\infty}(\mathcal{H}) is defined as \displaystyle\mathcal{FC}_{b}^{\infty}(\mathcal{H})=\left\{x\mapsto f(\left<\varphi_{1},x\right>,\cdots,\left<\varphi_{m},x\right>)\mathrel{\bigg|}m\in\mathbf{N},\ f\in\mathcal{C}_{0}^{\infty}(\mathbf{R}^{m})\right\}. We write f_{\varphi_{1},\cdots,\varphi_{m}}(x)=f(\left<\varphi_{1},x\right>,\cdots,\left<\varphi_{m},x\right>) for x\in\mathcal{H} . Here, \mathcal{C}_{0}^{\infty}(\mathbf{R}^{m}) is the space of smooth functions on \mathbf{R}^{m} that vanish at infinity, which serves as a canonical class of test functions in usual finite-dimensional analysis. Weak formulation. Let \mathcal{L} be an operator such that {\mathcal{L}}\psi:\mathcal{H}\to\mathbf{R} is in L^{1}(\mathcal{H},\mu) for all \psi\in\mathcal{FC}_{b}^{\infty}(\mathcal{H}) . We say {\mathcal{L}}^{\ast}\mu=0 if \displaystyle\int_{\mathcal{H}}{\mathcal{L}}f_{\varphi_{1},\cdots,\varphi_{m}}(x)\mu(dx)=0,\quad\forall f_{\varphi_{1},\cdots,\varphi_{m}}\in\mathcal{FC}_{b}^{\infty}(\mathcal{H}). In a similar manner, for a family of measures \{\nu_{t}\} , we shall understand the equation \mathcal{L}^{\ast}\mu=\partial_{t}\nu_{t} in a weak sense

Read the complete text