Supervised Guidance Training for Infinite-Dimensional Diffusion Models

Elizabeth L. Baker, Alexander Denker, Jes Frellsen · 2026

Read Supervised Guidance Training for Infinite-Dimensional Diffusion Models by Elizabeth L. Baker, Alexander Denker, Jes Frellsen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20756.txt.

Introduction

Elizabeth Baker, Alexander Denker, and Jes Frellsen extend diffusion models to functions, not just finite data. Their 2026 paper works with score-based diffusion models on infinite-dimensional function spaces. Such models can help solve inverse problems that come from partial differential equations. An inverse problem asks what hidden cause produced a set of noisy observations.

In the Bayesian view, the goal is to sample from a posterior distribution over functions. That distribution blends prior knowledge with the observed data. The paper adds supervised guidance to steer that sampling. Guidance uses extra information to push samples toward answers that fit. The work matters wherever the unknown is a whole function, not a single number. It shows how modern generative models can serve classic inference tasks.

Catalog record

CREATORElizabeth L. Baker, Alexander Denker, Jes Frellsen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20756

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Supervised Guidance Training for Infinite-Dimensional Diffusion Models Elizabeth L. Baker, Alexander Denker, Jes Frellsen Abstract Score-based diffusion models have recently been extended to infinite-dimensional function spaces, with uses such as inverse problems arising from partial differential equations. In the Bayesian formulation of inverse problems, the aim is to sample from a posterior distribution over functions obtained by conditioning a prior on noisy observations. While diffusion models provide expressive priors in function space, the theory of conditioning them to sample from the posterior remains open. We address this, assuming that either the prior lies in the Cameron-Martin space, or is absolutely continuous with respect to a Gaussian measure. We prove that the models can be conditioned using an infinite-dimensional extension of Doob’s h h -transform, and that the conditional score decomposes into an unconditional score and a guidance term. As the guidance term is intractable, we propose a simulation-free score matching objective (called Supervised Guidance Training ) enabling efficient and stable posterior sampling. We illustrate the theory with numerical examples on Bayesian inverse problems in function spaces. In summary, our work offers the first function-space method for fine-tuning trained diffusion models to accurately sample from a posterior. † † affiliationnotice: Equal contribution 1 Introduction Inverse problems, which aim to infer unknown parameters from observations, are ubiquitous in science and engineering. These problems are typically ill-posed, meaning that the solution may be unstable in the presence of noise, and multiple parameters may map to the same observed state. The Bayesian framework offers a principled solution by targeting a posterior distribution conditioned on data rather than a single point estimate ( Stuart 2010 ) . Frequently, these problems originate from partial differential equations (PDEs), where the unknown parameters exist in function spaces. This means that the target probability measures are inherently infinite-dimensional. While score-based diffusion models (SDMs) have recently been extended to infinite dimensions ( Franzese et al. 2023 ; Pidstrigach et al. 2024 ; Lim et al. 2025 ; Franzese & Michiardi 2025 ) , conditioning such models to sample from posterior distributions remains an open problem. In finite-dimensional settings, conditioning is often achieved via Doob’s h -transform ( Rogers & Williams 2000 ; Zhao et al. 2025 ) , where the conditional score decomposes into an unconditional score and a guidance term ( Dhariwal & Nichol 2021 ; Chung et al. 2023 ) . Various methods have been proposed to approximate this typically intractable guidance term, ranging from heuristic approximations ( Chung et al. 2023 ; Yu et al. 2023 ; Finzi et al. 2023 ) to learning the term directly ( Denker et al. 2024 ; Zhang et al. 2023 ) . Some of these methods have been extended to infinite-dimensional SDMs. Baldassari et al. 2023 propose a framework to train a conditional infinite-dimensional SDM to directly estimate the posterior distribution. FunDPS ( Yao et al. 2025 ) extends DPS ( Chung et al. 2023 ) to infinite dimensions in the case where the data is contained in the Cameron-Martin space of the Wiener process. Lifting these results into infinite dimensions is not straightforward. For example, Lebesgue densities do not exist, so score functions must be designed without referring to densities. Furthermore, the definition of conditional scores relies on transition operators of the forward and reverse stochastic differential equations (SDEs) being differentiable; this does not always hold in infinite dimensions. We must also contend with issues arising from the Feldman-Hájek theorem: shifting Gaussian measures by an element not in their Cameron-Martin space leads to mutually singular measures. This means we cannot rely on the existence of densities between the data and the measure induced by the Wiener process of the SDE. At the same time, if we ignore the infinite-dimensional setting by discretising the data immediately, it is unclear how good the approximation is and whether the solutions will behave as the resolution of the discretisation is increased. By treating the theory in infinite dimensions and only discretising in the last step, we can better understand the errors induced by discretisation. Moreover, we unlock alternative and more flexible options in terms of discretisation, training and loss functions. This results in a more general resolution-invariant framework for conditioning infinite-dimensional SDMs. Contributions In this work, we develop a principled framework for conditioning diffusion models in function space using an infinite-dimensional version of Doob’s h -transform. We prove the existence of the conditional diffusion process under two general settings: (1) the prior distribution lies in the Cameron-Martin space of the Wiener process used in the diffusion process, or (2) the prior is absolutely continuous with respect to a Gaussian measure. Further, we prove a decomposition of the conditional score term into the unconditional score term and a guidance term. We introduce Supervised Guidance Training , an algorithm for learning the typically intractable guidance term. This approach is based on a denoising score matching loss and thus avoids the computational overhead of simulation-based training regimes. Further, we explore connections between our framework, stochastic optimal control, and Tweedie-based approximations, where we prove the approximation in FunDPS ( Yao et al. 2025 ) with less restrictive assumptions. Finally, we validate our method on Bayesian inverse problems, demonstrating that explicitly learning the infinite-dimensional guidance term yields superior accuracy compared to heuristic baselines. 2 Background and setup 2.1 Bayesian Inverse Problems A major motivation for our work is Bayesian inverse problems in function spaces ( Stuart 2010 ; Dashti & Stuart 2017 ) . Consider a measurable forward operator G:\mathcal{H}\to\mathcal{Y} for two Hilbert spaces \mathcal{H} and \mathcal{Y} and assume that we have access to observations of the form y=G(f)+\eta , where f is the parameter to be identified and \eta\in\mathcal{Y} denotes observation noise. We place a prior measure \pi on f\in\mathcal{H} and model the observation noise with a measure \pi_{0} on \mathcal{Y} such that f and \eta are independent. The conditional random variable y|f then has a measure \pi_{f} , which is given as \pi_{0} translated by G(f) . We assume \pi_{f}\ll\pi_{0} for f \pi -almost surely and for some potential \Phi:\mathcal{H}\times\mathcal{Y}\to\mathbb{R} , write the likelihood as \displaystyle\frac{\mathrm{d}\pi^{f}}{\mathrm{d}\pi_{0}}(y)=\exp(-\Phi(f,y)). (1) By the infinite-dimensional Bayes theorem (Theorem 14 in Dashti & Stuart 2017 ), the posterior satisfies \pi^{y}\ll\pi for almost every y and for \xi=\mathbb{E}_{f\sim\pi}[\exp(-\Phi(f,y))] \displaystyle\frac{\mathrm{d}\pi^{y}}{\mathrm{d}\pi}(f)=\frac{1}{\xi}\exp(-\Phi(f,y)). (2) This framework supports Bayesian inference for PDE-based inverse problems in their natural function-space setting. In many applications, the noise distribution can be selected or calibrated from prior experimental data, while the prior must be modelled to encode structural information about f . In this work, we construct this prior using an infinite-dimensional diffusion model, enabling a data-driven representation of the underlying function space. 2.2 Conditioning in Finite Dimensions We start by conditioning SDMs in the finite-dimensional setting, with state space \mathbb{R}^{d} . Let x_{t} be the forward process, satisfying an SDE with drift \mathbf{f}(t,x) and diffusion g(t) , such that x_{0}\sim\pi_{\text{data}} . Denote by p_{t}(x) the marginal density of x_{t} . Letting \tilde{t}\coloneqq T-t and \mathbf{w}_{t} be a Wiener process, the time-reversal of the forward SDE satisfies \displaystyle\mathrm{d}z_{t}=\left[\mathbf{f}(\tilde{t},z_{t})\!+\!g^{2}(\tilde{t})\nabla_{z}\log p_{\tilde{t}}(z_{t})\right]\mathrm{d}t+g(\tilde{t})\mathrm{d}\mathbf{w}_{t}, (3) with z_{0}\sim\text{Law}(x_{T})\approx\mathcal{N}(0,C_{\pi_{\text{data}}}) , see Song et al. 2021 . To solve an inverse problem given observations y , we seek to sample from the posterior distribution p_{0}(x_{0}|y) . This requires conditioning the reverse

Read the complete text