Solving Inverse Problems via Diffusion Optimal Control

Henry Li, Marcus Pereira · 2024

Read Solving Inverse Problems via Diffusion Optimal Control by Henry Li, Marcus Pereira (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16748.txt.

Introduction

Henry Li and Marcus Pereira recast a popular image-repair method as a steering problem. Diffusion models can rebuild a clean image from a blurred, noisy, or partly missing one, a task called an inverse problem. The usual approach samples from a target distribution, which brings tangles: the odds are hard to compute, and early guesses of the clean image are poor. Their 2024 paper takes another road.

The pair treat the image-building process as a chain of control steps and design a controller modeled on an iterative regulator used in robotics. The method accepts any forward model that can be differentiated, from sharpening and filling gaps to highly nonlinear neural classifiers. The older sampling equation falls out as a special case. Tested against rival solvers, the method set a new baseline for reconstruction quality on standard inverse problems.

Catalog record

CREATORHenry Li, Marcus Pereira
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16748

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Solving Inverse Problems via Diffusion Optimal Control Henry Li, Marcus Pereira Abstract Existing approaches to diffusion-based inverse problem solvers frame the signal recovery task as a probabilistic sampling episode, where the solution is drawn from the desired posterior distribution. This framework suffers from several critical drawbacks, including the intractability of the conditional likelihood function, strict dependence on the score network approximation, and poor 𝐱 0 \mathbf{x}_{0} prediction quality. We demonstrate that these limitations can be sidestepped by reframing the generative process as a discrete optimal control episode. We derive a diffusion-based optimal controller inspired by the iterative Linear Quadratic Regulator (iLQR) algorithm. This framework is fully general and able to handle any differentiable forward measurement operator, including super-resolution, inpainting, Gaussian deblurring, nonlinear deblurring, and even highly nonlinear neural classifiers. Furthermore, we show that the idealized posterior sampling equation can be recovered as a special case of our algorithm. We then evaluate our method against a selection of neural inverse problem solvers, and establish a new baseline in image reconstruction with inverse problems 1 1 1 Code is available at https://github.com/lihenryhfl/diffusion_optimal_control . . 1 Introduction Diffusion models Song and Ermon 2019 ; Ho et al. 2020 have been shown to be remarkably adept at conditional generation tasks Dhariwal and Nichol 2021 ; Ho and Salimans 2022 , in part due to their iterative sampling algorithm, which allows the dynamics of an uncontrolled prior score function \nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}) to be directed towards an arbitrary posterior distribution by introducing an additive guidance term \mathbf{u} . When this guidance term is the conditional score \nabla_{\mathbf{x}}\log p_{t}(\mathbf{y}|\mathbf{x}) , the resulting sample is provably drawn from the desired conditional distribution p(\mathbf{x}|\mathbf{y}) Song et al. 2020 . A central obstacle to this framework is the general difficulty of obtaining the conditional score function \nabla_{\mathbf{x}}\log p_{t}(\mathbf{y}|\mathbf{x}_{t}) due to its dependence on the noisy diffusion variate \mathbf{x}_{t} rather than just the final sample \mathbf{x}_{0} Chung et al. 2023a . In large-scale conditional generation tasks such as class- or text-conditional sampling the computational overhead of training a time-dependent conditional score function from scratch is deemed acceptable, and is indeed the approach taken by Rombach et al. 2022 , Saharia et al. 2022 , and many others. However, this solution is not acceptable in inverse problems where the goal is to design a generalized solver that will work in a zero-shot capacity for an arbitrary forward model. This bottleneck has spawned a flurry of recent research dedicated to approximating the conditional score \nabla_{\mathbf{x}}\log p_{t}(\mathbf{y}|\mathbf{x}_{t}) as a simple function of the noiseless likelihood \log p(\mathbf{y}|\mathbf{x}_{0}) Choi et al. 2021 ; Chung et al. 2022 ; Rout et al. 2024 ; Chung et al. 2023a ; Kawar et al. 2022 ; Chung et al. 2023b . However, as we will demonstrate in this work, these approximations impose a significant cost to the performance of the resulting algorithm. To address these issues, we propose a novel framework built from optimal control theory where such approximations are no longer necessary. By framing the reverse diffusion process as an optimal control episode, we are able to detach the inverse problem solver from the strict requirements of the conditional sampling equation given by Song et al. 2020 , while still leveraging the exceptionally powerful prior of the unconditional diffusion process. Moreover, we find that the desired score function directly arises as the Jacobian of the value function. We summarize our contributions as follows: • We present diffusion optimal control, a framework for solving inverse problems via the lens of optimal control theory, using pretrained unconditional off-the-shelf diffusion models. • We show that this perspective overcomes many core obstacles present in existing diffusion-based inverse problem solvers. In particular, the idealized posterior sampling score Song et al. 2021 — approximated by existing methods — can be recovered exactly as a specific case of our method. • We showcase the advantages of our model empirically with quantitative experiments and qualitative examples, and demonstrate state-of-the-art performance on the FFHQ 256\times 256 dataset. Figure 1: Conceptual illustration comparing a probabilistic posterior sampler to our proposed optimal control-based sampler. In a probabilistic sampler, the model relies on an approximation \tilde{\mathbf{x}}_{0}\approx\mathbf{x}_{0} to guide each step (left) . We are able to compute \mathbf{x}_{0} exactly on each step, resulting in much higher quality gradients \nabla\log p(\mathbf{y}|\tilde{\mathbf{x}}_{0}) and an improved trajectory update (right) . 2 Background Notation We use lowercase letters for denoting scalars a\in\mathbb{R} , lowercase bold letters for vectors \mathbf{a}\in\mathbb{R}^{n} and uppercase bold letters for matrices \mathbf{A}\in\mathbb{R}^{m\times n} . Subscripts indicate Jacobians and Hessians of scalar functions, e.g. l_{\mathbf{x}}\in\mathbb{R}^{n} and l_{\mathbf{x}\mathbf{x}}\in\mathbb{R}^{n\times n} for l(\mathbf{x}):\mathbb{R}^{n}\rightarrow\mathbb{R} , respectively. We overload notation for time-dependent variables, where subscripts imply dependence rather than derivatives w.r.t. time, e.g., \mathbf{x}_{t}=\mathbf{x}(t) . Furthermore, V(\mathbf{x}_{t}) and Q(\mathbf{x}_{t},\mathbf{u}_{t}) are scalar functions despite being uppercase, in line with existing optimal control literature Betts 1998 . 2.1 Diffusion Models The diffusion modeling literature uses the following reverse-time Itö SDE to generate samples Song et al. 2021 , \mathrm{d}\mathbf{x}_{t}=\big[\mathbf{f}(\mathbf{x}_{t})-g(t)^{2}\nabla_{\mathbf{x}_{t}}\log p_{t}(\mathbf{x}_{t})\big]\mathrm{d}t+g(t)\mathrm{d}\mathbf{w}_{t}, (1) where \mathbf{x}_{t}\in\mathbb{R}^{n} is the state vector, \mathbf{f}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n} and g:\mathbb{R}\rightarrow\mathbb{R} are drift and diffusion terms that can take different functional forms (e.g., Variance-Preserving SDEs (VPSDEs) and Variance-Exploding SDEs (VESDEs) in Song et al. 2021 ), \nabla_{\mathbf{x}_{t}}\log p_{t}(\mathbf{x}_{t}) is the score-function and \mathbf{w}_{t}\in\mathbb{R}^{n} is a vector of mutually independent Brownian motions. The above SDE has an associated ODE called the probability-flow (PF) ODE given by \mathrm{d}\mathbf{x}_{t}=\mathrm{d}\mathbf{x}_{t}+\big[\mathbf{f}(\mathbf{x}_{t})-\frac{1}{2}g(t)^{2}\nabla_{\mathbf{x}_{t}}\log p_{t}(\mathbf{x}_{t})\big]\mathrm{d}t, (2) with the same marginals p_{t}(\mathbf{x}_{t}) as the SDE, which allow for likelihood computation ( Song et al. 2021 ; Li et al. 2024 ) . All practical implementations of diffusion samplers require a time-discretization of the PF-ODE. One such discretization is the well-known Euler-discretization which gives, \mathbf{x}_{t-1}=\mathbf{x}_{t}-[\mathbf{f}(\mathbf{x}_{t})-\frac{1}{2}g(t)^{2}\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t})]\Delta t (3) where, \Delta t is the length of the discretization interval and we have reversed the time evolution by changing the sign of the drift. We are not restricted to only using the Euler-discretization and any high-order discretization techniques can also be employed. More concisely, we have, \mathbf{x}_{t-1}=\mathbf{h}(\mathbf{x}_{t}),\,\,\text{ where }\mathbf{h}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n} (4) which describes the general non-linear dynamics of the corresponding discrete-time diffusion sampler. Figure 2: Predicted \mathbf{x}_{0} used in a probabilistic framework (above) compared to ours (below) for a general diffusion trajectory. The full forward rollout in our proposed framework allows for the predicted \mathbf{x}_{0} (and therefore \nabla_{\mathbf{x}_{t}}\log p(\mathbf{y}|\mathbf{x}_{0}) ) to be efficiently computed for all t=0,\dots,T . 2.2 Posterior Sampling for Inverse Problems Inverse problems are a general class of problems where an unknown signal is reconstructed from observations obtained by a forward measurement process Ongie et al. 2020 . The forward process is usually lossy, resulting in an ill-posed signal recovery task where a unique solution does not exist. The forward model can generally be written as y=\mathcal{A}(\mathbf{x}_{0})+\eta, (5) where \mathcal{A}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{d} is the forward operator, y\in\mathbb{R}^{d} the measured signal, \mathbf{x}_{0}\in\mathbb{R}^{n} the unknown signal to be recovered, and \eta\sim\mathcal{N}(0,\sigma\mathbf{I}_{d}) the noise (with variance \sigma^{2} ) in the measurement process. Given the forward model Eq. ( 5 ) and a measurement \mathbf{y} , sampling from the posterior distribution p_{\theta}(\mathbf{x}|\mathbf{y}) can then be performed by solving the corresponding conditional Itö SDE \displaystyle\mathrm{d}\mathbf{x} \displaystyle=[\mathbf{f}(\mathbf{x})-g(t)^{2}\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}|\mathbf{y})]\mathrm{d}t+g(t)\mathrm{d}\mathbf{w}, (6) where, invoking Bayes rule, \nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}|\mathbf{y})=\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x})+\nabla_{\mathbf{x}}\log p_{t}(\mathbf{y}|\mathbf{x}). (7) As with the unconditional dynamics, Eq. ( 6 ) has a corresponding ODE \displaystyle\mathrm{d}\mathbf{x} \displaystyle=[\mathbf{f}(\mathbf{x})-\frac{1}{2}g(t)^{2}\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}|\mathbf{y})]\mathrm{d}t, (8) which

Read the complete text