Image Restoration via Primal Dual Hybrid Gradient and Flow Generative Model

2025

Read Image Restoration via Primal Dual Hybrid Gradient and Flow Generative Model (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06748.txt.

Introduction

Ji Li and Chao Wang present a method for restoring damaged images, such as blurry, noisy, or partly missing pictures. Their approach plugs a modern flow-based generative model into a classic optimization system as the denoising step. The generative model supplies a strong learned prior about what clean images look like.

Older plug-and-play methods mostly assume smooth Gaussian noise. The new algorithm, built on the primal-dual hybrid gradient method, also handles rougher noise such as Poisson and impulse noise by allowing different measures of fit to the observed image. Tests on denoising, super-resolution, deblurring, and inpainting show it is efficient, light on memory, and more accurate under non-Gaussian noise.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06748

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Image Restoration via Primal Dual Hybrid Gradient and Flow Generative Model Thanks: Corresponding author. Abstract Regularized optimization has been a classical approach to solving imaging inverse problems, where the regularization term enforces desirable properties of the unknown image. Recently, the integration of flow matching generative models into image restoration has garnered significant attention, owing to their powerful prior modeling capabilities. In this work, we incorporate such generative priors into a Plug-and-Play (PnP) framework based on proximal splitting, where the proximal operator associated with the regularizer is replaced by a time-dependent denoiser derived from the generative model. While existing PnP methods have achieved notable success in inverse problems with smooth squared ℓ 2 \ell_{2} data fidelity—typically associated with Gaussian noise—their applicability to more general data fidelity terms remains underexplored. To address this, we propose a general and efficient PnP algorithm inspired by the primal-dual hybrid gradient (PDHG) method. Our approach is computationally efficient, memory-friendly, and accommodates a wide range of fidelity terms. In particular, it supports both ℓ 1 \ell_{1} and ℓ 2 \ell_{2} norm-based losses, enabling robustness to non-Gaussian noise types such as Poisson and impulse noise. We validate our method on several image restoration tasks, including denoising, super-resolution, deblurring, and inpainting, and demonstrate that ℓ 1 \ell_{1} and ℓ 2 \ell_{2} fidelity terms outperform the conventional squared ℓ 2 \ell_{2} loss in the presence of non-Gaussian noise. Introduction Many image restoration tasks in image processing and computer vision can be formulated as solving a linear inverse problem: \bm{y}=\text{noisy}(\bm{A}\bm{x}), (1) where \bm{x}\in\mathbb{R}^{n} denotes the unknown clean image, \bm{y}\in\mathbb{R}^{m} and linear matrix \bm{A}\in\mathbb{R}^{m\times n} describes the degradation model. Image restoration is to recover \bm{x} from the noisy observation \bm{y} . In general, problem ( 1 ) is ill-posed, and direct inversion is infeasible without incorporating prior knowledge about the solution. A widely adopted approach is to compute the maximum a posteriori (MAP) estimate of the posterior distribution p(\bm{x}|\bm{y}) . The MAP estimate corresponds to the most probable solution and is given by: \hat{\bm{x}}=\argmax_{\bm{x}\in\mathbb{R}^{n}}\quad\{\log p(\bm{y}|\bm{x})+\log p(\bm{x})\}, (2) where p(\bm{y}|\bm{x}) models the noise distribution, and p(\bm{x}) is the image prior. Since directly computing \log p(\bm{x}) is often intractable, MAP estimation is commonly reformulated as a regularized optimization problem: \hat{\bm{x}}=\argmin_{\bm{x}\in\mathbb{R}^{n}}\quad F(\bm{A}\bm{x})+G(\bm{x}), (3) where F(\cdot) encodes the data fidelity, measuring consistency between \bm{A}\bm{x} and \bm{y} , while G(\cdot) encodes the regularization, promoting prior knowledge about \bm{x} . Problem ( 3 ) is typically solved using proximal splitting methods ( Parikh, Boyd et al. 2014 ) , especially when G(\bm{x}) is convex. In proximal splitting, the proximal step associated with G(\bm{x}) can be interpreted as a denoising operation. This insight has led to replacing the proximal step with advanced denoisers, such as BM3D ( Dabov et al. 2006 ; Dabov et al. 2007 ) , or learned neural network-based denoisers ( Zhang et al. 2017 ; Zhang et al. 2020 ; Ryu et al. 2019 ) . More recently, generative models that learn image priors directly from data have been incorporated into the Plug-and-Play (PnP) framework ( Martin et al. 2024 ; Zhu et al. 2023 ; Song et al. 2023 ; Graikos et al. 2022 ; Hurault, Leclaire, and Papadakis 2022 ; Kamilov, Mansour, and Wohlberg 2017 ) . Although generative models can in principle provide \log p(\bm{x}) ( Chen et al. 2018 ) , evaluating its gradient via backpropagation is computationally expensive. PnP methods circumvent this issue by directly substituting a denoiser induced by a generative model for the proximal operator. Such approaches have achieved state-of-the-art results in inverse problems using powerful generative priors, including diffusion models ( Chung et al. 2022a ; Chung et al. 2022b ; Song et al. 2022 ; Kawar et al. 2022 ) and flow matching models ( Pokle et al. 2024 ; Zhang et al. 2024 ; Martin et al. 2024 ) . However, prior PnP methods have predominantly focused on inverse problems with Gaussian noise, where the data fidelity term F(\cdot) is typically the squared \ell_{2} loss. While effective under Gaussian noise, this loss function is suboptimal for non-Gaussian noise models. For example, Poisson noise is the more appropriate modeling for the shot noise of photons in the observation. Salt-and-pepper noise, also known as impulse noise, is another form of noise in digital images. Although one can still use squared \ell_{2} loss, the mismatch in noise modeling often leads to subpar restoration quality. In this work, we extend PnP methods based on flow matching generative models to handle non-Gaussian noise by generalizing the data fidelity term F(\cdot) in ( 3 ). Specifically, we consider proximity-friendly fidelity functions such as \ell_{1} and \ell_{2} norms, which are better suited for modeling Poisson and impulse noise, respectively. For sparse impulse noise, \ell_{1} -norm fidelity is widely adopted due to its robustness to outliers. While the Poisson likelihood leads naturally to a specific form of F , in practice, \ell_{2} norm fidelity is often preferred for its numerical stability. Our approach unifies various noise models within a general PnP framework, encompassing squared \ell_{2} loss as a special case. The main contributions of this papers are as follows: • We propose a generalized PnP framework using the primal-dual hybrid gradient (PDHG) method to solve ( 3 ) with non-Gaussian noise, by accommodating flexible fidelity terms F(\cdot) . • The proximal step for G(\cdot) is implemented using an implicit denoiser derived from flow matching generative models. • We seamlessly integrate flow matching generative models into the classical PDHG method, achieving an efficient and implementation-friendly solution. • Extensive experiments on two benchmark datasets across diverse tasks, including denoising, deblurring, inpainting, and super-resolution, demonstrate the superiority of our method under non-Gaussian noise, outperforming existing PnP methods designed specifically for Gaussian noise. Background The performance of imaging inverse problems has significantly improved with the advent of generative models that learn expressive image priors. Among these, diffusion models ( Song et al. 2020 ; Ho, Jain, and Abbeel 2020 ) and flow matching models ( Lipman et al. 2023 ; Tong et al. ) have attracted considerable attention due to their superior generative capabilities compared to earlier approaches. These models have demonstrated remarkable potential for image restoration when used as priors within optimization frameworks. In this section, we focus on flow matching generative models, emphasizing their simulation-free training and their application as denoisers. These denoisers play a critical role in the Plug-and-Play (PnP) approach for solving the regularized optimization problem in ( 3 ). Flow matching generative model Flow matching aims to learn a continuous transformation that maps samples from a simple distribution p(\bm{x}_{0}) ( e.g . , Gaussian) to samples from a data distribution p(\bm{x}_{1}) . Let \mathcal{P}(\mathbb{R}^{n}) denote the space of probability measures over \mathbb{R}^{n} , and consider two probability measures P_{0} and P_{1} representing the latent and data distributions, respectively. Define \Gamma(P_{0},P_{1}) as the set of couplings \pi\in\mathcal{P}(\mathbb{R}^{n}\times\mathbb{R}^{n}) with marginals P_{0} and P_{1} . Flow matching constructs a continuous probability path t\mapsto P_{t} for t\in[0,1] , interpolating between P_{0} and P_{1} . Given a coupling (\bm{x}_{0},\bm{x}_{1}) , one defines the interpolated point \bm{x}_{t}=(1-t)\bm{x}_{0}+t\bm{x}_{1} , which induces a trajectory governed by the ordinary differential equation (ODE): \frac{d\bm{x}_{t}}{dt}=\bm{v}(\bm{x}_{t},t), (4) where \bm{v}(\bm{x}_{t},t) is the velocity field. The goal is to learn this velocity field from data. To

Read the complete text