Deep End-to-End Posterior ENergy (DEEPEN) for image recovery
Jyothi Rikhab Chand, Member, IEEE, Mathews Jacob, Fellow, IEEE · 2025
Read Deep End-to-End Posterior ENergy (DEEPEN) for image recovery by Jyothi Rikhab Chand, Member, IEEE, Mathews Jacob, Fellow, IEEE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17244.txt.
Introduction
Jyothi Rikhab Chand and Mathews Jacob introduce DEEPEN, a framework for image recovery. Their 2025 paper learns the energy of the posterior distribution directly. Standard methods yield a single best-guess image but cannot sample alternatives. Diffusion models can sample yet resist end-to-end training.
DEEPEN trains the energy by maximum likelihood, end to end. The learned energy sums data mismatch and a prior term. No unrolled iterations are needed, so memory and compute stay small. Unlike plug-and-play methods, it needs no contraction limits. Tests show sharper best-guess images than rival methods and faster sampling than diffusion. The model also held up when imaging settings changed.
Catalog record
| CREATOR | Jyothi Rikhab Chand, Member, IEEE, Mathews Jacob, Fellow, IEEE |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep End-to-End Posterior ENergy (DEEPEN) for image recovery Jyothi Rikhab Chand, Member, IEEE, Mathews Jacob, Fellow, IEEE Abstract Current end-to-end (E2E) and plug-and-play (PnP) image reconstruction algorithms approximate the maximum a posteriori (MAP) estimate but cannot offer sampling from the posterior distribution, like diffusion models. By contrast, it is challenging for diffusion models to be trained in an E2E fashion. This paper introduces a Deep End-to-End Posterior ENergy (DEEPEN) framework, which enables MAP estimation as well as sampling. We learn the parameters of the posterior, which is the sum of the data consistency error and the negative log-prior distribution, using maximum likelihood optimization in an E2E fashion. The proposed approach does not require algorithm unrolling, and hence has a smaller computational and memory footprint than current E2E methods, while it does not require contraction constraints typically needed by current PnP methods. Our results demonstrate that DEEPEN offers improved performance than current E2E and PnP models in the MAP setting, while it also offers faster sampling compared to diffusion models. In addition, the learned energy-based model is observed to be more robust to changes in image acquisition settings. I Introduction Computational algorithms that can recover images from sparse and noisy Fourier measurements have revolutionized magnetic resonance (MR) imaging. Traditional compressed sensing (CS) algorithms rely on hand-crafted image priors as regularizers [ 1 , 2 , 3 ] ; wherein at each iteration they alternate between the update of data consistency (DC) and the proximal map of the regularizer to recover the unknown image. Data-driven deep learning methods have significantly improved reconstruction performance over classical methods. Plug-and-play (PnP) methods use an iterative algorithm that alternates between denoising and the DC update step, where the CNN-based pre-trained denoiser replaces the proximal map in the CS algorithm [ 4 , 5 , 6 ] . A contraction constraint is often needed to ensure fixed-point convergence [ 7 ] , which translates to reduced performance [ 8 ] . Recently, PnP methods that use explicit CNN-based energy functions to model the negative log-prior [ 9 , 10 , 11 ] and are trained using denoising score matching (DSM) [ 12 ] were introduced. These methods do not require a contraction constraint to guarantee stationary point convergence, translating into improved performance [ 9 , 10 , 11 ] . Energy models also enable the generation of samples from the target distribution. The implicit multi-scale energy model (i-MuSE), which pre-learns a single energy function using DSM at multiple noise scales, was found to offer improved convergence and hence superior performance compared to a single-scale DSM [ 11 ] . Unlike PnP models that pre-learn CNN, end-to-end (E2E) methods [ 13 , 14 , 15 , 16 , 17 ] unroll the iterative algorithm and optimize the parameters of the CNN denoiser so that the reconstructed image matches the reference image. These approaches offer better performance over PnP methods that are agnostic to the specific forward model. However, the E2E models are associated with increased memory demand, which can be reduced using the deep equilibrium (DEQ) formulation, but requires a contraction constraint similar to PnP methods that restricts its performance [ 8 , 18 , 19 ] . Since energy-based DEQ methods do not require this constraint [ 20 ] , they offer improved performance. The above approaches approximate the maximum a posteriori (MAP) estimate. In contrast, we learn the posterior distribution in an E2E fashion. Once trained, the learned posterior can be used to derive the MAP estimate and also sample from it. We derive the parameters of our Deep E2E Posterior ENergy (DEEPEN) network by maximizing the likelihood of data samples as in [ 21 ] . In particular, we minimize the negative log-posterior of the training samples, which is the sum of the negative log-likelihood and the negative log-prior modeled by the CNN. The above training strategy simplifies to the minimization of the energy of the true reference samples, while maximizing the energy of the fake samples that are obtained using the Langevin sampling algorithm using the gradient of the learned posterior. The training strategy thus resembles a generative adversarial network (GAN) [ 22 , 23 ] . The key difference is that the generator involves a Langevin-based iterative algorithm specified by the energy model, and the classifier is also specified by the energy model. We show that when the Langevin noise level is small, the above training strategy does not require algorithm unrolling or DEQ strategies associated with high memory demand and computational complexity. This E2E training strategy is computationally and memory-efficient because it does not require unrolling or fixed-point iterations. Moreover, unlike current E2E methods it does not use Mean Squared Error (MSE) loss to learn a MAP estimate. We note that the DSM approaches used in PnP models are trained to remove Gaussian noise perturbations, where the pixels of the perturbations are independent and identically distributed. However, the MR imaging inverse problem aims to recover images from corrupted undersampled Fourier measurements, where the corruption/perturbation often has highly correlated pixel values. Unfortunately, DSM-trained models are often not efficient in estimating and removing correlated noise. By contrast, our experiments show that DEEPEN gradients are more effective in estimating Gaussian as well as correlated perturbations, thus offering improved performance in MAP image recovery. Our experiments also show that the learned E2E approach also generalizes to unseen acquisition settings, unlike traditional E2E approaches that are not robust to mismatch in forward models from the training settings. The DEEPEN model can also enable sampling from the posterior, where it can offer 10x faster sampling than diffusion models [ 24 ] , with comparable image quality, and reduced uncertainty, even when CNN complexity is 5x smaller. We note that deterministic ODE flows [ 25 , 26 ] that learn straighter paths between probability distributions have been introduced to speed up prior sampling drastically. However, these models still require > 100 steps in the posterior sampling setting. In particular, since they are trained only along the path between the distribution, noise often needs to be added after each gradient descent step involving data-consistency to ensure that the integration path remains in the regions trained originally [ 26 ] . A preliminary conference version of this approach was previously published with limited emperical results [ 27 ] . II End-to-end learned energy model Let {\boldsymbol{b}}\in\mathbb{C}^{n} denote the noisy undersampled measurements from which we wish to recover the unknown image {\boldsymbol{x}}\in\mathbb{C}^{m} . The image and the measurements are related through a known linear forward operator {\mathbf{A}}\in\mathbb{C}^{n\times m} : {\boldsymbol{b}}={\mathbf{A}}{\boldsymbol{x}}+{\boldsymbol{n}} (1) where {\boldsymbol{n}}\in\mathcal{N}(0,\eta^{2}{\mathbf{I}}) is additive complex white Gaussian noise. The recovery of the image from the measurements is often formulated as a MAP estimation problem, where the solution is the maximum of the log-posterior: \log q({\boldsymbol{x}}|{\boldsymbol{b}})=\log p({\boldsymbol{b}}|{\boldsymbol{x}})+\log q({\boldsymbol{x}}) (2) Here, q({\boldsymbol{x}}) is the prior distribution, and the log-likelihood term \log p({\boldsymbol{b}}|{\boldsymbol{x}}) is specified by: \log p({\boldsymbol{b}}|{\boldsymbol{x}})=-\dfrac{\|{\mathbf{A}}{\boldsymbol{x}}-{\boldsymbol{b}}\|^{2}}{2\eta^{2}}+P (3) where P is the normalization constant. The log-posterior can also be used to derive samples from the posterior distribution, which can inform about uncertainties in the estimation and offer likely solutions. II-A Deep End-to-End Posterior ENergy (DEEPEN) We approximate the prior distribution q({\boldsymbol{x}}) by a parametric energy model p_{\boldsymbol{\theta}}({\boldsymbol{x}}) as in [ 21 ] :