i-DEQ: A stable inertial deep equilibrium model for image restoration
Antonin Clerc, Marien Renaud, Baudouin Denis De Seneville, Nicolas Papadakis · 2026
Read i-DEQ: A stable inertial deep equilibrium model for image restoration by Antonin Clerc, Marien Renaud, Baudouin Denis De Seneville, Nicolas Papadakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19705.txt.
Introduction
Clerc and team propose i-DEQ, a stable inertial deep model for image restoration. Deep equilibrium models learn a regularization by solving a fixed-point, yet suffer high cost and training instability. Their inertial variant learns a nonconvex regularization, and momentum inside fixed-point iterations brings convergence guarantees and faster rates. It also proves more stable during training and robust to rough initialization than standard models. Experiments on linear and nonlinear inverse problems show reconstruction quality matching state-of-the-art methods while halving inference time.
Catalog record
| CREATOR | Antonin Clerc, Marien Renaud, Baudouin Denis De Seneville, Nicolas Papadakis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67480 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19705
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
i-DEQ: A stable inertial deep equilibrium model for image restoration Antonin Clerc, Marien Renaud, Baudouin Denis De Seneville, Nicolas Papadakis Abstract Deep Equilibrium Models (DEQs) are an established framework for image restoration that learn a problem-adapted regularization by solving a fixed-point (i.e. equilibrium) problem. While flexible and expressive, DEQs are often hindered by high computational cost and training instability. We propose an inertial DEQ (i-DEQ) that learns an explicit nonconvex regularization within the DEQ formulation. By using momentum within the fixed-point iterations, i-DEQ has convergence guarantees and accelerated rates. Moreover, we observe that i-DEQ is significantly more stable during the training and robust to rough initialization than DEQs. Numerical experiments on various linear and nonlinear inverse problems demonstrate that i-DEQ achieves reconstruction quality comparable to state-of-the-art methods, while reducing DEQ’s inference time by a factor of two. 1 Introduction Inverse problems in imaging aim to recover an unknown image x\in\mathbb{R}^{d} from noisy and degraded observations y\in\mathbb{R}^{m} , modeled as y=A(x)+n , where A:\mathbb{R}^{d}\to\mathbb{R}^{m} is a known forward operator and n is typically an additive Gaussian noise. These problems, that arise in many applications from computer vision to biomedical imaging [ 9 , 98 ] , are generally ill-posed [ 46 ] , meaning that the observations do not uniquely determine the unknown image. To address this issue, model-based approaches incorporate prior knowledge through regularization [ 32 ] . A standard formulation is Maximum a Posteriori (MAP) estimation, which seeks \hat{x}\in\arg\min_{x\in\mathbb{R}^{d}}F(x):=f(x,y)+\lambda g(x), (1) where f measures the fidelity to the data y and g encodes prior information on the image to recover. Many explicit priors have been proposed, including sparsity in transform domains [ 29 , 69 ] , total variation [ 88 ] , low-rank models [ 13 ] , and non-local self-similarity [ 12 ] . Leveraging the expressiveness of deep neural networks, Model-Based Deep Learning (MBDL) has emerged, in which priors are no longer hand-crafted but learned from data [ 75 ] . Several classes of MBDL methods have been proposed, see a recent review in [ 51 ] . In particular, plug-and-play (PnP) [ 102 ] approaches integrate a denoiser within iterative optimization algorithms. However, since the regularization imposed by the denoiser is implicit, establishing convergence guarantees becomes challenging and requires strong assumptions on the underlying operators [ 23 ] . Therefore, without adequate safeguards, the execution of a PnP algorithm can result in divergence throughout the restoration procedure [ 77 , 84 , 92 , 100 , 111 ] . In contrast, PnP methods relying on explicit regularization define a well-defined optimization problem [ 53 , 87 ] , thereby facilitating theoretical analysis and convergence guarantees. Nevertheless, MBDL approaches typically require a large number of iterations to converge, and their performance depends on careful parameter tuning. Rather than explicitly designing a regularization functional, End-to-End (E2E) deep learning learns a direct mapping from measurements to images, in a purely data-driven manner [ 56 ] . These methods achieve state-of-the-art performance in many imaging tasks [ 99 ] . While highly effective, they can disregard the underlying acquisition model and do not explicitly enforce data-consistency, which may limit robustness and generalization [ 4 ] . Unfolded approaches map a finite number of iterations of an optimization algorithm onto a neural network trained end-to-end for a specific forward model [ 16 , 43 , 75 ] . This leads to improved reconstruction performance by incorporating domain knowledge into the architecture [ 75 ] . However, these methods suffer from high memory consumption during training, as backpropagation requires storing intermediate states across the unrolled iterations, leading to a cost that increases with the number of layers. As a result, this memory cost limits the depth to a small number of iterations, typically fewer than 10, and restricts the scalability. In addition, they typically lack an explicit optimization objective, making theoretical analysis more involved. Deep Equilibrium (DEQ) [ 6 , 41 ] models bridge the gap between E2E approaches and MBDL ones. They inherit their optimization structure from MBDL, but unlike PnP methods, which rely on off-the-shelf denoisers, DEQs learn the underlying operator directly from the data of the tackled problem. This yields problem-adapted models that can match or even surpass E2E approaches [ 31 , 63 , 96 , 101 , 109 ] , while still preserving a variational interpretation, in contrast to standard E2E [ 25 , 115 ] . Moreover, DEQs are more memory-efficient to train than unfolded methods, as they rely on implicit differentiation and avoid backpropagation through all intermediate iterations [ 6 , 34 ] . However, computing the solution requires solving a fixed-point problem, which remains computationally demanding despite the use of acceleration techniques or black-box fixed-point solvers [ 7 , 62 , 106 ] . In addition, training DEQ models is often prone to instability, and designing effective strategies to stabilize the training process remains challenging [ 3 , 8 , 10 , 27 , 38 , 72 ] . Contributions In this work, we introduce an accelerated tuning-free DEQ-based method that stabilizes the training by incorporating restarted inertia in the optimization iterations. • By relying on an explicit regularization formulation derived from PnP, the method enjoys well-definedness guarantees, including existence of fixed-points and convergence of the iterative scheme under standard assumptions (see Proposition 3.1 ). • We leverage inertial dynamics within the fixed-point iterations to accelerate computation of the fixed-point during training, which empirically leads to improved training stability compared to classical DEQ formulations (see Figure 1 left). • The use of inertial acceleration scheme also significantly reduce the number of iterations required to reach fixed-point equilibrium at inference, and enable faster reconstructions without loss of performance (see Figure 1 right). Figure 1: Training on 100 images and inference on a test set of 20 images for DEQ-based methods on a subset of FastMRI [ 57 ] . Left: impact of the number of fixed-point iterations in the forward pass. Standard DEQ becomes unstable for both 100 and 200 iterations, with earlier divergence at 100, whereas i-DEQ remains stable and shows a steady PSNR improvement comparable to DEQ prior to divergence. Right: PSNR versus inference time for reaching the stopping criterion. i-DEQ, trained with 100 and 200 fixed-point iterations, is compared to a classical DEQ with matching iteration budgets and to RISP [ 85 ] ; it achieves similar PSNR 2–5 \times faster. Outline of the paper In Section 2, we introduce some useful tool and survey related approaches in data-driven reconstruction methods. In Section 3, we present the proposed inertial Deep Equilibrium (i-DEQ) and provide its theoretical accelerate rate. In Section 4, we validate our method on inverse problems including compressed sensing MRI, inpainting and non-linear Rician denoising. In particular, we show competitive performances with state-of-the-art approaches, while exhibiting improved stability during training. In addition, we demonstrate faster restoration time compared to PnP and standard DEQs. Code, data, and network weights are provided to the reviewers and will be made publicly available upon publication. Finally, conclusions are drawn in Section 5. 2 Relationship to Prior Work In this section, we review the main approaches related to our work. We first recall classical first-order optimization methods. We then discuss plug-and-play frameworks, which incorporate