Deep Image Prior with L0 Gradient Regularizer for Image Smoothing
Nhat Thanh Tran, Kevin Bui, Jack Xin · 2026
Read Deep Image Prior with L0 Gradient Regularizer for Image Smoothing by Nhat Thanh Tran, Kevin Bui, Jack Xin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13400.txt.
Introduction
Nhat Thanh Tran, Kevin Bui, and Jack Xin smooth pics with a deep prior plus a sharp edge guard. Pic smoothing keeps big forms, such as strong edges and rims, and wipes fine grain and texture. Old tools use local stats or solve an opt task. New deep tools score well but need a hand picked train set.
Their tool needs no train set. It uses a deep image prior, the bias of a net's own plan toward true pic forms. It adds an L0 gradient guard, a term that counts edge jumps and keeps the count low. That guard holds strong edges while grain is wiped. For pic teams, a train free smoother that keeps edges is a handy base tool.
Catalog record
| CREATOR | Nhat Thanh Tran, Kevin Bui, Jack Xin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13400
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Image Prior with L0 Gradient Regularizer for Image Smoothing Nhat Thanh Tran, Kevin Bui, Jack Xin Abstract Image smoothing is a fundamental image processing operation that preserves the underlying structure, such as strong edges and contours, and removes minor details and textures in an image. Many image smoothing algorithms rely on computing local window statistics or solving an optimization problem. Recent state-of-the-art methods leverage deep learning, but they require a carefully curated training dataset. Because constructing a proper training dataset for image smoothing is challenging, we propose DIP- ℓ 0 \ell_{0} , a deep image prior framework that incorporates the ℓ 0 \ell_{0} gradient regularizer. This framework can perform high-quality image smoothing without any training data. To properly minimize the associated loss function that has the nonconvex, nonsmooth ℓ 0 \ell_{0} “norm”, we develop an alternating direction method of multipliers algorithm that utilizes an off-the-shelf ℓ 0 \ell_{0} gradient minimization solver. Numerical experiments demonstrate that the proposed DIP- ℓ 0 \ell_{0} outperforms many image smoothing algorithms in edge-preserving image smoothing and JPEG artifact removal. † † address: Department of Mathematics; University of California, Irvine; Irvine, CA 92697, United States 1 Introduction As one of the most fundamental operations in computer vision and image processing, image smoothing decomposes an image into the base layer and the detail layer. Being piecewise smooth, the base layer contains the image’s structural information, such as edges and contours; being sparse, the detail layer contains the image’s residual information, such as details and textures. This image decomposition has several applications, including clip-art compression artifacts removal [ 17 , 25 ] and texture removal [ 3 , 27 ] . Image smoothing algorithms are classified into three main groups: local filters, global filters, and deep filters. Local filters compute the weighted average of a pixel and its associated neighbors within a window to smooth the input image. Examples include bilateral filter [ 22 ] and guided image filter [ 6 ] . Although local filters are computationally efficient, they are prone to creating artifacts, such as halos and gradient reversals, because they rely on computing local image statistics. Global filters, on the other hand, solve an optimization problem whose objective function typically consists of a fidelity term for approximating the original image and a regularizer for smoothing the original image. Global filters include weighted least square (WLS) [ 4 , 16 ] , \ell_{0} smoothing [ 17 , 25 , 18 ] , relative total variation [ 27 , 11 ] , and bilateral total variation [ 7 ] . Unlike local filters, global filters avoid creating halos and gradient reversals, but they are computationally expensive. Lastly, deep filters are deep neural networks (DNNs) that learn to emulate either local or global filters. They include deep edge-aware filters [ 26 ] and JESS-Net [ 5 ] . Although deep filters demonstrate outstanding performance, they need to be trained on a large dataset consisting of the original, natural images as inputs and their smoothed counterparts as targets so that they can be applicable for any kind of images. Typically, the smoothed counterparts are created by other smoothing methods, thereby limiting deep filters’ potential to generate better smoothing results. As of now, the NKS dataset [ 24 ] and the SPS dataset [ 5 ] are the only datasets that have the smoothed images as the original, natural images and the “noisy” images generated from the smoothed images via texture blending. Hence, training datasets for supervised deep filters are currently limited. To circumvent the lack of high-quality training data for image smoothing, we propose an unsupervised deep filter based on deep image prior (DIP) [ 23 ] . DIP has demonstrated remarkable performance without any training data in image denoising, image inpainting, and image superresolution by solving them as optimization problems. Since image smoothing can be formulated as an optimization problem as done by global filters, we demonstrate that it can be achieved by DIP when incorporating the \ell_{0} gradient regularizer. The \ell_{0} gradient regularizer preserves strong edges and boundaries in images, but any general \ell_{0} minimization problem is difficult to solve because of its nonconvexity and nonsmoothness. Therefore, this work not only proposes a DIP framework with \ell_{0} gradient regularizer specifically for image smoothing but also designs a specialized optimization algorithm for it. The main contributions of our paper are summarized as follows. First, we propose a DIP framework for image smoothing by incorporating the \ell_{0} gradient regularizer. As a result, it is an unsupervised deep filter that resembles a global filter because it solves an optimization problem. Second, instead of using the subgradient method as done in [ 23 , 13 ] , we minimize the loss function of DIP with alternating direction method of multipliers (ADMM) [ 1 ] that utilizes an approximation algorithm proposed in [ 17 ] for \ell_{0} gradient minimization. Third, in a series of experiments, we demonstrate that our proposed method performs well against other state-of-the-art algorithms in image smoothing and JPEG artifact removal. 2 Review of Deep Image Prior Utilizing the superior performance of DNNs without any training data for solving classical imaging problems such as image denoising, DIP [ 23 ] solves the optimization problem \displaystyle\min_{\theta}\|f-g_{\theta}(x)\|_{2}^{2}, (1) where f is the input image, x is a fixed but random input, and g_{\theta}(\cdot) is a DNN parameterized by the weights \theta . With \theta^{*} as the optimal solution to ( 1 ), the output of the DNN g_{\theta^{*}}(x) is the restored image for the given image processing task. ( 1 ) can be further improved by adding an explicit regularizer: \displaystyle\min_{\theta}\|f-g_{\theta}(x)\|_{2}^{2}+\lambda\mathcal{R}(g_{\theta}(x)). (2) Using total variation (TV) [ 20 ] as a regularizer improves DIP’s performance [ 13 ] . Usually, ( 1 ) and ( 2 ) are solved entirely by stochastic gradient descent (SGD) via backpropagation [ 23 , 13 ] . However, when the regularizer \mathcal{R}(\cdot) is non-smooth, ( 2 ) is solved by ADMM, as done in [ 2 ] . To our knowledge, DIP has not yet been applied to image smoothing. Therefore, in this paper, we demonstrate its extension to image smoothing by specifying the appropriate regularizer in ( 2 ) and tailoring an ADMM algorithm for it. 3 Proposed Method Setting \mathcal{R}(\cdot) in ( 2 ) as the \ell_{0} gradient regularizer, we propose DIP- \ell_{0} for image smoothing: \displaystyle\min_{\theta}\|f-g_{\theta}(x)\|_{2}^{2}+\lambda\|\nabla g_{\theta}(x)\|_{0}. (3) The \ell_{0} gradient regularizer \|\nabla u\|_{0} counts the number of non-zero gradients in an image u [ 25 ] . It enforces the output image g_{\theta}(x) to be piecewise constant, which helps preserve corners and sharp edges. Algorithm 1 ADMM Algorithm for DIP- \ell_{0} 0: • f\in[0,1]^{3} - normalized input image • g_{\theta}(x) - neural network architecture with fixed input x • \lambda>0 - regularizer parameter • \beta>0 - penalty parameter • T - number of ADMM iterations • K - number of SGD iterations for solving ( 6a ) • \alpha - learning rate for SGD • \gamma - weighing parameter for ( 7 ) 0: Smoothed image u^{T} 1: Initialize v^{0},w^{0} as random uniform [0,1] tensors with the same dimension as the input image f 2: Initialize u^{0}=f 3: Initialize \theta_{K}^{0} randomly as weights for the neural network g_{\theta} 4: for