Beyond sparse denoising in frames: minimax estimation with a scattering transform

Nathanaël Cuvelle–Magar, Stéphane Mallat · 2025

Read Beyond sparse denoising in frames: minimax estimation with a scattering transform by Nathanaël Cuvelle–Magar, Stéphane Mallat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19612.txt.

Introduction

Nathanaël Cuvelle-Magar and Stéphane Mallat build a denoising method for images that goes beyond the usual sparse-representation playbook. Classic estimators clean noisy signals by thresholding frame coefficients, but those representations miss some geometric regularities. For cartoon-like images with smooth edges, wavelet and curvelet frames fall short when the edge smoothness is unknown.

Their estimator works with wavelet scattering coefficients, quantities borrowed from simplified neural network models. By jointly shrinking and boosting different subsets of these coefficients, it separates noise from several kinds of geometric structure. Experiments reach the best possible asymptotic error for all edge smoothness levels tested. The authors state the result as a conjecture and frame the method as a bridge between harmonic analysis and deep network denoising.

Catalog record

CREATORNathanaël Cuvelle–Magar, Stéphane Mallat
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE115 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19612

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond sparse denoising in frames: minimax estimation with a scattering transform Nathanaël Cuvelle–Magar, Stéphane Mallat Abstract A considerable amount of research in harmonic analysis has been devoted to non-linear estimators of signals contaminated by additive Gaussian noise. They are implemented by thresholding coefficients in a frame, which provide a sparse signal representation, or by minimising their ℓ 1 \ell^{1} norm. However, sparse estimators in frames are not sufficiently rich to adapt to complex signal regularities. For cartoon images whose edges are piecewise 𝐂 α {\bf C}^{\alpha} curves, wavelet, curvelet and Xlet frames are suboptimal if the Lipschitz exponent α ≤ 2 \alpha\leq 2 is an unknown parameter. Deep convolutional neural networks have recently obtained much better numerical results, which reach the minimax asymptotic bounds for all α \alpha . Wavelet scattering coefficients have been introduced as simplified convolutional neural network models. They are computed by transforming the modulus of wavelet coefficients with a second wavelet transform. We introduce a denoising estimator by jointly minimising and maximising the ℓ 1 \ell^{1} norms of different subsets of scattering coefficients. We prove that these ℓ 1 \ell^{1} norms capture different types of geometric image regularity. Numerical experiments show that this denoising estimator reaches the minimax asymptotic bound for cartoon images for all Lipschitz exponents α ≤ 2 \alpha\leq 2 . We state this numerical result as a mathematical conjecture. It provides a different harmonic analysis approach to suppress noise from signals, and to specify the geometric regularity of functions. It also opens a mathematical bridge between harmonic analysis and denoising estimators with deep convolutional network. K eywords Sparse denoising \cdot minimax estimation \cdot geometric regularity \cdot wavelets 1 Introduction Signals may be contaminated by additive Gaussian white noise during measurements or transmission. Such noise removal has thus been a central signal processing topic since the early work of Wiener on optimal linear denoisers [ 56 ] . Donoho and Johnstone [ 14 ] have proved that nearly optimal denoising estimators are obtained over signal classes which have sparse decomposition coefficients over a basis or a frame. The denoising estimator is computed by thresholding the noisy signal coefficients in this frame, or by minimising their \ell^{1} norm as explained in Section 2.1 . In wavelet bases, it defines nearly minimax estimators over balls of Besov spaces [ 15 ] , which correspond to functions having specific local isotropic regularity properties. Section 2.2 gives a brief review. For images having anisotropic directional regularities, a considerable effort has been devoted to build frames such as curvelets or shearlets, which define sparse representations. However, bases and frames have a limited flexibility to adapt to various geometric regularities. A major issue in harmonic analysis addressed in this paper is to develop new mathematical tools to capture sparsity properties and improve these estimators. The paper gives a mathematical conjecture sustained by numerical results and partial mathematical justifications to do so, after reviewing important harmonic analysis results. Images with edges having some geometric regularity are important examples introduced by Tsybakov and Korostelev [ 30 ] . We shall consider an image model [ 45 ] of piecewise {\bf C}^{\alpha} (uniformly Lipschitz \alpha ) geometric images, which are {\bf C}^{\alpha} over disjoint image domains separated by regular edges. These edges are themselves regular {\bf C}^{\alpha} curves. Edge junctions define corners which are geometric singularities. Sparse representations are obtained with curvelets and shearlets if \alpha=2 and if there is no corner, as explained in Section 2.3 . However, there is no frame providing an optimally sparse representation for such images having corners, for a Lipschitz exponent \alpha\leq 2 which may vary. Indeed, it would require to adapt the geometry of frame vectors to the geometric regularity of edges in the image. If the edge regularity varies then we must choose the decomposition vectors in dictionaries of functions which are much larger than a frame. Optimally sparse representations have been obtained by selecting an orthonormal basis in a dictionary of bandlets [ 45 , 18 ] , whose geometry is adapted to each noisy image in terms of location, orientation and degree of regularity of edges. However, this approach becomes progressively too complex as the signal diversity increases, because the dictionary size must also increase. More recently, Gaussian noise removal has gained considerable attention for AI data generation with score diffusion. New data are generated by transporting a Gaussian white noise with a stochastic differential equation which performs a progressive denoising, with a drift term computed from a denoiser implemented with a deep neural network [ 51 , 24 , 27 ] . Denoising capabilities of deep convolutional neural networks go well beyond the performance of previous estimators based on wavelets, curvelets, Xlets and bandlets but their mathematical properties are not understood. A deep convolutional network computes a cascade of convolutional operators which have a narrow square support, and pointwise non-linearities. Simplified wavelet scattering models have been introduced in [ 41 , 2 ] by replacing these convolutional operators with wavelet transforms and a modulus non-linearity. As in deep convolutional networks, all wavelet filters have a square support, as explained in Section 3.1 . The main theorem of the paper in Section 3.2 proves that applying a second wavelet transform over the modulus of a first wavelet transform can still take advantage of the anisotropic geometric regularity of edges. This regularity appears through the fast decay of the \ell^{1} norm of a subset of scattering coefficients at fine scales. More surprisingly, the concentration of large wavelet coefficients along edge curves is also captured by the \ell^{1} norms of another subset of scattering coefficients, which become large and not small. Geometric image regularities are thus captured by both small and large \ell^{1} norms of scattering coefficients, which only involve wavelet filters. A denoising algorithm is implemented by minimising a weighted average of these scattering \ell^{1} norms with positive and negative weights to insure a joint minimisation and maximisation of these norms. Numerical simulations in Section 3.3 demonstrate that this estimator reaches the minimax asymptotic rate over images having {\bf C}^{\alpha} edges for all values of \alpha\leq 2 . All experiments are reproducible with the software available on https://github.com/ScatteringDenoisingCode/SD . Based on these numerical results, we state a mathematical conjecture on the minimax denoising rate of such denoising algorithm, which is not proved. This conjecture gives a new characterization of sparsity properties through minimisation and maximisation of different \ell^{1} norms. It does not require to use large size dictionaries with vectors whose support are adapted to the image geometry. It is a mathematical bridge towards deep convolutional network estimators. 2 Denoising geometrically regular images This first section reviews the minimax optimality of denoising estimators, which relies on sparsity properties in harmonic analysis bases or frames. Section 2.1 reviews minimax estimations with regularised estimators promoting sparsity. Section 2.2 reviews sparse estimations in wavelet bases, which are suboptimal to remove noise from images having some form of geometric regularity. Section 2.3 reviews results obtained with curvelets and shearlets frames, bandlet dictionaries and deep convolutional neural networks. 2.1 Variational and sparse denoising estimations Suppose that f\in{\bf L}^{2}[0,1]^{2} is contaminated by the addition of a Wiener measure of variance \sigma^{2} (Gaussian white noise): g=f+dB , where B is a

Read the complete text