UNet-AF: An alias-free UNet for image restoration

Jérémy Scanvic 1,2 Quentin Barthélemy 2 Julián Tachella 1 [email protected] [email protected] [email protected] · 2026

Read UNet-AF: An alias-free UNet for image restoration by Jérémy Scanvic 1,2 Quentin Barthélemy 2 Julián Tachella 1 [email protected] [email protected] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11323.txt.

Introduction

Jérémy Scanvic, Quentin Barthélemy, and Julián Tachella rebuild the UNet image design to be equivariant to translations, which its aliasing-prone layers work against. They choose every main part in alias-free form: convolution, pooling, downsampling, activation, and normalization. This yields sub-pixel translation equivariance. Against non-equivariant baselines, the alias-free networks match them on image restoration and score far higher on equivariance. Tests that remove one choice at a time show every choice matters.

Catalog record

CREATORJérémy Scanvic 1,2 Quentin Barthélemy 2 Julián Tachella 1 [email protected] [email protected] [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21128
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11323

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UNet-AF: An alias-free UNet for image restoration Jérémy Scanvic 1,2 Quentin Barthélemy 2 Julián Tachella 1 [email protected] [email protected] [email protected] Abstract The simplicity and effectiveness of the UNet architecture makes it ubiquitous in image restoration, image segmentation, and diffusion models. They are often assumed to be equivariant to translations, yet they traditionally consist of layers that are known to be prone to aliasing, which hinders their equivariance in practice. To overcome this limitation, we propose a new alias-free UNet designed from a careful selection of state-of-the-art translation-equivariant layers. We evaluate the proposed equivariant architecture against non-equivariant baselines on image restoration tasks and observe competitive performance with a significant increase in measured equivariance. Through extensive ablation studies, we also demonstrate that each change is crucial for its empirical equivariance 1 1 1 Our implementation is available at https://github.com/jscanvic/UNet-AF . I Introduction UNet networks are encoder-decoder neural networks that operate across multiple scales using pairs of pooling/downsampling and upsampling layers, and have concatenation layers bridging the encoder and decoder at each scale. Notably, they are commonly applied in medical imaging [ 1 , 2 ] , in image super-resolution [ 3 ] and in diffusion models [ 4 ] . In many applications, UNet networks are expected to be equivariant to translations, i.e., that their output translates with their input. To do so, they typically rely on the building blocks of convolutional neural networks (CNN) [ 5 ] . However, recent work shows that standard CNNs are only approximately equivariant to translations, mostly due to aliasing in pooling and upsampling layers [ 6 , 7 , 8 ] . They propose to combine pooling and upsampling layers with anti-aliasing filters to reduce aliasing, and thus increase approximate equivariance. [ 9 ] identify activation functions as another source of aliasing, which hinders translation-equivariance. They introduce adaptive pooling and upsampling whereby the sampling grid shifts with the input, thereby recovering perfect equivariance to whole-pixel translations. However, this solution does not improve equivariance to sub-pixel (continuous) translations [ 10 ] . Fig. 1: Equivariance and robustness to circular translations. (Top) Performance on an horizontally translated image, with 0.01 px steps, (bottom) adversarial performance on the worst translation up to a maximum displacement, with 0.25 px steps. UNet-AF is stable and robust, unlike the baselines. [ 11 ] emphasize the importance of sub-pixel equivariance for image generation to avoid texture sticking and reframe the problem using a traditional approach built upon sampling theory. Discrete finite images are viewed as representations of bandlimited periodic images related through sampling and Whittaker-Shannon (sinc) interpolation [ 12 ] . Instead of using adaptive sampling, they reduce the aliasing caused by activation functions by low-pass filtering them. They also advocate for the use of anti-aliasing filters with a sharper cut-off, and a flatter response in the pass and stop bands, although more computationally expensive. [ 10 ] go one step further and use exact sinc filters implemented using the fast Fourier transform (FFT). They also identify the lack of smoothness of ReLU as a cause of aliasing and substitute it for polynomial activations, and propose an alias-free substitute to layer normalization. Fig. 2: Overview of the architecture. UNet-AF is obtained by changing the architectures of [ 1 ] and [ 2 ] . Anti-aliasing filters are added in upsampling layers, max pooling layers are replaced with blur pooling layers, ReLU activations are replaced with filtered GELU layers and batch normalization is replaced by alias-free layer normalization. The padding mode in convolutions is left unconstrained and the residual connection is optional. Despite having been applied successfully to classifiers (encoder type) [ 10 , 13 ] , and image generators (decoder type) [ 11 ] , layers equivariant to continuous sub-pixel translations remain, to the best of our knowledge, to be applied to encoder-decoder architectures including UNet architectures. Our contributions are the following: • We propose a new alias-free UNet architecture (UNet-AF) using state-of-the-art translation-equivariant layers. • We evaluate its effectiveness, training stability, computational efficiency, and empirical equivariance on multiple image restoration tasks. • We conduct extensive ablation studies to validate its design, and identify its tradeoffs. II Alias-free UNet Our proposed architecture is based on that of [ 2 ] (UNet-Jin) which features an additional residual connection on top of the architecture of [ 1 ] (UNet-Ronneberger). This base architecture is not translation-equivariant, and we apply a series of modifications to render it equivariant. While we focus on this specific UNet, our analysis can be easily applied to other UNet backbones. In this section, we discuss every layer type used in the base architecture, i.e., convolutions, residual connections, concatenation layers, pooling layers, upsampling layers, activation functions, and normalization layers, presenting the necessary changes to obtain a translation-equivariant counterpart. Formally, a discrete image x\in\mathbb{R}^{n} is interpolated as \varphi*x where \varphi is the continuous sinc function and * denotes circular convolution [ 12 , 11 ] . Sampling the convolution of a translated version of \varphi with x on the original grid yields the translation of x denoted as \mathop{T_{g}}x where g\in\mathbb{R}^{2} is the displacement. In this setting, translation-equivariance is expressed as f(\mathop{T_{g}}x)=\mathop{T_{g}}f(x),\quad\forall x\in\mathbb{R}^{n},\ \forall g\in\mathbb{R}^{2}, (1) where f denotes the UNet. Convolutions While convolutions are not prone to aliasing, a mismatch in padding mode with the translations causes boundary errors, and thus a loss of equivariance. Therefore, we use circular convolutions for circular translations, and zero-padded convolutions for crop-translations [ 7 ] . Residual connections The original architecture features a global residual connection, and while it does not introduce aliasing, we find that it is not always beneficial to performance and training stability. It is made optional in UNet-AF, and it is enabled when it is measurably beneficial to performance. Concatenation layers Intermediate concatenation layers are present in the base architecture, sometimes referred to as skip connections. These do not introduce aliasing, and they remain present in the UNet-AF architecture. Pooling layers We substitute max-pooling for blur pooling with periodic sinc anti-aliasing implemented using the FFT, similarly to [ 10 ] . Due to the contribution of every input pixel to each output pixel, this anti-aliasing scheme is more computationally costly than the filters used by [ 7 , 9 , 11 ] , including the box filter used in average pooling, and windowed sinc filtering. Instead, it eliminates aliasing within numerical precision, which results in higher equivariance. We use this anti-aliasing consistently in other layers. Upsampling layers The original upsampling scheme consists in merely inserting zeros between input pixels, which preserves input information in the output low-frequencies, yet fills the high-frequencies with aliasing. It is eliminated in the proposed architecture using an anti-aliasing filter. Activation functions We replace ReLU activations with filtered activation functions [ 11 , 10 ] , which consist in first resampling the input on a finer grid, applying a base activation function, removing the high-frequencies, and resampling on the coarse grid. Instead of corrupting the output, part of the aliasing is removed with the high-frequencies, thereby improving equivariance. The upsampling rate is set to two, following previous work. The GELU function is chosen as the underlying base activation function, similarly to [ 13 ] . Its smoothness limits the addition of high-frequencies in the finest grid, which improves equivariance,

Read the complete text