F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal
Jie Cai, Kangning Yang, Ling Ouyang, Lan Fu, Jiaming Ding, Huiming Sun, Chiu Man Ho, Zibo Meng · 2025
Read F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal by Jie Cai, Kangning Yang, Ling Ouyang, Lan Fu, Jiaming Ding, Huiming Sun, Chiu Man Ho, Zibo Meng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05489.txt.
Introduction
Jie Cai and co-authors design a network that scrubs reflections out of a single photograph. Shots taken through glass mix the scene with glare and ghost images, and real reflections vary in strength, shape, and spread, which defeats one-size methods. Their design, F2T2-HiT, folds two transformer styles into a U-shaped frame. One kind works in the frequency domain, where broad reflection patterns stand apart from the scene. The other works at several scales at once, suiting reflections of many sizes.
On three public test sets, the combined system reached the best scores reported for the task. The authors credit the pairing: frequency blocks catch the global wash of a reflection, while the layered blocks chase its sharp local edges. They present the model as a general answer to a stubborn photo flaw.
Catalog record
| CREATOR | Jie Cai, Kangning Yang, Ling Ouyang, Lan Fu, Jiaming Ding, Huiming Sun, Chiu Man Ho, Zibo Meng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-05489
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal Jie Cai, Kangning Yang, Ling Ouyang, Lan Fu, Jiaming Ding, Huiming Sun, Chiu Man Ho, Zibo Meng Abstract S ingle I mage R eflection R emoval (SIRR) technique plays a crucial role in image processing by eliminating unwanted reflections from the background. These reflections, often caused by photographs taken through glass surfaces, can significantly degrade image quality. SIRR remains a challenging problem due to the complex and varied reflections encountered in real-world scenarios. These reflections vary significantly in intensity, shapes, light sources, sizes, and coverage areas across the image, posing challenges for most existing methods to effectively handle all cases. To address these challenges, this paper introduces a U-shaped F ast F ourier T ransform T ransformer and Hi erarchical T ransformer (F2T2-HiT) architecture, an innovative Transformer-based design for SIRR. Our approach uniquely combines Fast Fourier Transform (FFT) Transformer blocks and Hierarchical Transformer blocks within a UNet framework. The FFT Transformer blocks leverage the global frequency domain information to effectively capture and separate reflection patterns, while the Hierarchical Transformer blocks utilize multi-scale feature extraction to handle reflections of varying sizes and complexities. Extensive experiments conducted on three publicly available testing datasets demonstrate state-of-the-art performance, validating the effectiveness of our approach. † † address: OPPO AI Center 1 Introduction The SIRR technique aims to decompose a reflection-contaminated input image I into two distinct components: the reflection layer R and the reflection-free transmission layer T . The primary objective is to ensure that the generated transmission layer T closely approximates its corresponding ground truth B , which represents the ideal reflection-free image. This paper proposes a U-shaped network where the Fast Fourier Transform Transformer (F2T2) block and the Hierarchical Transformer (HiT) block serve as fundamental building blocks for generic reflection removal. F2T2-HiT is built upon the elegant architecture of NAFNet [ 1 ] , where the convolutional layers are replaced with Transformer blocks while retaining the overall hierarchical encoder-decoder structure and skip connections. Specifically, we propose two core designs to make F2T2-HiT suitable for reflection removal tasks. First, we note that a significant limitation of most current SIRR architectures is their limited effective receptive field. To tackle this issue, we introduce an FFT-based Transformer block, which acts as an efficient and effective foundational component for global feature extraction. This block allows the model to learn long-range dependencies and capture spatial relationships across the entire input. A large effective receptive field is critical for comprehending an image’s global structure, a key requirement for reflection removal. Moreover, for large reflection areas, even an expanded receptive field achieved through larger or dilated convolutions may still miss the information needed to produce a high-quality reflection-free image. To overcome this limitation, we leverage Fast Fourier Convolutions (FFCs) [ 2 ] , which provide a larger receptive field that spans the entire image even in the early layers of the network. This property of FFCs improves both perceptual quality and parameter efficiency of the network. Second, we note that most existing SIRR architectures struggle to effectively capture both local details and global context across multiple scales. To tackle this, we introduce a hierarchical window-partition Transformer block, which processes images through multi-scale window partitions. This design enables the block to capture fine-grained local details while maintaining global context by dividing the input into varying window sizes. By integrating multi-scale attention mechanisms, our method achieves a more precise understanding of reflection-transmission relationships while preserving computational efficiency through localized attention. Crucially, this design delivers superior reflection removal performance without significant computational cost. In summary, our key contributions include: • We propose a novel U-shaped Transformer-based architecture designed for efficient and effective SIRR. • We introduce a novel FFT Transformer block to replace standard self-attention. In this block, the spatial-domain branch extracts local features, while the frequency-domain branch captures global relationships. • We present a simple yet effective HiT block, transforming popular Transformer-based methods into hierarchical Transformers. This improves SIRR performance by leveraging multi-scale features and long-range dependencies. 2 Related Work Over recent decades, numerous innovative methods have been proposed to address the problem of reflection removal. Some methods rely on additional inputs, such as multi-frames [ 3 ] , polarization [ 4 ] , and flash-only priors [ 5 ] . Others utilize traditional image priors, including relative smoothness, “ghosting” cues, and dark channels, to separate and remove reflection artifacts. In contrast, our research is primarily dedicated to exploring and advancing deep learning techniques for effectively removing reflections from single images. With the advancement of deep learning, learning-based SIRR methods [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ] have achieved remarkable performance gains across diverse reflection scenarios, becoming the leading solutions in this field. This paper [ 11 ] presents a LANet for SIRR. It employs a reflection detection module that generates a probabilistic confidence map using multi-scale Laplacian features. The network, designed as a recurrent model, progressively refines reflection removal, with Laplacian kernel parameters highlighting strong reflection boundaries to improve detection and enhance the quality of the results. V-DESIRR [ 17 ] introduces a lightweight model for reflection removal using an innovative scale-space architecture, which processes the corrupted image in two stages: a Low Scale Sub-network (LSSNet) for the lowest scale and a Progressive Inference (PI) stage for higher scales. To minimize computational complexity, the PI stage sub-networks are significantly shallower than LSSNet, and weight sharing across scales enables the model to generalize to high resolutions without the need for retraining. YTMT [ 18 ] introduces a simple yet effective interactive strategy called ”Your Trash is My Treasure”. This approach constructs dual-stream decomposition networks by facilitating block-wise communication between the streams and transferring deactivated ReLU information from one stream to the other, leveraging the additive property of the components. RobustSIRR [ 12 ] introduces a transformer-based model incorporating cross-scale attention, multi-scale fusion, and adversarial training for enhanced SIRR. RAG [ 13 ] refines transmission layer estimation by leveraging predicted reflections. DSRNet [ 14 ] consists of two cascaded stages and a learnable residue module (LRM), where the first stage extracts hierarchical semantics and the second stage refines decomposition by addressing the linearity assumption. The framework in [ 15 ] comprises RDNet and RRNet, with RDNet employing a pretrained backbone and interpolation for reflection mask estimation, and RRNet utilizing this estimate for reflection removal. A language-guided approach [ 16 ] aligns textual descriptions with image layers through cross-attention and contrastive learning, aided by a gated network and randomized training to mitigate layer ambiguity. Compared to existing methods, our model exhibits superior generalization and robustness, effectively handling diverse reflection cases while maintaining high computational efficiency. 3 Methodology Figure 1: An illustration of our U-shaped F2T2-HiT architecture. Certain details have been deliberately omitted for simplicity, such as downsampling/upsampling layers, feature fusion modules, input/output shortcuts, and more. Both (c) the HiT block and (d) the F2T2 block follow the structure of LayerNorm – Self-Attention (SA) – LayerNorm – Feed-Forward Network (FFN). Best viewed in color. We first introduce the fundamental methodology of the F2T2-HiT architecture in Sec. 3.1 , followed by the block-level designs detailed in Sec. 3.2