Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba

Liwen Pan, Longguang Wang, Guangwei Gao, Jun Wang, Jun Shi, Juncheng Li · 2025

Read Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba by Liwen Pan, Longguang Wang, Guangwei Gao, Jun Wang, Jun Shi, Juncheng Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03852.txt.

Introduction

Rain, snow, and fog degrade the traffic camera images that intelligent transport systems depend on. Liwen Pan and colleagues attack that restoration problem in a 2025 computer vision preprint. Most existing methods model only the spatial layout of pixels, they note, while ignoring frequency information. Their framework, Frequency-Aware Mamba, brings frequency guidance into a sequence model built for long-range dependencies.

Two components do the heavy lifting. A dual-branch extraction block scans the image adaptively in two-dimensional frequency space, adjusting its path to the texture of each sub-band. A prior-guided block then rebuilds fine detail by learning from wavelet-based high-frequency residuals. An adaptive scanning mechanism lets the Mamba architecture sweep different frequency subgraphs in turn. Extensive experiments, the authors report, show the design restores weather-damaged traffic images efficiently and with precise detail, outperforming approaches that stay in the spatial domain alone.

Catalog record

CREATORLiwen Pan, Longguang Wang, Guangwei Gao, Jun Wang, Jun Shi, Juncheng Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-03852

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba Liwen Pan, Longguang Wang, Guangwei Gao, Jun Wang, Jun Shi, Juncheng Li Abstract Traffic image restoration under adverse weather conditions remains a critical challenge for intelligent transportation systems. Existing methods primarily focus on spatial-domain modeling but neglect frequency-domain priors. Although the emerging Mamba architecture excels at long-range dependency modeling through patch-wise correlation analysis, its potential for frequency-domain feature extraction remains unexplored. To address this, we propose Frequency-Aware Mamba (FA-Mamba), a novel framework that integrates frequency guidance with sequence modeling for efficient image restoration. Our architecture consists of two key components: (1) a Dual-Branch Feature Extraction Block (DFEB) that enhances local-global interaction via bidirectional 2D frequency-adaptive scanning, dynamically adjusting traversal paths based on sub-band texture distributions; and (2) a Prior-Guided Block (PGB) that refines texture details through wavelet-based high-frequency residual learning, enabling high-quality image reconstruction with precise details. Meanwhile, we design a novel Adaptive Frequency Scanning Mechanism (AFSM) for the Mamba architecture, which enables the Mamba to achieve frequency-domain scanning across distinct subgraphs, thereby fully leveraging the texture distribution characteristics inherent in subgraph structures. Extensive experiments demonstrate the efficiency and effectiveness of FA-Mamba. I Introduction One of the great challenges facing intelligent transportation systems is how to work effectively in adverse weather conditions. However, traffic images captured under such conditions often suffer from visual degradation, significantly affecting downstream visual tasks such as semantic segmentation, object detection, and autonomous driving [ 1 , 2 , 3 , 4 , 5 ] . To overcome this challenge, it is crucial to restore clean images from degraded ones, such as rain or snow images. However, it is a pretty difficult task due to occlusion and the loss of important perceptual frequency information in the images. Fig. 1 : Comparison of different modeling methods. Our FA-Mamba can enhance the ability of the Mamba to restore image texture details from the frequency perspective. Most traditional methods rely on prior knowledge or estimate task-specific degradation functions to achieve image restoration, but these methods usually have mediocre results and poor robustness. Recently, deep learning-based methods [ 6 , 7 , 8 , 9 , 10 , 11 , 12 ] have revolutionized image restoration by utilizing various deep network architectures, such as CNN and Transformer, to learn the mapping relationship between images and obtain higher quality images than traditional methods. Especially, the attention mechanism in Transformers has achieved remarkable success due to its ability to capture and model internal correlations within an image. While, as illustrated in Fig. 1 , the attention mechanism encounters scalability issues because of its quadratic complexity, which becomes a major challenge when processing large images. To address these limitations, the improved structured state-space sequence model (S4) with a selective scanning mechanism, Mamba, stands out for its ability to model long-range sequence dependencies with linear complexity. However, it is important to note that Mamba relies solely on pixel sequences to model image degradation in visual tasks, which limits its ability to perceive degradation in both spatial and frequency domains. To solve this issue, we aim to extend it to the wavelet domain and restore the image from both spatial and frequency domains. As is well known, the low-frequency information in the wavelet transform mainly corresponds to the global structure of the image, while the high-frequency information primarily captures details and textures. In image restoration tasks, recovering texture details is both crucial and challenging. To address this, we leverage the ability of Mamba to capture regional correlations in the spatial domain while integrating high-frequency analysis to perceive degradation better. This enhances the capability of the model to restore intricate textures more effectively. Specifically, we develop an effective Frequency-Aware Mamba for image restoration, named FA-Mamba. The infrastructure of FA-Mamba consists of a series of FA-Blocks, which include two key components: the Dual-Branch Feature Extraction Block (DFEB) and the Prior-Guided Block (PGB). The DFEB (Dual-Branch Feature Extraction Block) represents a hybrid module that integrates Mamba and CNN architectures, enabling the extraction and fusion of both local and global features. Compared to Transformer-based models, DFEB significantly reduces computational resource consumption while maintaining competitive performance. Meanwhile, the PGB (Prior-Guided Block) introduces enhanced high-frequency information representation, thereby improving the capability of the model in capturing intricate image texture details. Furthermore, we design a novel Adaptive Frequency Scanning Mechanism (AFSM) for the Mamba architecture. This mechanism facilitates frequency-domain scanning across distinct subgraphs, thereby fully leveraging the texture distribution characteristics inherent in subgraph structures. By introducing this innovative scanning approach from the frequency-domain perspective, AFSM offers a new method for exploration within the two-dimensional spatial domain. Our contributions can be summarized as follows: • We propose an efficient Frequency-Aware Block (FA-Block), which is composed of a Dual-Branch Feature Extraction Block (DFEB) and a Prior-Guided Block (PGB). This module is capable of simultaneously capturing both local and global features of the image and leveraging high-frequency information to generate high-quality restoration results. • We design a novel Adaptive Frequency Scanning Mechanism (AFSM) for the Mamba architecture. This mechanism facilitates frequency-domain scanning across distinct subgraphs, thereby fully leveraging the texture distribution characteristics inherent in subgraph structures. • We propose a Frequency-Aware Mamba (FA-Mamba) for traffic image restoration under adverse weather. Extensive experiments demonstrate the efficiency and effectiveness of FA-Mamba. II Related Works II-A Image Restoration The field of image restoration under adverse weather conditions has evolved significantly, transitioning from early model-based methodologies to sophisticated data-driven approaches. Traditional model-based techniques [ 13 , 14 , 15 , 16 ] primarily rely on the identification of appropriate weather-specific priors to address the image restoration problem. With the advent of deep learning in recent years, image restoration has achieved remarkable advancements in reconstruction quality. Foundational works such as SRCNN [ 17 ] and DnCNN [ 18 ] leveraged compact convolutional neural networks (CNNs) to deliver impressive performance in image restoration tasks. Nevertheless, CNN-based methods typically encounter challenges in effectively modeling global dependencies across image structures. Subsequently, the Transformer architecture—originally developed for natural language processing—has demonstrated extraordinary potential in computer vision and image restoration, rapidly emerging as a cornerstone for such tasks [ 8 , 10 , 9 , 11 ] . The Transformer excels in capturing global dependencies and modeling intricate relationships, which CNNs struggle to address. However, its self-attention mechanism introduces computational complexity that scales quadratically with input image size, posing significant challenges for larger image restoration tasks where computational efficiency becomes increasingly critical. Recently, diffusion models have garnered substantial attention in image restoration [ 19 , 20 , 21 ] , primarily due to their exceptional generative capabilities that enable precise data distribution modeling. Despite these advantages, the high computational resource requirements and prolonged inference times remain significant barriers to their widespread adoption in practical applications. II-B State Space Models Originating from classical control theory [ 22 ] , State Space Models (SSMs) [ 23 , 24 , 25 ] have attracted substantial attention from researchers due to their unique capability to process long-range dependencies with linear scalability relative to sequence length, thereby achieving remarkable computational efficiency in deep learning applications. A particularly noteworthy advancement is Mamba [ 26 ] , a selective, data-aware SSM specifically optimized for hardware acceleration. Mamba has demonstrated superior performance to Transformers

Read the complete text