MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability
Paul Hill, Zhiming Liu, Nantheera Anantrasirichai · 2025
Read MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability by Paul Hill, Zhiming Liu, Nantheera Anantrasirichai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17700.txt.
Introduction
Hill, Liu, and Anantrasirichai clear heat shimmer from video with a Mamba-based network in a 2025 paper. Air turbulence warps surveillance footage and blurs detection. Their system, MAMAT, works in two stages. Flexible three-dimensional filters first re-align the shifting frames. A second stage then restores contrast and detail.
Against top learning methods, MAMAT lifted picture quality by up to three percent. Object detection on the cleaned video rose by up to fifteen percent. The authors stress the second gain: restored video should serve real watching tasks, not just look better. The work targets long-range surveillance in hot or unstable air.
Catalog record
| CREATOR | Paul Hill, Zhiming Liu, Nantheera Anantrasirichai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17700
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability Paul Hill, Zhiming Liu, Nantheera Anantrasirichai Abstract Restoration and enhancement are essential for improving the quality of videos captured under atmospheric turbulence conditions, aiding visualization, object detection, classification, and tracking in surveillance systems. In this paper, we introduce a novel Mamba-based method, the 3D Mamba-Based Atmospheric Turbulence Removal (MAMAT), which employs a dual-module strategy to mitigate these distortions. The first module utilizes deformable 3D convolutions for non-rigid registration to minimize spatial shifts, while the second module enhances contrast and detail. Leveraging the advanced capabilities of the 3D Mamba architecture, experimental results demonstrate that MAMAT outperforms state-of-the-art learning-based methods, achieving up to a 3% improvement in visual quality and a 15% boost in object detection. It not only enhances visualization but also significantly improves object detection accuracy, bridging the gap between visual restoration and the effectiveness of surveillance applications. 1 Introduction Atmospheric turbulence distortions, caused by significant temperature gradients between the ground and air, can severely degrade image and video quality. This phenomenon leads to rapidly rising air layers, altering the index of refraction along the optical path and resulting in visual distortions such as blurriness, ripples, and intensity fluctuations. These effects pose significant challenges for long-range surveillance, particularly in identifying distant objects. Modeling atmospheric turbulence in video footage is a computationally demanding and complex task, primarily because it is an ill-posed problem with distortions that vary across space and time. Model-based approaches often hinder real-time implementation, limiting their effectiveness in dynamic settings [ 1 , 2 , 3 , 4 ] . Additionally, merging multiple images to enhance quality can introduce artifacts, especially in the presence of moving objects, due to misalignment. Consequently, there is growing interest in leveraging deep learning-based video restoration techniques, which offer more robust solutions. The initial deep learning advancements in atmospheric turbulence correction predominantly relied on convolutional neural networks (CNNs) [ 5 , 6 , 7 , 8 , 9 , 10 ] . More recently, there has been a shift towards Transformer-based models, which have demonstrated improved performance in turbulence mitigation [ 11 , 12 , 13 , 14 , 15 , 16 ] . Given the complexity of atmospheric distortions, modern systems typically incorporate dual modules: one dedicated to frame alignment and enhancement [ 7 , 16 ] and another focused on correcting tilt and blur [ 10 , 14 ] . These modules collectively demonstrate significant efficacy in addressing the intricate problem of atmospheric turbulence, highlighting the potential of integrated approaches for improving visual clarity and stability. Though Transformer-based methods show promising results, the computational demands present challenges in processing complex, distorted images. Structured State Space sequence (S4) models offer efficient alternatives, excelling in long-sequence data analysis. The Mamba framework [ 17 ] enhances S4, particularly in handling time-variant operations, and has shown promising results in image analysis and computer vision. To date, Mamba has not been adapted for atmospheric turbulence removal, highlighting a promising direction for future research. In this paper, we propose a novel Mamba-based method for atmospheric turbulence mitigation, referred to as 3D Mam ba-Based A tmospheric T urbulence Removal (MAMAT). We employ a dual-module strategy: the first module performs non-rigid registration to minimize spatial shifts between frames caused by random temporal permutations, utilizing deformable 3D convolutions. The second module enhances contrast, sharpness, and textural details using the 3D Mamba architecture [ 18 ] , known for its efficiency in processing spatiotemporal data. By utilizing this framework, MAMAT effectively addresses complex distortions in video sequences, setting a new benchmark for clarity and precision in turbulence mitigation. It is important to note that existing methods may effectively mitigate atmospheric turbulence but may not optimize imagery for crucial automatic object detection in surveillance. Techniques that reduce visual distortions, such as wavy or ripple effects, could inadvertently remove vital features necessary for object identification. Therefore, we demonstrate that our proposed MAMAT not only enhances visual quality but also significantly improves object detection accuracy, effectively bridging the gap between visual restoration and practical surveillance applications. In summary, our main contributions can be listed as follows: • We propose a novel framework, MAMAT, for restoring long-range videos affected by spatiotemporal distortions. • We apply deformable 3D convolutions in all feature scales, providing flexibility in capturing the different shapes of distorted objects. • MAMAT is the first method to leverage the 3D Mamba block for atmospheric turbulence removal. • We demonstrate that MAMAT enhances visual quality and also improves object detection accuracy, both of which are crucial for various applications. 2 Related Work Atmospheric Turbulence Mitigation. Early work in deep learning-based approaches employed convolutional neural networks, including the Gaussian denoiser architecture [ 5 ] , the UNet architecture [ 19 ] , and stacked hourglass networks [ 6 , 7 ] . A WGAN (Wasserstein generative adversarial network) is employed in [ 8 ] , where the multiple lucky frames are fed into the UNet generator. DATUM [ 11 ] introduces deformable attention alignment and temporal fusion via multi-head temporal channel self-attention. Turb-Seg-Res [ 15 ] stabilizes dynamic video frames using normalized cross-correlation, segments moving objects, and refines the background via adaptive filtering before merging it with a separately extracted foreground. DeTurb [ 16 ] employs a pyramid architecture with deformable 3D convolutions to eliminate spatial-temporal distortions. It then reconstructs a restored image using a multi-scale architecture composed of 3D Swin Transformers, thereby improving contrast and reducing blur. For a more comprehensive review of recent deep learning techniques for atmospheric turbulence removal, see [ 20 ] . Object Detection. In clear mediums, well-known CNN-based methods such as Faster R-CNN [ 21 ] and RetinaNet [ 22 ] utilize multi-scale feature extraction. YOLO (You Only Look Once), particularly its latest iteration, YOLOv11 [ 23 ] , is the most popular real-time detector, incorporating advanced techniques to enhance performance. For atmospheric turbulence conditions, traditional object detection methods primarily focus on long-distance objects that often appear small and lack detail [ 24 , 25 ] . These methods are limited by the reduced clarity and specificity of the images. Despite these challenges, deep learning techniques specifically designed for these conditions are rare. Face recognition using Generative Adversarial Networks (GANs) [ 26 ] has achieved some success, although this method struggles in complex scenarios involving various object types. To extend the capabilities, Hu et al. [ 27 ] have improved Faster R-CNN and YOLOv4 [ 28 ] with deformable convolutional layers to address frame displacement. Uzun et al. [ 29 ] demonstrated that YOLOR [ 30 ] outperforms VfNet [ 31 ] and TOOD [ 32 ] in both precision and speed, showing a promising direction for enhancing object detection in turbulent conditions. 3 Methodology The diagram of the proposed MAMAT framework is shown in Fig. 1 , comprising two modules: (i) the S patial D isplacement A cross T ime Removal (SDAT) module, which removes spatial displacements that occur over time, and (ii) the E nhancement and D etail P reserve (EDP) module, which enhances visual quality spatially and temporally with 3D Mamba. Figure 1: Diagram of the proposed MAMAT framework 3.1 SDAT: Spatial Displacement Across Time Removal Module Addressing the challenge of atmospheric turbulence, particularly the