ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes

Zhiming Liu, Zhicheng Zou, Nantheera Anantrasirichai · 2026

Read ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes by Zhiming Liu, Zhicheng Zou, Nantheera Anantrasirichai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21440.txt.

Introduction

Zhiming Liu, Zhicheng Zou, and Nantheera Anantrasirichai present ReMATF, a recurrent method for turbulence mitigation in dynamic scenes. Atmospheric turbulence degrades video through warping, blur, and flickering, harming clarity and time consistency. ReMATF restores videos using only two frames at a time. Its architecture pairs a multi-scale encoder-decoder with temporal warping and a motion-adaptive fusion module. Video from heat haze is hard to watch, and the goal is a clean, steady view.

Catalog record

CREATORZhiming Liu, Zhicheng Zou, Nantheera Anantrasirichai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47792
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21440

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes Zhiming Liu, Zhicheng Zou, Nantheera Anantrasirichai Abstract Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering, posing significant challenges to both visual clarity and temporal consistency. Current state-of-the-art methods are based on transformer, 3D architectures and require multi-frame input, but their large computational cost and memory usage limit real-time deployment, especially in resource-constrained scenarios. In this work, we propose ReMATF, a lightweight recurrent framework that restores videos using only two frames at a time while preserving spatial detail and temporal stability. ReMATF combines a multi-scale encoder–decoder with temporal warping and a motion-adaptive temporal fusion module that performs per-pixel fusion between the warped previous output and the current prediction to enhance coherence without enlarging the temporal window. This design reduces flicker, sharpens details, and remains efficient. Experiments on synthetic and real turbulence datasets show consistent improvements in PSNR/SSIM and perceptual quality (LPIPS), along with substantially faster inference than multi-frame transformer baselines, making ReMATF suitable turbulence mitigation in resource-constrained scenarios. 1 Introduction Atmospheric turbulence (AT) causes image degradation in long-range imaging systems due to spatial and temporal variations in the air’s refractive index along the optical path 4 . The primary effects include geometric distortion, where images appear to shimmer, warp, and ripple as turbulent air motion perturbs the light path. Rapid turbulent fluctuations act as a time-varying lens, smearing fine details and resulting in image blur 28 . In addition, intensity fluctuations cause random local brightening and dimming, an effect that becomes more pronounced at longer ranges and with smaller aperture sizes. These distortions not only deteriorate visual quality but also impair scene interpretation by both humans and machines, particularly under severe conditions. Mitigating AT distortion is therefore essential. Prior to the advent of deep learning, traditional approaches such as non-rigid registration, multi-frame fusion, and deconvolution were commonly employed 16 ; 2 ; 40 . These methods can produce clean and stable results, but are largely limited to static scenes, where hundreds of frames can be exploited under the assumption that pixel displacements have approximately zero mean and exhibit quasi-periodic motion 22 ; 11 . Consequently, effective AT mitigation in dynamic scenes remains challenging, as temporal redundancy for the same objects is insufficient. Deep learning has emerged as a promising solution, as models can be trained on large collections of distorted sequences rather than relying on per-sequence processing, as in traditional image processing methods 5 ; 12 . However, these data-driven approaches (discussed in Section 2 ) are often trained on synthetic datasets due to the lack of ground-truth data for atmospheric turbulence. As a result, they typically perform well only within the distortion regimes seen during training. Moreover, in real-world scenarios, turbulence severity can vary significantly and often co-occur with other degradations, such as haze, dust, and wind, limiting the generalisation capability of existing methods. Consistent with traditional approaches, static scenes benefit from an increased number of input frames, leading to improved restoration quality 36 . However, the number of frames is constrained by system memory. In dynamic scenes, five frames have been reported to achieve the best performance 3 . Notably, the trade-off between temporal information in scenes containing both static backgrounds and moving objects remains underexplored. In this paper, we focus on AT mitigation in dynamic, real-world scenes. We systematically analyse the influence of distortion severity, training data composition, and temporal window size on generalizable restoration performance. To address memory limitations while preserving long-range temporal cues, we introduce a recurrent transformer-based framework that explicitly separates short- and long-term temporal reasoning. In summary, we offer the following contributions: • We propose a lightweight transformer-based method, ReMATF , which for the first time mitigates AT distortions through coupled short- and long-term recurrent mechanisms. • The short-term recurrence provides immediate restoration cues to guide the network in suppressing ripple artifacts. • The long-term recurrence integrates the current restored output with the previous restored frame via a motion-aware weight map, estimated by our novel Motion-Adaptive Temporal Fusion (MATF) module. This design enables memory-efficient long-range temporal aggregation, where information from earlier frames is implicitly attenuated through recurrent propagation. • We introduce turbulence-level conditioning to help the network adapt to a wide range of distortion severities. • We construct a paired real-world AT dataset by replicating dynamic content from static scenes, enabling the generation of pseudo ground truth for supervised training. 2 Related Work Early learning based video approaches primarily use residual CNN restoration with limited explicit temporal modelling. Gao et al. 10 introduce a DnCNN-style network and exploit temporal cues via multi-frame aggregation. To incorporate stronger alignment priors, Hoffmire et al. 17 apply block matching-based registration and multi-shot averaging followed by CNN refinement. Later, Anantrasirichai 3 explores complex-valued convolution, motivated by the need to better model phase information altered by AT 6 . Generative and adversarial learning have also been explored. Jin et al. 20 propose TSR-WGAN that integrates spatial and temporal information in a 3D input, and employs a large-scale dataset spanning algorithm-simulated, physically simulated, and real sequences. TurbuGAN 9 removes the need for paired ground truth by combining a physics-based forward model with adversarial distribution matching between captured and simulated measurements for multi-frame blind deconvolution. ATVR GAN 8 introduces a recurrent GAN-based video restoration approach that estimates turbulent optical flow and propagates temporal information. Pyramid GAN and inverting GAN are used in 33 and 26 , respectively. Recent methods aggregate temporal information using attention mechanisms or recurrent designs. TMT 37 introduces a two-stage pipeline that decouples tilt correction from deblurring and extracts features using temporal attention. PiRN 18 incorporates physics-based turbulence priors into a transformer framework. DATUM 34 translates the classical lucky imaging principle into a learnable architecture, using deformable attention and temporal channel attention TurbSegRes 29 proposes a segment-then-restore pipeline, separating moving foreground from background before restoration. RMFAT 25 employs a recurrent multi-scale feature design that iteratively refines restoration across scales. DeTurb 41 combines deformable 3D convolutions for non-rigid alignment with a 3D Swin-based enhancement network. NeRT 19 leverages implicit neural representations together with a physically grounded tilt-then-blur turbulence model, while Nair 27 employs diffusion models. In parallel, generic video restoration backbones have been adapted to turbulence, such as EDVR 30 style designs that leverage deformable alignment and multi-frame fusion to improve stability. Relatedly, generic video restoration architectures originally proposed for video deblurring can also serve as transferable backbones for AT studies, including the transformer-based VRT 23 , the efficient spatio-temporal recurrent ESTRNN 38 , and recurrent multi-scale bi-directional propagation designs such as RNN-MBP 39 . To scale temporal modelling without the quadratic complexity of self-attention, MambaTM 36 and MAMAT 15 apply selective state space models for long-range spatiotemporal aggregation, and introduce a learned latent phase distortion representation to provide turbulence-specific guidance. Task-driven formulations further couple mitigation with downstream objectives, such as DMAT 14 , which jointly trains an AT mitigator and an object detector in an end-to-end framework, enabling feature exchange between low-level restoration and high-level semantics. Table 1 : Performance comparison of models trained and tested with AT levels. Model (trained on) Tested on Avg. Weak Medium Strong PSNR/SSIM/LPIPS PSNR/SSIM/LPIPS PSNR/SSIM/LPIPS PSNR/SSIM/LPIPS MAMAT 15 ( weak ) 27.6543 / 0.8426 /

Read the complete text