Simulating Refractive Distortions and Weather-Induced Artifacts for Resource-Constrained Autonomous Perception

Moseli Mots’oehli, Feimei Chen, Hok Wai Chan, Itumeleng Tlali, Thulani Babeli, Kyungim Baek, Huaijin Chen · 2025

Read Simulating Refractive Distortions and Weather-Induced Artifacts for Resource-Constrained Autonomous Perception by Moseli Mots’oehli, Feimei Chen, Hok Wai Chan, Itumeleng Tlali, Thulani Babeli, Kyungim Baek, Huaijin Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05536.txt.

Introduction

Moseli Mots'oehli and co-authors build software that fakes the visual flaws of cheap cameras and bad weather on driving footage, in a 2025 paper. Self-driving research leans on data from rich regions, leaving African roads, paved and unpaved alike, underrepresented. Their pipeline augments ordinary dashcam video with lens distortion, turbulence warps, splines, fog of two kinds, and lens flare, tuned to low-resource settings.

Three image restoration models were run as baselines on the doctored footage. The team releases the distortion toolkit, the augmented dataset splits, and the benchmark scores. The aim is to let perception systems train for hard conditions without costly new data collection.

Catalog record

CREATORMoseli Mots’oehli, Feimei Chen, Hok Wai Chan, Itumeleng Tlali, Thulani Babeli, Kyungim Baek, Huaijin Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05536

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Simulating Refractive Distortions and Weather-Induced Artifacts for Resource-Constrained Autonomous Perception Moseli Mots’oehli, Feimei Chen, Hok Wai Chan, Itumeleng Tlali, Thulani Babeli, Kyungim Baek, Huaijin Chen Abstract The scarcity of autonomous vehicle datasets from developing regions, particularly across Africa’s diverse urban, rural, and unpaved roads, remains a key obstacle to robust perception in low-resource settings. We present a procedural augmentation pipeline that enhances low-cost monocular dashcam footage with realistic refractive distortions and weather-induced artifacts tailored to challenging African driving scenarios. Our refractive module simulates optical effects from low-quality lenses and air turbulence, including lens distortion, Perlin noise, Thin-Plate Spline (TPS), and divergence-free (incompressible) warps. The weather module adds homogeneous fog, heterogeneous fog, and lens flare. To establish a benchmark, we provide baseline performance using three image restoration models. To support perception research in underrepresented African contexts, without costly data collection, labeling, or simulation, we release our distortion toolkit, augmented dataset splits, and benchmark results. Figure 1 : Overview of our two‐stage restoration pipeline. The Synthetic Distortion Module applies random refractive distortions and weather-induced artifacts described in Section 3 . The Refractive U-Net predicts a dense UV displacement field that’s used to correct the distortion in the input image. The Deweathering U-Net removes weather-induced artifacts from the input image. The two models are trained separately. 1 Introduction Autonomous driving research has advanced rapidly in urban and highway environments in developed countries, yet datasets from developing countries, particularly those in Africa, remain scarce [ 19 ] . Deploying delicate multi-modal sensor platforms and launching large-scale data collection and labeling efforts or performing high-budget simulations in these regions pose significant challenges. On the contrary, low-cost dashcams casually recording daily driving are becoming increasingly common. Even where such footage exists, it remains insufficient to capture the full diversity of African driving scenarios, including optical distortions from a wide range of camera models, turbulence-induced refraction, and complex weather variability. This paper addresses these gaps by procedurally augmenting real-world African dashcam footage with refractive distortions and weather-induced artifacts. With limited resources, we aim to generate sufficient training data as a first step toward enabling practical autonomous driving solutions in the Global South. Specifically, our contributions are as follows: • Scalable refractive distortion simulation in low-resource settings: Our synthetic distortion and artifact pipeline avoids costly hardware and fieldwork as well as high-budget physics-based rendering, enabling affordable dataset expansion in low-resource settings that models the most common refraction distortions. • Context-aware weather artifact simulation: We simulate harsh driving conditions, such as Cape Town fog or Limpopo sun glare, through a modular, efficient pipeline that produces paired clean/corrupted frames. • Un-distortion and de-weathering baselines and benchmarks: We show recovery of clean images from above mentioned artifacts and distortion using three baseline models: a ResUnet (CNN encoder), SegFormer-B1 (Transformer encoder), and DDPM (Diffusion probabilistic model), and provide a benchmark for future work in the area. 2 Related Work Our work, as it relates to prior research, draws from refractive lens distortion modeling, weather-induced artifact simulation, and deep learning-based image restoration. Below we organize prior related works in these three areas. 2.1 Refractive Lens Distortions The Brown-Conrady model [ 3 ] is widely used in the synthetic generation of geometric lens distortions, and normally paired with Perlin noise [ 22 ] , Thin-Plate Spline methods [ 1 ] , or the Truncated-Power Law [ 25 ] for more realistic refractive artifacts due to natural geometric irregularities and atmospheric turbulence at moderate computational cost. More recent lens distortion works, while more flexible and realistic through the use of deep learning to predict distortion parameters directly from images [ 7 , 16 ] , incur significantly higher computational costs, making them especially unsuitable for the low-resource settings we seek to address. Similarly, work by Li et al. [ 13 ] proposed a divergence-free vector field formulation, which is effective but computationally expensive for atmospheric turbulence simulation in low-resource settings. Our refractive lens distortion augmentation pipeline combines some of these methods, balancing computational efficiency and South African landscape scene realism in the context of resource-constrained autonomous perception scenarios. 2.2 Weather-Induced Artifact Simulation Recent works in fog and haze simulation combine physics-inspired methods with deep learning for better realism and lower inference overhead. In particular, in [ 14 ] , the authors integrate domain-adaptive object detection within synthetic fog environments, while [ 29 ] unifies weather classification and object detection, demonstrating robust performance gains through computationally efficient augmentations. Other resource-intensive synthetic weather simulation approaches show that the use of Generative Adversarial Networks (GANs) or physics-based rendering engines can improve generalization in adverse weather conditions [ 9 ] . Computational photography research continues to refine lens flare modeling. The Flare7K++ dataset [ 5 ] significantly advanced realistic flare generation, allowing networks to better generalize under adverse optical conditions. Such approaches incur moderate computational costs but deliver valuable realism crucial for autonomous driving scenarios, especially in challenging lighting conditions. 2.3 Deep Learning-Based Restoration Early deep learning approaches for image restoration primarily used encoder–decoder architectures with skip connections, such as U-Net [ 23 ] . Variants like U-Net++ and residual U-Nets further improved performance in low-resource settings [ 30 , 10 ] . More recent Transformer-based architectures, such as Segformer [ 26 ] and EvenFormer [ 18 ] , use lightweight or deformable attention mechanisms to improve efficiency and reconstruction quality. Diffusion models, while computationally expensive to train, represent the state-of-the-art in image artifact correction and image generation [ 11 , 21 ] , sometimes without the need for task-specific auxiliary loss functions [ 24 ] . Some approaches, such as Diff-UNet [ 27 ] , improve efficiency by combining lightweight priors with diffusion, achieving competitive volumetric segmentation results in fewer steps. More recent work introduces parameter-efficient adapter modules [ 12 ] to large pre-trained diffusion models without retraining the full network [ 15 ] , making scalable deployment of high-fidelity restoration models achievable in a low-resource setting without the high computation cost of training large diffusion models. 3 Methodology In this section, we describe our end-to-end data generation pipeline, which takes raw dashcam images from South African driving scenarios and generates a wide variety of refractive and weather-induced artifacts in the image. We first calibrate the camera using a standard ChArUco board to estimate intrinsic parameters and distortion coefficients, rectify the images with the calibrated parameters, then generate four different refractive and three weather-induced distortions based on the rectified images. We provide three baseline models, a fine-tuned ImageNet-1k pre-trained ResNet34 U-Net, a SegFormer, and a diffusion-based image reconstruction model to remove the distortion and artifacts and evaluate the results. Figure 1 shows our pipeline. We describe the data generation pipeline and the baseline restoration network in detail below. 3.1 Refractive Distortions Cheap dashcam lenses or atmospheric turbulence cause refractive optical distortion. By simulating these phenomena on undistorted frames, we can generate paired data (clean frame \leftrightarrow distorted frame and the warping UV map) that closely resemble what an autonomous perception system would see on Southern African roads. We use OpenCV [ 2 ] and GSTools [ 20 ] to implement the UV warping vector field maps described below. Figure 2 : Visualization of refractive distortions for randomly selected samples: Columns (1) Undistorted frame, (2) Distorted frame, (3) Corresponding

Read the complete text