High-speed Imaging through Turbulence with Event-based Light Fields

Yu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler · 2026

Read High-speed Imaging through Turbulence with Event-based Light Fields by Yu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14023.txt.

Introduction

Huang and co-authors build the first system that images fast, flexible objects through strong air turbulence at high frame rates. Event cameras record at thousands of frames per second but cannot tell object motion from turbulence shimmer. Their light field version grabs many views at once. True motion gives events that line up across views, while turbulence events do not. Tabletop tests image objects moving at up to 16,000 pixels per second through heavy turbulence.

Catalog record

CREATORYu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41734
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14023

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

High-speed Imaging through Turbulence with Event-based Light Fields Yu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler Abstract This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second. Figure 1 : Method overview. We introduce an event-based light field camera to image through turbulence at high speed. The event camera avoids motion blur by detecting brightness changes at microsecond time resolution. The light field offers cross-view constraints on the scene. 1 Introduction Long-range imaging through the atmosphere is the primary goal of terrestrial astronomy, remote sensing, surveillance, and other applications. Atmospheric turbulence represents the central impediment to this goal. Regardless of sensor resolution, aperture size, platform stability, or cost, atmospheric turbulence significantly degrades image fidelity by introducing blur, geometric distortion, and wander. Traditionally, turbulence has been mitigated with hardware-based solutions, such as adaptive optics (AO) [ 17 ] , or software-based solutions, such as lucky imaging [ 11 ] or, more recently, deep learning [ 53 , 54 , 55 ] . However, almost all existing solutions share the same fundamental constraint: they rely upon conventional image sensors which integrate light over a fixed exposure window, irrecoverably blending any scene dynamics or turbulence fluctuations that occur over a time scale shorter than the exposure time. In this work, we overcome this limitation with event cameras. Event cameras mitigate motion blur by measuring brightness change asynchronously at microsecond timescales, making them a natural candidate for recovering high-speed dynamics. Several lines of work have used frames and events to image through turbulence [ 2 , 3 , 27 , 25 ] . However, they have relied upon restrictive rigid-body assumptions about scene motion [ 2 , 3 ] or have restricted the reconstructions to the rate of the frame-based camera [ 27 , 25 ] , thereby giving up event-camera’s central advantage. Event-only imaging of dynamic scenes through turbulence is nontrivial. If the scene were static and all events were due to turbulence, one may be able to exploit these events to help reconstruct the scene [ 18 ] . If the turbulence was static and the scene was dynamic, one could use existing event-to-video frameworks to reconstruct a (distorted) high-speed video [ 37 , 38 , 40 ] . However, when both the turbulence and the scene are dynamic, both will produce (largely indistinguishable) events, thereby invalidating the assumptions that existing methods rely upon. In this work, we combine event cameras with light fields to overcome this ambiguity [ 36 ] . Light fields provide multiple sub-aperture views simultaneously, each corrupted by a different turbulence realization, while scene dynamics remain largely consistent across all views. Thus, each light field view provides a constraint on the singular scene image, and inconsistent motion across views can be attributed to turbulence. We exploit this structure (implicitly) using the recurrent encoder-decoder architecture E2VID [ 37 ] with stacked multi-view event inputs and train it to reconstruct clean, high-speed video through turbulence. In summary, our contributions are: • The first event-only method for high-speed imaging through turbulence, breaking the frame-rate barrier of conventional cameras. • Demonstration that light field optics provide sufficient information for removing turbulence while preserving scene motion. • Real (in-lab) and simulated experiments demonstrating our method significantly outperforms single-view reconstruction. 2 Related works 2.1 Light Field Imaging Light field imaging acquires multiple simultaneous views of a scene from distinct viewpoints, encoding both spatial and angular information of light [ 20 , 46 ] . Camera arrays achieve this by positioning multiple independent sensors at spatially separated viewpoints [ 49 ] . While conceptually straightforward, they require precise inter-sensor synchronization and their physical bulk limits deployment in constrained platforms. Single-shot alternatives using microlens arrays overcome the synchronization problem by imaging all sub-apertures onto a single sensor [ 1 , 34 ] . However, the dense lenslet grid demands precise optical alignment and manufacturing tolerances. Kaleidoscope-based designs avoid both drawbacks [ 29 , 36 ] . Planar mirrors placed at the intermediate image plane fold the optical path, routing light from distinct regions of the aperture to separate areas of the sensor and producing multiple virtual viewpoints without additional optics per channel. This construction requires no microlens fabrication and eliminates inter-camera synchronization entirely, accordingly we adopt it here. 2.2 Imaging Through Aberrations with Light Fields Ng and Hanrahan first demonstrated that light fields could be used to correct for optical aberrations back in 2006 [ 33 ] . Light fields have since been used to image through turbulence on multiple occasions [ 44 , 28 , 45 , 47 ] . Intuitively, by capturing multiple perspectives of the same scene at the exact same time, these cameras create valuable spatial redundancy that can be used to remove turbulence and other aberrations. These systems have relied on conventional image sensors. While a few recent works have combined light fields with event cameras to perform high-speed 3D imaging [ 15 , 36 ] , ours is the first work to use event-based light field cameras to image through turbulence. 2.3 Software-based Imaging through Turbulence Classical turbulence mitigation algorithms often use spatial registration and temporal fusion [ 30 , 5 , 39 ] to recover imagery. Accordingly, these methods generally depend on static references or distortion-free patches and fail in dynamic scenes or strong turbulence. Learning-based methods relax these requirements: self-supervised and test-time optimization use priors such as lucky images [ 11 ] , blind degradation estimation [ 21 , 26 ] , or pretrained mitigation models [ 10 , 4 ] and adapt without paired data, but are iterative and ill-suited to high-speed use; meanwhile physics-based simulators [ 31 ] enable supervised learning with fast inference via feed-forward sequence-to-sequence [ 55 ] or recurrent [ 53 , 12 ] networks. All of these approaches remain limited by conventional sensors’ fixed exposure, which conflates scene dynamics and turbulence at high frame rates. Event cameras, with microsecond-level temporal resolution, are a natural candidate for resolving this bottleneck. An early work, used an event camera to identify lucky regions in conventional frames [ 2 , 3 ] . Two recent works explore this direction with machine learning: EvTurb [ 27 ] employs events to guide single-frame restoration, while EGTM [ 25 ] leverages event temporal resolution to compute adaptive fusion weights for conventional frames. In both, the final image formation depends on the integration time of the conventional sensor, leaving output constrained to standard frame rates. A fundamental bottleneck thus persists across all

Read the complete text