Towards Closing the Domain Gap with Event Cameras
M. Oltan Sevinc, Liao Wu, Francisco Cruz · 2025
Read Towards Closing the Domain Gap with Event Cameras by M. Oltan Sevinc, Liao Wu, Francisco Cruz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-16178.txt.
Introduction
M Oltan Sevinc, Liao Wu, and Francisco Cruz test a camera swap for self-driving in this 2025 arXiv paper, accepted to an Australasian robotics conference. Cars trained on daytime images often fail at night, a mismatch called the domain gap. The authors ask whether event cameras, sensors that report only brightness changes, suffer less from that gap.
Event cameras react to motion and edges rather than full frames, so lighting shifts may matter less. In their tests, event camera systems kept performance steadier across lighting conditions than grayscale frame systems. Their cross-domain penalties were generally comparable or smaller, and their baseline results in shifted conditions were better. The findings suggest event cameras could make driving perception more robust when conditions change, without retraining for each new setting.
Catalog record
| CREATOR | M. Oltan Sevinc, Liao Wu, Francisco Cruz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-16178
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Closing the Domain Gap with Event Cameras M. Oltan Sevinc, Liao Wu, Francisco Cruz Abstract Although traditional cameras are the primary sensor for end-to-end driving, their performance suffers greatly when the conditions of the data they were trained on does not match the deployment environment, a problem known as the domain gap. In this work, we consider the day-night lighting difference domain gap. Instead of traditional cameras we propose event cameras as a potential alternative which can maintain performance across lighting condition domain gaps without requiring additional adjustments. Our results show that event cameras maintain more consistent performance across lighting conditions, exhibiting domain-shift penalties that are generally comparable to or smaller than grayscale frames and provide superior baseline performance in cross-domain scenarios. 1 Introduction End-to-end driving is a holistic approach to autonomous driving, where sensor data is directly used to generate control outputs. It stands in contrast to module-by-module optimized approaches, where aspects of a vehicle control task are broken down into human-understandable chunks and tackled one at a time, using methods such as neural networks [ 9 ] . End-to-end approaches may be preferable to module-by-module optimized approaches as they [ 8 ] : • Allow the loss function to propagate end-to-end, and therefore ensure there are no module losses that do not line up with the end goal. • Scale well with data-driven approaches, as the entire system can improve with more data. • Perform data fusion intuitively. Frame-based cameras are well established as the go-to sensor for end-to-end driving [ 3 , 6 , 25 , 24 ] . However, neural networks trained on frame-based camera data are often troubled by the domain gap problem, a degradation of performance when aspects of the training set, such as lighting conditions or environmental characteristics do not match those of the testing environment [ 22 , 23 ] . Unlike standard cameras, which capture absolute intensity frames, event cameras operate on pixel-level relative brightness changes [ 21 ] . We hypothesize that this quality reduces any domain gap based on lighting difference, and leads to less performance degradation compared to a frame-based camera when a neural network trained on event camera data encounters novel lighting conditions. This illumination invariance is particularly valuable for autonomous robotic systems that must operate reliably across varying environmental conditions without human intervention. To evaluate this hypothesis, we train end-to-end driving models on datasets biased toward daytime and nighttime driving for two sensor modalities: a conventional grayscale camera and a framed event camera. We evaluate each model on both daytime and nighttime test sets and quantify the performance degradation incurred when models are applied to out-of-domain illumination conditions. Proceeding from this hypothesis, our contributions are as follows: • We inspect the domain shift penalty encountered by grayscale and framed event camera recordings. We show that a neural network trained on a day-biased event camera dataset strongly outperforms its grayscale counterpart in night conditions. • We compare grayscale and framed event camera recordings from day and night to show event-camera data profiles remain consistent across lighting conditions. 2 Related Work Behavior cloning [ 1 ] , a branch of imitation learning, has gained prominence as a method for tackling the end-to-end driving problem. Behavior cloning involves the application of supervised learning over a dataset collected by a human expert solving the problem. It is an enticing approach because it simplifies the problem to the well-studied supervised learning one, with a clear loss signal. Behavior cloning has a rich history dating back to ALVINN [ 20 ] , which used a rudimentary neural network to steer a vehicle. The modern era of neural network-powered end-to-end driving took off after Bojarski et al. [ 3 ] demonstrated the potential modern deep-learning powered neural network approaches present in this domain, using a convolutional neural network with standard camera input. Event cameras are another candidate sensor for end-to-end driving. An initial dataset and attempt at event camera-based end-to-end behavior cloning for driving came in DDD17 [ 2 ] . DDD17 used dynamic vision sensors (DVSs, a subset of event cameras) in conjunction with an active pixel sensor (APS) generating grayscale frames, for steering angle regression. They aggregated their DVS events into frames, inspired by the work of [ 19 ] . An aggregation method was necessary, as event cameras are asynchronous in nature and thus cannot be readily processed by established methods in neural network literature, without first being converted to a format that parallelizes them into a compatible representation. This was taken a step further by Maqueda et al. [ 18 ] who introduced a sophisticated event framing method based on 2D histograms of ON and OFF events. Maqueda et al. then successfully used this method on the DDD17 dataset to establish a baseline for event frame-based end-to-end driving. A large extension to the DDD17 dataset was presented in DDD20 [ 14 ] . A biased dataset can negatively affect the performance of any behavior cloning model significantly [ 10 ] . For frame-based cameras, one of the sources of this bias can be the lighting conditions [ 22 , 8 ] , with many datasets focused on more idealized daytime lighting conditions [ 11 ] . One approach to address this problem is lighting-invariant transforms of RGB images [ 17 ] . Another approach is the real-time conversion of night images into the day domain better known to the network [ 22 ] . Event cameras have shown their promise across different lighting conditions, with their ability to detect small relative lighting changes being applied to enhance dark images [ 15 ] as well as for performing object detection across varying lighting conditions [ 5 ] . We find the next logical step is to utilize this potential to perform across varying illumination conditions to tackle the biased dataset problem, a connection that to our knowledge has not previously been made in literature. 3 Proposed Approach Working towards our goal of evaluating the lighting invariance of event cameras compared to grayscale cameras in an end-to-end driving task, we: 1. select a dataset that captures driving data across different lighting conditions using both types of cameras, 2. propose a preprocessing and data augmentation pipeline to improve the viability of the dataset for the comparison, 3. train separate end-to-end driving models for each sensor type using training data biased towards either daytime or nighttime samples, and 4. evaluate each trained model on test sets composed exclusively of daytime or exclusively of nighttime samples. 3.1 Dataset For our experiments we use the DDD20 dataset, introduced in Section 2 . This dataset presents driving footage captured by the DAVIS [ 4 ] sensor array, the vehicle’s steering angle, and other data captured across different lighting conditions. The DAVIS sensor array is made of an APS along with a DVS. The APS has an average refresh rate of 50 ms. We choose the vehicle steering angle as the output variable we want our model to learn. Using the utilities provided with [ 14 ] , we aggregate the event camera data in the dataset by combining 2D histograms of ON and OFF events over a period of 50 ms, creating event frames. The