AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery
Raju Imandi 1 , Chethana B 2 , Bharatesh C 3 , Yong-Guk Kim 4 , Manipriya S 5 , Pavan Kumar B N 5 · 2026
Read AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery by Raju Imandi 1 , Chethana B 2 , Bharatesh C 3 , Yong-Guk Kim 4 , Manipriya S 5 , Pavan Kumar B N 5 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24691.txt.
Introduction
Raju Imandi and others build a detector for extreme low light, where normal RGB cameras fail but event cameras still sense per-pixel brightness changes. Their AdaFuse-Det framework merges contrast-enhanced RGB frames and voxelized event tensors through adaptive fusion grounded in minimum-variance estimation. The learned attention map recovers Gauss-Markov optimal fusion weights in the limit, with bounds on conservation and resolution. On LLE-VOS it posts an F1-score of 59.12 percent under severe light loss, beating single-modality detectors on recall.
Catalog record
| CREATOR | Raju Imandi 1 , Chethana B 2 , Bharatesh C 3 , Yong-Guk Kim 4 , Manipriya S 5 , Pavan Kumar B N 5 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24770 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24691
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery Raju Imandi 1 , Chethana B 2 , Bharatesh C 3 , Yong-Guk Kim 4 , Manipriya S 5 , Pavan Kumar B N 5 Abstract Detecting objects reliably under extreme low-light conditions is an open problem in computer vision, with practical urgency in applications ranging from nighttime surveillance to search-and-rescue robotics. Conventional RGB cameras degrade sharply at low photon flux, while event cameras which record asynchronous per-pixel brightness changes at microsecond resolution and high dynamic range provide complementary structural cues that are largely illumination-invariant. We present AdaFuse-Det , a dual-stream framework that fuses CLAHE-enhanced RGB frames with voxelized event tensors through an Adaptive Cross-Modal Fusion (ACMF) module grounded in minimum-variance linear estimation theory. We formally show that the learned attention map asymptotically recovers the Gauss–Markov optimal fusion weights, and establish event conservation and temporal resolution bounds for the voxelization stage. On the LLE-VOS benchmark, AdaFuse-Det achieves a Recall of 65.54 % 65.54\% , Precision of 53.85 % 53.85\% , and F1-Score of 59.12 % 59.12\% under severe illumination degradation, outperforming single-modality detectors in recall by a margin that reflects the theoretically predicted illumination-adaptation behavior. I Introduction Object detection under low-light conditions is a persistent challenge in safety-critical systems autonomous driving, nighttime surveillance, and search-and-rescue operations all demand reliable perception in conditions where conventional cameras routinely fail [ 20 , 10 ] . The fundamental difficulty is photophysical: at low photon flux, the signal-to-noise ratio degrades as \mathrm{SNR}\propto\sqrt{N_{p}} , where N_{p} is the expected photon count per pixel per exposure interval [ 1 ] . Longer exposures improve SNR but introduce motion blur, and the two degradations compound in a way that standard detectors designed for clean, high-contrast imagery handle poorly. Prior work has taken two broad routes. Enhancement-first approaches apply Retinex-based decomposition [ 2 ] , learned denoising [ 3 ] , or histogram equalization before running a standard detector. The trouble is that enhancement amplifies noise as well as signal, and the resulting artifacts can actively mislead downstream features. End-to-end approaches train detectors directly on degraded data, which avoids the artifact problem but still relies entirely on RGB information that may be close to uninformative at extreme darkness. Event cameras sidestep this bottleneck [ 4 , 5 ] . Rather than integrating photons over a fixed window, each pixel fires asynchronously the moment its log-luminance changes by more than a threshold \theta , emitting an event e=(x,y,t,p) with polarity p\in\{-1,+1\} . The resulting data stream has microsecond temporal resolution, a dynamic range exceeding 120\,\mathrm{dB} , and is naturally edge and motion selective. The catch is that event data is sparse and carries almost no photometric or semantic texture event-only detectors struggle with static scenes and fine-grained classification [ 11 ] . These two modalities are practically complementary: RGB is semantically rich but illumination-sensitive, while events are structurally informative but semantically thin. The key question is how to fuse them so that each modality’s contribution scales with its local reliability rather than being fixed at design time. Naïve approaches channel concatenation or fixed-weight averaging cannot do this, since the relative quality of RGB versus event information varies spatially and across scenes. I-A Contributions This paper makes the following specific contributions: • A formal treatment of multimodal fusion as minimum-variance linear estimation, yielding closed-form optimal attention weights (Theorem II.3 ) and an illumination-adaptation corollary (Corollary II.4 ) that explains the recall-precision asymmetry we observe empirically. • Theoretical characterization of the voxelization stage, including a proof of exact event conservation (Lemma II.1 ) and a temporal resolution bound (Lemma II.2 ) that informs bin-count selection. • The ACMF module itself, which we prove strictly subsumes unimodal and uniform fusion as limiting cases (Proposition II.5 ), with principled behavior at feature-equality locations (Proposition II.6 ). • Empirical evaluation of AdaFuse-Det on LLE-VOS, a synchronized RGB-event benchmark captured under controlled extreme low-light conditions. I-B Related Work Several recent systems have explored RGB-event fusion for detection under adverse conditions. Tomy et al. [ 14 ] demonstrated that a dual-stream RetinaNet-50 using voxelized events is substantially more robust to image corruptions than its RGB-only counterpart. More recent work SFNet [ 17 ] and MCFNet [ 18 ] further improves temporal alignment and adaptive fusion but is aimed at traffic scenes under variable (not extreme) illumination. Liu et al. [ 12 ] specifically studied event cameras in nighttime settings and found that event-derived motion cues remained reliable well below the operating range of frame-based cameras a finding that directly motivates our design. Within our group, earlier work surveyed event cameras and event simulators [ 13 , 19 ] for detection and tracking [ 20 ] and studied LiDAR-based human detection as an alternative sensing modality for safety-critical autonomy [ 21 ] . The present paper builds on those efforts by targeting the specific failure mode of extreme low-light RGB, where LiDAR is not always available and event cameras offer a practical path forward. II Methodology Notation. We use bold lowercase for vectors ( \bm{\theta} ), bold uppercase for matrices and tensors ( \mathbf{V} , \mathbf{F} ), and calligraphic letters for sets ( \mathcal{E} , \mathcal{S} ). The symbol \odot is the Hadamard (element-wise) product, \mathbf{1}\!\left[A\right] is the indicator of predicate A , and \sigma(z)=(1+e^{-z})^{-1} . Let \Omega\subset\mathbb{N}^{2} be the H\times W spatial pixel grid. An event stream is \mathcal{E}=\{e_{i}\}_{i=1}^{N} , with e_{i}=(x_{i},y_{i},t_{i},p_{i})\in\Omega\times\mathbb{R}_{+}\times\{-1,+1\} . Feature pyramids at scale s are written \mathbf{F}^{s}\in\mathbb{R}^{C_{s}\times H_{s}\times W_{s}} , where H_{s}=H/2^{s} and W_{s}=W/2^{s} . II-A System Overview Figure 1 shows the full pipeline. Given a low-light RGB frame \mathbf{I}\in[0,1]^{3\times H_{r}\times W_{r}} and a voxelized event tensor \mathbf{V}\in\mathbb{R}^{2B\times H_{e}\times W_{e}} from the same time window [t_{0},t_{0}+\Delta t] , the goal is to learn f_{\bm{\Theta}}:(\mathbf{I},\mathbf{V})\;\longmapsto\;\{(b_{n},c_{n})\}_{n=1}^{N_{\det}}, (1) where b_{n}\in\mathbb{R}^{4} and c_{n}\in\{1,2,3\} are the bounding box and class label for the n -th detection. Four sequential stages realize this mapping: event preprocessing (§ II-B ), image enhancement (§ II-C ), parallel feature extraction (§ II-D ), adaptive fusion and detection (§ II-E – II-F ). Event stream Image stream ACMF Event voxel grid \mathbf{V}\!\in\!\mathbb{R}^{2B\times H\times W} Event encoder ResNet-18 (8-ch) RGB + CLAHE \mathbf{I}_{\mathrm{norm}}\!\in\![0,1]^{3} Image encoder ResNet-18 (3-ch) Attention \alpha^{s} \sigma\!\bigl(\mathrm{Conv}_{3}\bigl(\mathrm{ReLU}\bigl( \mathrm{Conv}_{1}([\mathbf{F}_{\mathrm{img}};\mathbf{F}_{\mathrm{evt}}])\bigr)\bigr)\bigr) Weighted fusion \mathbf{F}^{s}\!=\!\alpha^{s}\!\odot\!\mathbf{F}_{\mathrm{img}} +\,(1\!-\!\alpha^{s})\!\odot\!\mathbf{F}_{\mathrm{evt}} YOLO head 4-scale anchors Detections bbox + class \alpha^{s} \mathbf{F}^{s}_{\mathrm{fused}} Preprocessing Feature extraction Adaptive fusion Detection Fig. 1 : AdaFuse-Det architecture. Dual-stream encoders process voxelized event data and CLAHE-enhanced RGB frames independently; the ACMF module learns spatially varying attention weights that balance modality contributions at each pyramid scale; a multi-scale YOLO-style head produces final detections. II-B Event Stream Representation II-B 1 Voxelization The raw event stream is a sparse, asynchronous point process over \Omega\times\mathbb{R}_{+} , which convolutional networks cannot consume directly. We convert it to a dense tensor via temporal voxelization following [ 6 ] , using B=4 temporal bins per polarity for a total of 8 channels. For the restricted stream \mathcal{E}_{\Delta t} over window [t_{0},t_{0}+\Delta t] , the normalized timestamp of event e_{i} is \tau_{i}\;:=\;\frac{t_{i}-t_{0}}{\Delta t}\cdot(B-1)\;\in\;[0,\,B-1], (2) and the bilinear interpolation kernel for bin b is k(\tau_{i},\,b)\;:=\;\max\!\bigl(0,\;1-|\tau_{i}-b|\bigr). (3) Setting q=(p_{i}+1)/2\in\{0,1\} for polarity, the (b,q) -th voxel channel accumulates V(b,\,q,\,x,\,y)\;:=\;\sum_{e_{i}\in\mathcal{E}_{\Delta t}}\mathbf{1}\!\left[x_{i}=x,\,y_{i}=y,\,q_{i}=q\right]\cdot k(\tau_{i},\,b). (4) Two properties of this construction are worth recording formally. Lemma