Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection

Sanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu, Valentina Donzella, Ciarán Eising · 2025

Read Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection by Sanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu, Valentina Donzella, Ciarán Eising (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-04347.txt.

Introduction

Sanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu, Valentina Donzella, and Ciaran Eising measure how weather-driven sensor occlusion degrades three-dimensional object detection for automated vehicles. They use the BEVFusion architecture on the nuScenes dataset. Bird's-eye-view fusion is a leading perception design, yet fog, haze, and blockages had not been isolated as a failure factor.

Moderate camera occlusion cut camera-only accuracy sharply, by about forty-one percent in mean average precision in the reported test. LiDAR held up until heavy occlusion, then fell by about forty-seven percent, hurting long-range detection most. In fused settings the damage depends on which sensor is occluded, a nuance designers must plan around.

Catalog record

CREATORSanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu, Valentina Donzella, Ciarán Eising
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-04347

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection Thanks: 1 Dept. of Electronic and Computer Engineering and the Data Driven Computer Engineering Research Centre, University of Limerick, Castletroy, Co. Limerick V94 T9PX, Ireland. 2 Valeo Vision Systems, Tuam, Co. Galway H54 Y276, Ireland. 3 Queen Mary University of London, Mile End Road, London E1 4NS, UK. Corresponding Author: Mr. Sanjay Kumar ( [email protected] ). Sanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu, Valentina Donzella, Ciarán Eising Abstract Accurate 3D object detection is essential for automated vehicles to navigate safely in complex real-world environments. Bird’s Eye View (BEV) representations, which project multi-sensor data into a top-down spatial format, have emerged as a powerful approach for robust perception. Although BEV-based fusion architectures have demonstrated strong performance through multimodal integration, the effects of sensor occlusions, caused by environmental conditions such as fog, haze, or physical obstructions, on 3D detection accuracy remain underexplored. In this work, we investigate the impact of occlusions on both camera and Light Detection and Ranging (LiDAR) outputs using the BEVFusion architecture, evaluated on the nuScenes dataset. Detection performance is measured using mean Average Precision (mAP) and the nuScenes Detection Score (NDS) . Our results show that moderate camera occlusions lead to a 41.3% drop in mAP (from 35.6% to 20.9%) when detection is based only on the camera. On the other hand, LiDAR sharply drops in performance only under heavy occlusion, with mAP falling by 47.3% (from 64.7% to 34.1%), with a severe impact on long-range detection. In fused settings, the effect depends on which sensor is occluded: occluding the camera leads to a minor 4.1% drop (from 68.5% to 65.7%), while occluding LiDAR results in a larger 26.8% drop (to 50.1%), revealing the model’s stronger reliance on LiDAR for the task of 3D object detection. Our results highlight the need for future research into occlusion-aware evaluation methods and improved sensor fusion techniques that can maintain detection accuracy in the presence of partial sensor failure or degradation due to adverse environmental conditions. I Introduction Modern automated vehicles rely on multiple sensors for accurate 3D perception [ 1 ] . Cameras provide rich appearance cues, such as colour and texture, but lack direct depth information, making it difficult to estimate distances accurately [ 2 ] . In contrast, LiDAR offers precise geometric measurements and depth data, but produces sparse point clouds that lack the high-resolution detail and semantic richness of camera images [ 3 ] . Fusing these complementary modalities into a BEV representation has emerged as an effective strategy for robust 3D object detection [ 4 ] . Despite progress in BEV-based sensor fusion architectures, most models are still evaluated under ideal, occlusion-free conditions [ 5 ] , [ 6 ] . However, in real-world deployments, sensor outputs are frequently degraded due to environmental factors. Cameras may be partially occluded by dirt, raindrops, or smudges on the lens, while LiDAR signals can be disrupted by adverse weather conditions such as fog or heavy rain, which reduce the number of returned points and the perception range [ 7 ] . These occlusions can significantly degrade detection performance and pose safety risks if not addressed properly. Previous work has only partially explored this issue. Xie et al. [ 8 ] investigate camera occlusions but do not consider LiDAR degradation for 3D object detection. Brophy et al. [ 9 ] analyse the impact of rain on camera-based detection but do not examine the effect of multimodal fusion or occlusion in the BEV -based domain. The effect of sensor-specific occlusions remains underexplored, particularly in the context of BEV fusion models. In this work, we systematically investigate the effect of occlusions on both camera and LiDAR sensors using the BEVFusion [ 10 ] architecture and the nuScenes dataset [ 11 ] . Camera occlusions are simulated using soiling masks from the Woodscape dataset [ 12 ] to replicate real-world lens obstructions. For LiDAR , we simulate occlusion by randomly dropping a proportion of the point cloud, mimicking the degradation caused by adverse weather conditions such as fog or rain. We evaluate detection performance using standard metrics, including Mean Average Precision (mAP) and nuScenes Detection Score (NDS), and analyse how different levels of occlusion affect each sensor individually. The main contributions of this paper are: 1. We simulate occlusions on both the camera (using the Woodscape soiled dataset) and LiDAR (via random point cloud dropout). 2. We evaluate the impact of these occlusions on 3D object detection using the BEVFusion architecture and the nuScenes dataset. 3. We analyse modality-specific and fused performance under occlusion, highlighting the limitations of current fusion models and motivating the need for occlusion-aware fusion strategies. The remainder of the paper is structured as follows: Section 2 reviews related work. Section 3 describes the nuScenes dataset and explains how the camera and LiDAR occlusions are applied. Section 4 outlines the methodology, including the BEVFusion model architecture and experimental setup. Section 5 presents the results of our experiments. Section 6 discusses key findings and future directions. II Related Work 3D object detection has become a critical task in automated driving, with methods generally categorised by their sensor inputs: camera-only, LiDAR -only, and multi-modal fusion of camera and LiDAR data. II-A Camera-Only Methods Recent advances have shown that 3D detection is possible using only image data. Li et al. [ 13 ] and Xie et al. [ 14 ] are two prominent researchers who developed approaches that construct a BEV representation from multi-camera images to facilitate detection. Li et al. [ 13 ] introduce a spatiotemporal transformer that aggregates camera views into a unified BEV space. In contrast, Xie et al. [ 14 ] jointly perform 3D detection and map segmentation by projecting multi-view image features into a shared voxel-based BEV grid. These methods avoid explicit depth supervision but still achieve competitive accuracy using camera-only inputs. However, camera-only methods struggle with accurate depth estimation, especially in low-light, distant, or occluded scenes. Without direct depth sensing, their 3D localisation is less reliable in challenging driving conditions. II-B LiDAR -Only Methods LiDAR -based approaches benefit from precise geometric information. Lang et al. [ 15 ] encodes point cloud data into a pseudo-image using vertical columns (pillars), allowing fast inference via 2D convolutions in BEV space. Yin et al. [ 16 ] further improve performance by adopting a centre-based detection framework, where object centres are predicted directly from BEV features, simplifying the regression pipeline and enhancing both accuracy and speed. However, LiDAR -only methods lack the rich semantic and texture information available in camera images, making it difficult to distinguish between visually similar objects and accurately interpret scene context, especially for small or distant objects. II-C Camera– LiDAR Fusion Methods To leverage the complementary strengths of LiDAR geometry and camera semantics, various fusion-based approaches have been proposed. Chen et al. [ 17 ] introduce learnable spatial offsets to align pixel-level image features with 3D LiDAR representations, enhancing the fusion process in BEV . Xu et al. [ 18 ] build on this by selectively injecting camera semantics into point cloud features via adaptive attention, improving performance, particularly for small or distant objects. Bai et al. [ 19 ] present a more

Read the complete text