SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions

Markus Essl 1 , Marta Moscati 1 , Mubashir Noman 2 , Muhammad Zaigham Zaheer 2 , Usman Naseem 3 , Shah Nawaz 1 , Markus Schedl 1,4 · 2026

Read SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions by Markus Essl 1 , Marta Moscati 1 , Mubashir Noman 2 , Muhammad Zaigham Zaheer 2 , Usman Naseem 3 , Shah Nawaz 1 , Markus Schedl 1,4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11799.txt.

Introduction

Markus Essl and six colleagues harden camera-LiDAR fusion for self-driving cars against broken or degraded sensors. Their framework-agnostic module plugs into BEVFusion and keeps 3D detection working when either modality goes missing or corrupts, instead of collapsing as unified representations do. On the MultiCorrupt dataset it beat existing approaches across many deterioration scenarios and led the field under extreme weather and outright sensor failure.

Catalog record

CREATORMarkus Essl 1 , Marta Moscati 1 , Mubashir Noman 2 , Muhammad Zaigham Zaheer 2 , Usman Naseem 3 , Shah Nawaz 1 , Markus Schedl 1,4
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22772
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11799

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions Markus Essl 1 , Marta Moscati 1 , Mubashir Noman 2 , Muhammad Zaigham Zaheer 2 , Usman Naseem 3 , Shah Nawaz 1 , Markus Schedl 1,4 Abstract Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independent sensors, such as camera and LiDAR, into a unified bird’s-eye view (BEV) representation for fusion. Although effective in ideal conditions, this strategy suffers from substantial performance deterioration when camera or LiDAR data are missing, corrupted, or noisy. To address this vulnerability, we develop a framework-agnostic fusion module for camera and LiDAR data that allows for handling cases when one of the two modalities is missing or corrupted. To demonstrate the effectiveness of our module, we instantiate it in BEVFusion [ 11 ] , a well-established framework to combine camera and LiDAR data for 3D object detection. By means of quantitative experiments on the MultiCorrupt dataset, we demonstrate that our module achieves favorable performance improvements under scenarios of missing and corrupted modalities, substantially outperforming existing unified representation approaches across a wide range of sensor deterioration scenarios and reaching state-of-the-art performance in scenarios of corrupted modality due to extreme weather conditions and sensor failure. † † address: 1 Johannes Kepler University Linz, Austria, 2 MBZUAI, UAE, 3 Macquarie University, Sydney, Australia, 4 Linz Institute of Technology, Austria 1 Introduction Perception in self-driving cars is predominantly dependent on two complementary sensors: camera and LiDAR [ 20 ] . The former sensor is responsible for rich visual appearance, while the latter offers the precise geometry necessary for accurate detection of objects [ 4 ] . To exploit this complementary information from camera and LiDAR, techniques for Bird’s-Eye View (BEV) such as BEVFusion [ 8 , 11 ] propose to fuse these two modalities, assuming that both are present and of high quality. However, real-world scenarios are not ideal and may lead to corrupted data, resulting in substantial performance deterioration of such systems [ 9 , 21 , 13 ] . These scenarios include situations like adverse weather conditions affecting the visibility from the camera, such as fog, rain, and malfunctions affecting the LiDAR system, such as beam reduction and occlusion [ 19 , 3 , 1 ] . Furthermore, spatial and temporal misalignment between the two modalities may cause considerable deterioration of the systems. Addressing these situations is an emerging field of research. For example, a recent work [ 1 ] has mainly focused on curating a benchmark to evaluate the state-of-the-art (SOTA) multimodal 3D detectors on their robustness against sensor failures or corruptions. To this end, we set out to develop a fusion technique capable of robustly handling corrupted or missing modalities, and at the same time, it can be seamlessly integrated as a fusion module in existing architectures. Specifically, we introduce a single-branch BEV detector (SB-BEVFusion) that allows the model to maintain good performance when camera or LiDAR data is missing. When both modalities are clean and present, the BEV features are combined by means of a pipeline relying on unweighted averaging, max pooling, cross-attention, and progressive modality decay. When one modality is absent or corrupted, our proposed architecture passes the available BEV representations without fusion. The novelty of our model relies on the fact that the downstream network and the detection head are shared across both modalities (camera (C) and LiDAR (L)) and trained on a shuffled mixture of L+C/L/C. This allows our model to handle both unimodal and multimodal scenarios in an effective way, therefore tackling the case of missing modality effectively. In summary, our main contributions are as follows. • We investigate various fusion strategies to combine the multimodal BEV feature representations, focusing on their robustness to missing and corrupted modality. • We introduce a module that is designed to explicitly operate under L+C/L/C scenarios. • We conduct extensive experiments on the MultiCorrupt dataset and empirically demonstrate that our proposed module demonstrates improved robustness towards the scenarios of missing and corrupted modality. Figure 1: Comparison between proposed SB-BEVFusion , BEVFusion [ 11 ] , and UniBEV [ 18 ] . a) Camera and LiDAR data is processed by modality-specific encoders to obtain features which are transformed to camera and LiDAR BEV representations. Afterwards, these BEV features are concatenated and processed by BEV encoder. b) UniBEV uses deformable attention before transformation to BEV space. It utilizes Channel Normalized Weights (CNW) fusion module to combine the BEV representations and feed them to BEV encoder. c) Proposed SB-BEVFusion does not add complexity to the architecture and combines the BEV representations using a fusion operator \mathcal{F} when both modalities are available or acts as an identity when one of them is absent. A shared BEV encoder processes the BEV features \mathbf{F}_{\text{in}} in all modes to obtain 3D detections. Figure 2: Illustration of various fusion operations to combine the BEV feature representations utilized in this study. 2 Methodology Setup and Notation. We adopt the pipeline of BEVFusion [ 11 ] , depicted in Fig. 1 , as underlying framework to apply our fusion module. As in BEVFusion, we first extract feature representations utilizing modality-specific encoders for the LiDAR point cloud \mathbf{P} and for the multi-view camera images \mathbf{I}=\{I_{k}\}_{k=1}^{K} , where K represents the number of camera views. These representations are then projected to a unified BEV space preserving the semantic and geometric information. The resulting representations in BEV space are given by \tilde{\mathbf{F}}_{\text{cam}}\in\mathbb{R}^{H\times W\times C_{\text{cam}}},\hskip 20.00003pt\tilde{\mathbf{F}}_{\text{lid}}\in\mathbb{R}^{H\times W\times C_{\text{lid}}} (1) where H,W,C represent the height, width, and number of channels, respectively. BEVFusion combines \tilde{\mathbf{F}}_{\text{cam}} and \tilde{\mathbf{F}}_{\text{lid}} , with simple concatenation \mathcal{F} , obtaining the multimodal representation which is fed to a convolutional BEV encoder \mathcal{E} , designed to adjust the local misalignments that can occur due to inaccurate depth. \mathbf{F}_{\text{fused}}=\mathcal{F}\left(\tilde{\mathbf{F}}_{\text{lid}},\tilde{\mathbf{F}}_{\text{cam}}\right),\hskip 20.00003pt\hat{\mathbf{y}}=\mathcal{E}\left(\mathbf{F}_{\text{fused}}\right) (2) It is well-known that the fusion operation is critical for effective combination of multimodal information [ 12 , 14 ] . This motivated us to investigate various fusion schemes and analyze their impact on improving model performance under different sensor corruptions. Moreover, our concern for the model robustness against sensor failure further encouraged us to introduce the SB-BEVFusion, which is based on effective training strategy and fusion operators and does not assume both modalities to be present. To this end, we first equalize channel widths such that C_{\text{cam}}=C_{\text{lid}}\triangleq C , yielding BEV tensors of same dimensions. This allows us to apply fusion \mathcal{F} when both modalities are present and only pass the features of the available modality to the BEV encoder otherwise, as illustrated in Fig. 1 . \mathbf{F}_{\text{in}}=\begin{cases}\mathcal{F}\left(\tilde{\mathbf{F}}_{\text{lid}},\tilde{\mathbf{F}}_{\text{cam}}\right),&a_{\text{lid}}\land a_{\text{cam}},\\ \tilde{\mathbf{F}}_{\text{lid}},&a_{\text{lid}},\\ \tilde{\mathbf{F}}_{\text{cam}},&a_{\text{cam}}.\end{cases}\hskip 20.00003pt\hat{\mathbf{y}}=\mathcal{E}\left(\mathbf{F}_{\text{in}}\right) (3) Fusion Operators. The fusion operator \mathcal{F} is responsible for mapping the same-sized multimodal BEV representations into a combined single BEV tensor, given by \mathbf{F}_{\text{in}}=\mathcal{F}(\tilde{\mathbf{F}}_{\text{lid}},\tilde{\mathbf{F}}_{\text{cam}})\in\mathbb{R}^{H\times W\times C} . The fusion operator is applied when both modalities are present and is replaced by the identity function when one modality is missing. The fusion operators investigated in our approach are depicted in Fig. 2 . 1. Averaging: We first combine the camera and LiDAR BEV representations by using unweighted average fusion. This is given by: \mathbf{F}_{\text{in}}=w\,\tilde{\mathbf{F}}_{\text{cam}}+(1-w)\,\tilde{\mathbf{F}}_{\text{lid}},\hskip 20.00003ptw=0.5 (4) 2. Max-Pooling: We apply the max-pooling operation on the two

Read the complete text