CBDES MoE: Hierarchically Decoupled Mixture-of-Experts for Functional Modules in Autonomous Driving
Qi Xiang, Kunsong Shi, Zhigui Lin, Lei He · 2025
Read CBDES MoE: Hierarchically Decoupled Mixture-of-Experts for Functional Modules in Autonomous Driving by Qi Xiang, Kunsong Shi, Zhigui Lin, Lei He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07838.txt.
Introduction
Qi Xiang, Kunsong Shi, Zhigui Lin, and Lei He restructure the perception stack of self-driving cars as a team of specialist networks, in a paper published in two thousand twenty-five. Their setting is bird's-eye view perception, which fuses camera and other sensor data into a map-like picture around the car.
Existing fusion methods adapt poorly to changed inputs, model relations weakly, and generalize badly. The team's system, CBDES MoE, splits the stack by function and staffs each function with a mixture of experts. The experts differ in structure, and a light self-attention router picks which ones run for each input. That sparse, input-aware routing keeps inference efficient. The authors describe it as the first mixture-of-experts design built at whole-module level for driving. Broad evaluations back the design. The work aims at perception that bends with sensor setups instead of breaking.
Catalog record
| CREATOR | Qi Xiang, Kunsong Shi, Zhigui Lin, Lei He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07838
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CBDES MoE: Hierarchically Decoupled Mixture-of-Experts for Functional Modules in Autonomous Driving Qi Xiang, Kunsong Shi, Zhigui Lin, Lei He Abstract Bird’s Eye View (BEV) perception systems based on multi-sensor feature fusion have become a fundamental cornerstone for end-to-end autonomous driving. However, existing multi-modal BEV methods commonly suffer from limited input adaptability, constrained modeling capacity, and suboptimal generalization. To address these challenges, we propose a hierarchically decoupled Mixture-of-Experts architecture at the functional module level, termed Computing Brain DEvelopment System Mixture-of-Experts (CBDES MoE). CBDES MoE integrates multiple structurally heterogeneous expert networks with a lightweight Self-Attention Router (SAR) gating mechanism, enabling dynamic expert path selection and sparse, input-aware efficient inference. To the best of our knowledge, this is the first modular Mixture-of-Experts framework constructed at the functional module granularity within the autonomous driving domain. Extensive evaluations on the real-world nuScenes dataset demonstrate that CBDES MoE consistently outperforms fixed single-expert baselines in 3D object detection. Compared to the strongest single-expert model, CBDES MoE achieves a 1.6-point increase in mAP and a 4.1-point improvement in NDS, demonstrating the effectiveness and practical advantages of the proposed approach. 1 School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China 2 State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing 100084, China 3 Department of Computing, Imperial College London, London SW7 2AZ, UK 4 School of Automotive Engineering, Wuhan University of Technology, Wuhan 430070, China Introduction With the rapid development of autonomous driving technologies, 3D perception has become a cornerstone for building safe, intelligent, and reliable driving systems. Among the mainstream solutions, Bird’s Eye View (BEV)-based multi-modal fusion frameworks—such as BEVFusion—have shown great promise by projecting raw sensor inputs from cameras and LiDAR into a unified spatial representation ( Huang et al. 2022 ; Li et al. 2022a ; Li et al. 2022b ; Liu et al. 2024 ; Liang et al. 2022 ) . These structured BEV features enable accurate 3D object detection and support a range of downstream driving tasks. Despite their success, most existing approaches adopt fixed single-backbone feature extractors for each modality. While this design choice simplifies the training pipeline, it severely restricts the model’s adaptability to diverse and dynamically changing driving environments, such as varying lighting, weather, and camera viewpoints. Moreover, the limited modeling capacity of single-backbone architectures compromises their ability to capture semantically rich and complex scene information, ultimately leading to degraded performance under domain shifts or task transitions. To address these challenges, prior studies have explored adaptive modules such as dynamic convolutions and deformable attention ( Chen et al. 2020 ; Xia et al. 2022 ; He et al. 2022 ) . While these techniques improve local flexibility, they remain confined within a rigid, monolithic network structure. Lacking coarse-grained architectural adaptability, such models fall short in dynamically reallocating capacity based on input variability—thereby hindering large-scale deployment in safety-critical autonomous driving systems. The Mixture-of-Experts (MoE) paradigm offers a compelling solution. By enabling dynamic expert selection through learned routing mechanisms, MoE models can flexibly balance computational efficiency with representational richness ( Jacobs et al. 1991 ) . Although MoE has achieved remarkable success in natural language processing and vision-language tasks ( Lepikhin et al. 2020 ; Fedus, Zoph, and Shazeer 2022 ; Dai et al. 2024 ; Riquelme et al. 2021 ) , its potential in BEV-based 3D perception remains largely untapped. In particular, designing heterogeneous expert backbones and routing strategies tailored for multi-modal fusion poses unique challenges that have not yet been adequately addressed. In this work, we propose CBDES MoE, a hierarchically decoupled Mixture-of-Experts architecture designed specifically for BEV perception tasks. Our method introduces a diverse pool of backbone experts, a lightweight gated routing network, and sparse activation mechanisms to enable efficient, adaptive, and scalable 3D perception. Comprehensive experiments on the nuScenes benchmark demonstrate that our approach consistently surpasses strong single-backbone baselines, especially under complex and diverse driving scenarios. Our main contributions are summarized as follows: • CBDES MoE introduces a novel expert-based architecture with a heterogeneous pool of multi-stage backbone designs. By supporting hierarchical decoupling and dynamic expert selection, the model achieves enhanced scene adaptability and feature representation capability. • A lightweight, layered routing mechanism is developed, integrating convolutional operations, self-attention, and multi-layer perceptrons. This module enables end-to-end learning of the input-to-expert mapping, supporting sparse activation and efficient dynamic inference. • Extensive experiments are conducted on the nuScenes dataset, demonstrating that CBDES MoE consistently surpasses strong single-backbone baselines, particularly under diverse and challenging environmental conditions. Related Work Multi-Modal BEV Perception As autonomous driving systems evolve, multi-sensor information fusion has become a key technique for improving perception accuracy and robustness. Early approaches often relied on independent pipeline stages or point cloud-based fusion methods (e.g., PointPillars, VoxelNet), which struggled to leverage the rich texture and semantics of images ( Lang et al. 2019 ; Zhou and Tuzel 2017 ) . In recent years, BEV representations have gained popularity due to their unified spatial projection properties. Methods like BEVDet, BEVDepth, BEVFormer, and others have demonstrated significant performance gains in 3D detection and map segmentation by projecting multi-view camera images into the BEV space ( Huang et al. 2022 ; Li et al. 2022a ; Li et al. 2022b ) . Further, several works, such as BEVFusion, proposed feature-level multi-modal fusion frameworks that align and jointly model camera and LiDAR information within the BEV space ( Liu et al. 2024 ; Liang et al. 2022 ) . However, these methods typically adopt fixed, single-backbone architectures (e.g., ResNet, Swin Transformer), lacking the ability to adapt to diverse and dynamic input conditions. Dynamic Design Paradigms Autonomous driving scenarios exhibit high variability—lighting, weather, viewpoints, and road layouts—requiring perception systems to have flexible and powerful modeling capabilities. While single, fixed architectures perform reliably under controlled conditions, they often degrade under domain shifts or task transitions. Recent studies have introduced dynamic convolution and deformable attention to enable adaptive parameter adjustment based on input features, thereby improving robustness ( Chen et al. 2020 ; Xia et al. 2022 ; He et al. 2022 ) . Nevertheless, these efforts primarily operate at the fine-grained module level within a single architecture and do not offer macro-level architectural diversity or dynamic path scheduling. The need remains for a mechanism that enables both structural diversity and dynamic selection, while maintaining inference efficiency—an important challenge in autonomous perception research. Mixture-of-Experts (MoE) Architectures Originally proposed by Jacobs et al., the Mixture-of-Experts (MoE) architecture enables dynamic expert selection based on input-dependent gating, enhancing representational capacity while controlling computational cost ( Jacobs et al. 1991 ) . In NLP, models such as GShard and Switch Transformers have demonstrated that sparsely activated experts can scale model capacity linearly without incurring heavy inference overhead ( Lepikhin et al. 2020 ; Fedus, Zoph, and Shazeer 2022 ) . Recently, DeepSeekMoE further pushes this boundary by scaling MoE models to over 100 billion parameters while maintaining efficiency through expert sparsity and optimized routing ( Dai et al. 2024 ) . In vision, Vision MoE has shown strong performance in tasks such as classification and detection by incorporating expert modules ( Riquelme et al. 2021 ) . However, systematic MoE integration into autonomous driving’s multi-modal BEV perception remains largely unexplored. Challenges include designing appropriate expert combinations, efficient gating, and maintaining