MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation
Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, Zhenliang Ni · 2025
Read MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation by Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, Zhenliang Ni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24243.txt.
Introduction
Fuqiang Gu and five colleagues fuse two kinds of camera data for scene segmentation. Ordinary cameras fail in hard conditions. Fast motion, low light, and strong contrast defeat them. Event cameras handle those conditions well. They report changes with fine timing and low latency. But they lack color and texture. Neither sensor suffices alone.
Their framework, MambaSeg, runs two branches in parallel. Mamba encoders process the image stream and the event stream. An interaction module then fuses the two. It works along the spatial dimension and the temporal dimension at once. That joint fusion reduces cross-modal ambiguity. It also aligns the modalities more finely. Two driving datasets hosted the experiments. The method reached state-of-the-art segmentation scores there. It did so at significantly lower computational cost.
Catalog record
| CREATOR | Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, Zhenliang Ni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24243
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation Fuqiang Gu, Yuanke Li, Xianlei Long, Kangping Ji, Chao Chen, Qingyi Gu, Zhenliang Ni Abstract Semantic segmentation is a fundamental task in computer vision with wide-ranging applications, including autonomous driving and robotics. While RGB-based methods have achieved strong performance with CNNs and Transformers, their effectiveness degrades under fast motion, low-light, or high dynamic range conditions due to limitations of frame cameras. Event cameras offer complementary advantages such as high temporal resolution and low latency, yet lack color and texture, making them insufficient on their own. To address this, recent research has explored multimodal fusion of RGB and event data; however, many existing approaches are computationally expensive and focus primarily on spatial fusion, neglecting the temporal dynamics inherent in event streams. In this work, we propose MambaSeg, a novel dual-branch semantic segmentation framework that employs parallel Mamba encoders to efficiently model RGB images and event streams. To reduce cross-modal ambiguity, we introduce the Dual-Dimensional Interaction Module (DDIM), comprising a Cross-Spatial Interaction Module (CSIM) and a Cross-Temporal Interaction Module (CTIM), which jointly perform fine-grained fusion along both spatial and temporal dimensions. This design improves cross-modal alignment, reduces ambiguity, and leverages the complementary properties of each modality. Extensive experiments on the DDD17 and DSEC datasets demonstrate that MambaSeg achieves state-of-the-art segmentation performance while significantly reducing computational cost, showcasing its promise for efficient, scalable, and robust multimodal perception. Code — https://github.com/CQU-UISC/MambaSeg Introduction Semantic segmentation is a fundamental task in computer vision with broad applications in autonomous driving, robotics, and scene understanding ( Lateef and Ruichek 2019 ) . Driven by advances in convolutional neural networks (CNNs) and Transformers, RGB image-based methods have achieved significant progress ( Badrinarayanan, Kendall, and Cipolla 2017 ; Xie et al. 2021 ; Ni et al. 2024 ; Ma, Ni, and Chen 2024a ) . However, their reliance on conventional image sensors poses inherent limitations under adverse conditions such as high-speed motion, low illumination, and high dynamic range scenes, where motion blur and latency severely degrade performance ( Rebecq et al. 2019 ) . Event cameras offer a compelling alternative by asynchronously capturing per-pixel intensity changes with microsecond latency, high temporal resolution, and wide dynamic range ( Gallego et al. 2020 ) . These properties make them particularly well-suited for dynamic and low-light environments ( Gehrig and Scaramuzza 2024 ) . Yet, event streams lack color and fine-grained texture information, limiting their standalone utility in dense prediction tasks such as semantic segmentation ( Sun et al. 2022 ; Jia et al. 2023 ) . To harness the complementary strengths of both modalities, recent research has focused on RGB-event fusion techniques that combine the spatial richness of images with the temporal precision of events. Figure 1: Comparison of event-image fusion segmentation methods in terms of mIoU (%), Parameters (M), and multiply-accumulate operations (G). State-of-the-art multimodal methods typically adopt Transformer-based backbones for joint feature modeling ( Zhang et al. 2023a ; Zhang et al. 2023b ) . These models leverage cross-attention mechanisms for modality interaction and often incorporate domain adaptation to transfer semantics from labeled image data to sparsely labeled event streams ( Sun et al. 2022 ; Xie, Gao, and Guo 2024 ) . While effective, such approaches are computationally demanding due to the quadratic complexity of self-attention and the high dimensionality of combined inputs. Moreover, most existing fusion strategies emphasize spatial-level alignment while underutilizing the unique temporal dynamics of event data, which can result in suboptimal cross-modal synergy and semantic inconsistency. To overcome these challenges, we draw inspiration from Mamba ( Gu and Dao 2023 ) , a recently proposed state space model that supports input-conditioned, long-range sequence modeling with linear computational complexity. Mamba has demonstrated strong performance and scalability in a variety of vision tasks, including classification ( Liu et al. 2024 ; Ma, Ni, and Chen 2024b ) , detection ( Wang et al. 2024 ) , and segmentation ( Ruan, Li, and Xiang 2024 ) . Motivated by these advantages, we present MambaSeg, a novel dual-branch semantic segmentation framework that leverages parallel Mamba encoders to model RGB images and event streams independently, enabling efficient and scalable multimodal representation learning. To reduce cross-modal ambiguity, we introduce the Dual-Dimensional Interaction Module (DDIM) to enhance the information exchange between different modalities, which fuses image and event features along both spatial and temporal dimensions. DDIM is composed of two key components: the Cross-Spatial Interaction Module (CSIM), which aligns spatial semantics by integrating dense texture features from images with structural edge cues from events; and the Cross-Temporal Interaction Module (CTIM), which exploits Mamba’s global modeling capacity to refine temporal dependencies via attention-guided fusion. This design ensures fine-grained, modality-aware feature integration while minimizing computational overhead. We validate MambaSeg on two public benchmarks, DDD17 and DSEC. Our method achieves 77.56% mIoU on DDD17 and 75.11% mIoU on DSEC, setting new SOTA performance while maintaining high efficiency. These results highlight the strength of our MambaSeg for multimodal segmentation. Our main contributions are summarized as follows: • We propose MambaSeg, a dual-branch semantic segmentation framework based on parallel Mamba encoders that model image and event modalities efficiently with long-range dependency modeling and linear complexity. • We design the DDIM module, which includes CSIM and CTIM for structured spatial-temporal fusion. This module jointly leverages Mamba and attention to enhance cross-modal feature alignment and reduce ambiguity. • We conduct extensive experiments on the DDD17 and DSEC datasets, where MambaSeg achieves SOTA performance while significantly reducing computational cost compared to Transformer-based baselines. Related Work Semantic Segmentation RGB-based semantic segmentation has advanced significantly with CNNs and Transformers. Encoder-decoder architectures like SegFormer ( Xie et al. 2021 ) and SegNeXt ( Guo et al. 2022 ) capture both spatial details and high-level semantics, achieving SOTA results. However, their dependence on conventional sensors limits robustness in fast motion, low-light, and high dynamic range scenarios, where motion blur and latency degrade performance. To address these issues, event cameras have attracted interest for their high temporal resolution, low latency, and wide dynamic range. EV-SegNet ( Alonso and Murillo 2019 ) pioneered event-based segmentation. Due to scarce labeled event data, methods like ESS ( Sun et al. 2022 ) and CMESS ( Xie, Gao, and Guo 2024 ) employ unsupervised domain adaptation to transfer semantic knowledge from images. Recent Transformer-based models such as EvSegFormer ( Jia et al. 2023 ) integrate motion priors into attention to exploit event dynamics. Spiking neural networks (SNNs), including Spike-BRGNet ( Long et al. 2024 ) and SLTNet ( Long et al. 2025 ) , have also been used to model temporal information efficiently. RGB-event fusion leverages complementary spatial and temporal cues for robust perception. Hybrid frameworks like HALSIE ( Das Biswas et al. 2024 ) and SpikingEDN ( Zhang et al. 2024 ) combine ANNs and SNNs to capture static textures and dynamic edges. RGB-X methods such as CMX ( Zhang et al. 2023a ) and CMNeXt ( Zhang et al. 2023b ) employ multi-scale alignment and cross-attention for spatial fusion. EISNet ( Xie et al. 2024 ) further introduces gated attention and progressive recalibration for adaptive feature alignment. However, many methods rely on computationally expensive Transformer attention and suffer from cross-modal ambiguity. To address this, we propose a