Multimodal HD Mapping for Intersections by Intelligent Roadside Units

Zhongzhang Chen, Miao Fan, Shengtong Xu, Mengmeng Yang, Kun Jiang, Xiangzeng Liu, Haoyi Xiong · 2025

Read Multimodal HD Mapping for Intersections by Intelligent Roadside Units by Zhongzhang Chen, Miao Fan, Shengtong Xu, Mengmeng Yang, Kun Jiang, Xiangzeng Liu, Haoyi Xiong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08903.txt.

Introduction

Zhongzhang Chen, Miao Fan, and co-workers mapped intersections from the roadside instead of the car in a 2025 paper. High-definition maps guide self-driving vehicles, but a car's own sensors see intersections through occlusions and at glancing angles. Elevated intelligent roadside units, fixed camera and LiDAR installations, watch the same junction from above with a stable view. The authors built a fusion setup for that vantage and a data set to develop it on, RS-seq. Extending a current vehicle-to-everything data set with roadside imagery, point clouds. Vector maps for seven intersections labeled down to lane dividers, crossings, and stop lines.

Their two-stage fusion, extracting traits per sensor then integrating semantics across modalities, beat single-sensor methods consistently in review. Roadside infrastructure, the work suggests, can maintain the maps vehicles merely consume.

Catalog record

CREATORZhongzhang Chen, Miao Fan, Shengtong Xu, Mengmeng Yang, Kun Jiang, Xiangzeng Liu, Haoyi Xiong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08903

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multimodal HD Mapping for Intersections by Intelligent Roadside Units Thanks: *Corresponding author: Miao Fan ([email protected]), professor at Beijing Institute of Graphic Communication, senior member of IEEE. Zhongzhang Chen, Miao Fan, Shengtong Xu, Mengmeng Yang, Kun Jiang, Xiangzeng Liu, Haoyi Xiong Abstract High-definition (HD) semantic mapping of complex intersections poses significant challenges for traditional vehicle-based approaches due to occlusions and limited perspectives. This paper introduces a novel camera-LiDAR fusion framework that leverages elevated intelligent roadside units (IRUs). Additionally, we present RS-seq, a comprehensive dataset developed through the systematic enhancement and annotation of the V2X-Seq dataset. RS-seq includes precisely labelled camera imagery and LiDAR point clouds collected from roadside installations, along with vectorized maps for seven intersections annotated with detailed features such as lane dividers, pedestrian crossings, and stop lines. This dataset facilitates the systematic investigation of cross-modal complementarity for HD map generation using IRU data. The proposed fusion framework employs a two-stage process that integrates modality-specific feature extraction and cross-modal semantic integration, capitalizing on camera high-resolution texture and precise geometric data from LiDAR. Quantitative evaluations using the RS-seq dataset demonstrate that our multimodal approach consistently surpasses unimodal methods. Specifically, compared to unimodal baselines evaluated on the RS-seq dataset, the multimodal approach improves the mean Intersection-over-Union (mIoU) for semantic segmentation by 4% over the image-only results and 18% over the point cloud-only results. This study establishes a baseline methodology for IRU-based HD semantic mapping and provides a valuable dataset for future research in infrastructure-assisted autonomous driving systems. I Introduction Semantic HD maps are essential for autonomous driving, as they provide precise road location details through semantic features such as lane dividers, stop lines, and pedestrian crossings. Recent advancements in deep learning technologies have enabled the automated generation of semantic HD maps by integrating camera images and LiDAR point clouds collected by vehicles. Several studies [ 1 , 2 , 3 ] have investigated methods for fusing these modalities to facilitate HD map production. However, despite these technological advancements, creating HD maps requires significant manual intervention, especially at intersections with complex environments and numerous road elements. These challenges hinder production efficiency. Additionally, the limited perspective of vehicle-mounted sensors requires multiple data collection trips to the same intersection, further increasing the cost of HD map generation. Traditional approaches also struggle to update road elements at intersections promptly road elements at intersections, reducing the applicability of HD maps in autonomous driving systems that require real-time accuracy. Fig. 1: Multimodal data acquisition and semantic HD map generation framework utilizing intelligent roadside units: integrating camera imagery and LiDAR point clouds for comprehensive semantic mapping. With the advancement and application of collaborative sensing technologies, efforts have been directed towards incorporating IRUs into creating semantic HD maps at intersections. Positioned at elevated locations, IRUs provide an expanded field of view and reduce obstructions, enabling the generation of more comprehensive semantic HD maps for these critical areas. Relying exclusively on LiDAR point clouds during HD map creation poses challenges such as difficulty in labelling and the absence of texture information. Conversely, exclusive dependence on camera imagery reduces accuracy and increases sensitivity to lighting conditions. Multimodal approaches that integrate LiDAR point clouds with camera imagery leverage the strengths of both modalities, enhancing the completeness and precision of semantic HD maps. To advance infrastructure-assisted HD mapping research, we systematically enhanced the V2X-seq dataset [ 4 ] by annotating key road elements, including lane dividers, pedestrian crossings, and stop lines, in both camera images and LiDAR point clouds. This enhanced dataset, RS-seq, facilitates the systematic investigation of cross-modal complementarity in semantic HD map generation. Our findings show that camera images capture high-resolution texture and colour information crucial for detecting lane dividers. At the same time, LiDAR point clouds offer precise geometric data valuable for identifying pedestrian crossings and stop lines. As illustrated in Fig. 1 , integrating these complementary features significantly improves mapping accuracy. Quantitative evaluations indicate that our multimodal approach consistently surpasses unimodal methods across diverse intersection scenarios. Using the RS-seq dataset, our approach achieves a 4% improvement in mIoU for semantic segmentation compared to the best-performing unimodal baseline. The key contributions of this work are summarized as follows: • We developed the RS-seq dataset, a richly annotated multimodal dataset that extends the V2X-seq dataset by systematically labelling camera images and LiDAR point clouds. This dataset uses HD semantic map construction data from roadside units. • Based on the RS-seq dataset, we investigated multimodal mapping methods for roadside data. We proposed a baseline method for generating HD semantic maps by fusing camera and LiDAR data. • We release the dataset and source codes at OneDrive 1 1 1 https://1drv.ms/f/c/76645c25a8914a0b/Eub_ulaT3KhKkiZgj31IKyQBkAtRiUZ9hkBvjghSpEjipg to facilitate reproducibility and further research on infrastructure-assisted mapping. II Related Work HD semantic mapping of urban intersections presents significant challenges for vehicle-based approaches due to occlusions and limited perspectives. Recent research has explored intelligent roadside infrastructure and multimodal sensor fusion as solutions to these limitations. This section reviews advancements in infrastructure-assisted mapping methods, sensor fusion techniques, and intersection-specific datasets. Fig. 2: The systematic framework for semantic HD map generation through intelligent roadside unit data processing: a pipeline from multimodal data acquisition to vectorized map representation. II-A Semantic Map Construction In the traditional offline HD map production process, a substantial amount of LiDAR point cloud data is initially collected using vehicle-mounted sensors. This raw data is subsequently processed through Simultaneous Localization and Mapping (SLAM) technology [ 5 ] , a procedure known for its considerable computational demands and algorithmic complexity, which ensures the accurate alignment and integration of the LiDAR point clouds. Following this, the LiDAR point clouds undergo segmentation and vectorization, and the final map data requires manual validation and correction to ensure accuracy and reliability. The conventional offline mapping approach is characterized by high costs and slow update cycles, rendering it less adaptable and scalable. Recent studies have investigated real-time mapping methods that integrate images from surround-view cameras with LiDAR point clouds to overcome these limitations and lessen dependence on manually annotated semantic maps. Notably, HDMapNet [ 1 ] leverages Multi-Layer Perceptrons (MLPs) to map photometric and geometric (PV) features to Bird’s Eye View (BEV) features within the camera branch while employing PointPillars [ 6 ] to encode BEV features in the LiDAR branch. Similarly, BEVFusion [ 7 ] uses Lift-Splat-Shoot (LSS) [ 8 ] for view conversion in the camera branch and VoxelNet [ 9 ] for the LiDAR branch, with integration of these modalities achieved through the BEV alignment module. Superfusion [ 10 ] further advances the field by integrating multilevel feature information from camera images and LiDAR point clouds to create long-range HD maps. However, these real-time mapping techniques are constrained by the limited field of view and occlusions from in-vehicle sensors, as well as variations in data quality caused by rapid sensor motion, which can lead to vulnerabilities and incompleteness in current online map construction schemes. II-B Roadside Unit Mapping Relying solely on vehicles for road data collection poses limitations due to the vehicle’s restricted field of view. For instance, certain road elements may remain unrecorded at intersections because they are occluded by other vehicles. However, advancements in collaborative perception technology have demonstrated that intelligent roadside units installed at elevated positions can enhance data collected by vehicles.

Read the complete text