VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving

Lei Wan, Prabesh Gupta, Andreas Eich, Marcel Kettelgerdes, Hannan Ejaz Keen, Michael Klöppel-Gersdorf, Alexey Vinel · 2025

Read VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving by Lei Wan, Prabesh Gupta, Andreas Eich, Marcel Kettelgerdes, Hannan Ejaz Keen, Michael Klöppel-Gersdorf, Alexey Vinel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06980.txt.

Introduction

Lei Wan and co-authors present a cooperative sensing system for self-driving cars that pools views across vehicles and road posts, in work from 2025. Single cars see too little: range is short, views get blocked, and one bad sensor can blind the system. Weather and dim light make it worse.

Their rig fuses laser, radar, color, and heat cameras on both cars and roadside poles. Heat cameras help spot people at night or in glare. Roadside views fill the gaps that block a car's own sight. A health monitor watches each sensor and flags faults early. The team built the system and tested it on a real-world track, reporting gains in reach and robustness.

Catalog record

CREATORLei Wan, Prabesh Gupta, Andreas Eich, Marcel Kettelgerdes, Hannan Ejaz Keen, Michael Klöppel-Gersdorf, Alexey Vinel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06980

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving Lei Wan, Prabesh Gupta, Andreas Eich, Marcel Kettelgerdes, Hannan Ejaz Keen, Michael Klöppel-Gersdorf, Alexey Vinel Abstract Perception is a core capability of automated vehicles and has been significantly advanced through modern sensor technologies and artificial intelligence. However, perception systems still face challenges in complex real-world scenarios. To improve robustness against various external factors, multi-sensor fusion techniques are essential, combining the strengths of different sensor modalities. With recent developments in V2X ( V2X ) communication, sensor fusion can now extend beyond a single vehicle to a cooperative multi-agent system involving CAV ( CAV ) and intelligent infrastructure. This paper presents VALISENS, an innovative multi-sensor system distributed across multiple agents. It integrates onboard and roadside LiDARs, radars, thermal cameras, and RGB cameras to enhance situational awareness and support cooperative automated driving. The thermal camera adds critical redundancy for perceiving VRU ( VRU ), while fusion with roadside sensors mitigates visual occlusions and extends the perception range beyond the limits of individual vehicles. We introduce the corresponding perception module built on this sensor system, which includes object detection, tracking, motion forecasting, and high-level data fusion. The proposed system demonstrates the potential of cooperative perception in real-world test environments and lays the groundwork for future C-ITS ( C-ITS ) applications. I INTRODUCTION The transportation system is undergoing a significant revolution, from isolated, manually operated vehicles and infrastructure toward a highly digitalized, intelligent, and connected ecosystem. At the core of this evolution are C-ITS ( C-ITS ), which integrate CAV ( CAV )s with intelligent infrastructure. These components are increasingly equipped with advanced sensors, high-performance computing units, and V2X ( V2X ) communication units, enabling them to perceive their surroundings and exchange data in real time. This shared situational awareness improves both autonomous navigation and infrastructure-supported transportation management. To achieve robust perception, CAV s typically rely on a combination of heterogeneous sensors, including LiDAR, radar, and RGB cameras [ 1 ] . LiDAR provides accurate 3D spatial measurements, radar offers robustness in adverse weather conditions, and RGB cameras contribute rich semantic information. Through multi-sensor fusion, vehicles can maintain reliable perception across varying environment, such as different lighting or weather conditions. However, the perception capabilities of an individual vehicle remain constrained in complex traffic environments, particularly due to limited sensing range and occlusions caused by other objects [ 2 ] . Recent advancements in V2X communication have made it possible to extend perception beyond the individual vehicle by enabling the sharing of sensor data among vehicles and infrastructure [ 3 ] . This cooperative perception paradigm significantly enhances the detection range and mitigates the impact of visual occlusions, contributing to improved safety and traffic efficiency. As a result, a dynamic, decentralized sensor network can be established, comprising multiple sensor-equipped entities, that supports a comprehensive and continuously updated understanding of the surrounding environment. This paper presents VALISENS, an innovative and integrated sensor system designed to support cooperative automated driving. The system combines on-board and roadside sensing units, including LiDARs, radars, RGB cameras, and thermal cameras, to enhance perception capabilities in complex urban environments. Compared to conventional in-vehicle sensor systems, the inclusion of thermal cameras provides additional benefits for detecting and protecting VRU s, particularly in low-visibility conditions. Furthermore, the V2X -enabled architecture facilitates the extension of the sensor network beyond individual vehicles, enabling flexible and scalable collaborative perception. To evaluate the proposed system, a perception module is developed that comprises components of object detection, tracking, and motion prediction. These modules are deployed and validated in real-world test environments using two sensor-equipped vehicles and infrastructure-based sensor units. The evaluation demonstrates the effectiveness of the VALISENS system in improving situational awareness and enhancing road safety, especially for VRU protection. (a) Camera node (b) LiDAR node and radar node (c) Sensor layout Fig. 1 : Sensor layout at the SmartIntersection in Dresden. The red circle shows the detection range of the LiDAR, the yellow ellipses show the detection area of the single radar sensors, and the blue (thermal) and green (thermal+RGB) polygons show the area covered by cameras. I-A Related Work I-A 1 Communication Communication plays a central role in C-ITS , enabling real-time cooperation between vehicles, infrastructure, and other road users. DSRC ( DSRC ) and cellular networks are two primary technologies used for message exchange [ 4 ] . DSRC allows direct communication between devices with low latency, making it suitable for safety-critical applications [ 5 ] . However, its limited bandwidth restricts the range of supported services. To address these limitations, C-V2X ( C-V2X ) has emerged as a complementary technology, offering greater bandwidth and supporting data-intensive applications such as cooperative perception. To enhance interoperability and deployment flexibility, dual-mode V2X modules supporting both DSRC and C-V2X are now commercially available [ 6 ] . The system proposed in this paper uses such a dual compatible V2X stack to facilitate testing of both communication protocols and to ensure interoperability between heterogeneous vehicles of different manufacturers. I-A 2 Perception Perception for autonomous vehicles has been extensively studied in recent years, evolving from single-sensor approaches to multi-sensor fusion systems, and from isolated in-vehicle or roadside sensors to dynamic, multi-agent sensor networks. In-vehicle sensor systems have been the foundation of many large-scale vehicle-perspective datasets such as KITTI [ 7 ] , nuScenes [ 8 ] , and Waymo Open Dataset [ 9 ] , supporting a wide range of perception tasks including object detection, tracking, semantic segmentation, and motion prediction. Recently, there has been a growing interest in roadside sensor systems, as reflected in the increasing number of roadside perception data sets such as Rope3D [ 10 ] , A9-Dataset [ 11 ] , and R-LiViT [ 12 ] . The integration of sensor data from both vehicles and infrastructure has led to the development of collaborative perception datasets such as DAIR-V2X [ 13 ] , V2X-Seq [ 14 ] , V2V4Real [ 15 ] , and V2X-Real [ 16 ] . Collectively, these datasets underscore the growing focus on multi-agent sensor networks that incorporate diverse modalities across vehicles and infrastructure. In such networks, data fusion is a key component for generating a unified and consistent understanding of the environment. Fusion strategies are commonly categorized into early fusion [ 17 ] , intermediate fusion [ 18 ] , and late fusion [ 19 ] . Among these, late fusion, where object-level information is exchanged and aggregated, has emerged as the most practical approach for deployment. It is model- and modality-agnostic and significantly reduces bandwidth requirements compared to early fusion, making it better aligned with the realistic constraints of V2X communication. The system proposed in this paper adopts a late fusion approach to enable modular perception across heterogeneous sensors and ensure compatibility with communication standards defined by ETSI ( ETSI ). I-A 3 Sensor Condition Monitoring The performance of a multi-modal perception system is directly dependent on error-free operation and data integrity of the underlying sensor hardware. As compiled by Goelles et al. [ 20 ] and Secci et al. [ 21 ] , especially in the case of optical sensors such as cameras and LiDAR, a multitude of failure mechanisms can, however, lead to severely degraded performance in terms of output

Read the complete text