Enhancing Indoor Mobility with Connected Sensor Nodes: A Real-Time, Delay-Aware Cooperative Perception Approach
Minghao Ning, Yaodong Cui, Yufeng Yang, Shucheng Huang, Zhenan Liu, Ahmad Reza Alghooneh, Ehsan Hashemi, Amir Khajepour · 2024
Read Enhancing Indoor Mobility with Connected Sensor Nodes: A Real-Time, Delay-Aware Cooperative Perception Approach by Minghao Ning, Yaodong Cui, Yufeng Yang, Shucheng Huang, Zhenan Liu, Ahmad Reza Alghooneh, Ehsan Hashemi, Amir Khajepour (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02624.txt.
Description
Abstract This paper presents a novel real-time, delay-aware cooperative perception system designed for intelligent mobility platforms operating in dynamic indoor environments. The system contains a network of multi-modal sensor nodes and a central node that collectively provide perception services to mobility platforms.
Catalog record
| CREATOR | Minghao Ning, Yaodong Cui, Yufeng Yang, Shucheng Huang, Zhenan Liu, Ahmad Reza Alghooneh, Ehsan Hashemi, Amir Khajepour |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02624
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Indoor Mobility with Connected Sensor Nodes: A Real-Time, Delay-Aware Cooperative Perception Approach Minghao Ning, Yaodong Cui, Yufeng Yang, Shucheng Huang, Zhenan Liu, Ahmad Reza Alghooneh, Ehsan Hashemi, Amir Khajepour Abstract This paper presents a novel real-time, delay-aware cooperative perception system designed for intelligent mobility platforms operating in dynamic indoor environments. The system contains a network of multi-modal sensor nodes and a central node that collectively provide perception services to mobility platforms. The proposed Hierarchical Clustering Considering the Scanning Pattern and Ground Contacting Feature based Lidar Camera Fusion improve intra-node perception for crowded environment. The system also features delay-aware global perception to synchronize and aggregate data across nodes. To validate our approach, we introduced the Indoor Pedestrian Tracking dataset, compiled from data captured by two indoor sensor nodes. Our experiments, compared to baselines, demonstrate significant improvements in detection accuracy and robustness against delays. The dataset is available in the repository 1 1 1 https://github.com/NingMingHao/MVSLab-IndoorCooperativePerception . I Introduction In recent years, intelligent indoor autonomy technology is gaining recognition and attention among healthcare professionals and researchers. Studies have shown that indoor transportation is the most urgent need from healthcare staff in hospitals and long-term care [ 1 ] . This rising demand is largely driven by workforce shortages and the high incidence of chronic injuries among healthcare staff, which often caused by transporting heavy materials. However, large scale commercial deployment of intelligent robotics platforms are still limited. Most existing indoor robots are designed to operate independently, relying on their built-in sensors to navigate and perform tasks. This restricts their effectiveness in the congested, dynamic, and unpredictable spaces of healthcare facilities. This paper presents a cooperative perception system consisting of a network of multiple sensor nodes, and a central node, to provide perception results/services to robotic mobility platforms. This system aimed to improve the operational safety and environmental awareness of intelligent robotic platforms, including autonomous hospital beds and delivery robots. Fig. 1: Overview of the proposed cooperative perception system There are several challenges associated with developing a cooperative perception system in densely populated indoor environments, such as hospitals. One primary challenge for local perception is the fast and accurate fusion of perception data from multiple sensor nodes. This task is complicated by the dynamic behavior of people within a confined space, which involves close interactions between individuals. For instance, people travel in small groups, or crossing paths at close quarters. These situations pose significant difficulties in maintaining consistent tracking identities across different nodes and merging perception data effectively. The physical layout of indoor environments presents another significant challenge for local perception. Architectural features and decorative elements, such as corners, pillars, and mirrors, present significant challenges in achieving continuous and accurate coverage across the entire area. These environmental factors can obstruct the sensor field of view and distort the sensor signal, leading to gaps in coverage or inaccuracies in perception. Fig. 2: The proposed delay-aware cooperative perception framework. The processing and communication delays poses a major challenge for global/cross-sensor perception in highly dynamic indoor environments. These cross-node delays can lead to the receipt of outdated or inaccurate representations of the dynamic environment at the center node. This impairs the center node’s ability to generate a cohesive and current understanding of the environment. To address these challenges, this paper proposes a delay-aware cooperative perception system designed for dynamic indoor environment. An overview of the proposed system is illustrated in Fig. 1 . Our contribution can be summarized as follows: • An adaptive clustering method coupled with ground-contact point-based LiDAR-camera fusion, enhancing the accuracy and reliability of local perception. • A delay-aware global perception framework that accounts for messaging delays and latency, ensuring timely and cohesive environmental understanding. • The creation of a multimodal cooperative indoor perception dataset specifically designed for dynamic and crowded healthcare environments. This provids a valuable resource for further research and development in this field. The rest of the paper is organized as follows, in section II, the related methods and dataset are reviewed, in section III, the overview of our method is presented, in section IV, the experiments and discussion are presented, and finally in section V the impact of our work is concluded. II Related Work II-A Indoor Perception Existing indoor infrastructure-based perception system often relies on basic sensors and cameras, which either lack high-level semantic understanding or precise measurement of object positions. In [ 2 ] , four Pyroelectric Infrared (PIR) sensors are combined as a sensor node and mounted on the ceiling to detect object trajectory. In [ 3 ] Radio frequency identification (RFID) is used to track objects embedded with RFID tags. Although these methods provide basic tracking functionalities, their perception range and accuracy are very limited. In [ 4 , 5 , 6 ] infrastructure-based cameras are used to detect and track pedestrians. However, such standalone pure vision-based systems are sensitive to lighting variations and occlusions and cannot accurately localize objects in 3D space. Alternatively, Brvsvcic et al. leveraged a combination of infrastructure-based RGB-D cameras, LiDAR, and marker-based motion tracking systems [ 7 ] . However, the cost of such setup makes them impractical for large-scale deployment. A more recent study used a motion capture system capable of producing ground truth data at a 100 Hz rate [ 8 ] . Despite its high accuracy, it is limited to areas where motion capture technology is available. These challenges highlight the need for more robust and cost-effective perception systems capable of operating reliably under the complex conditions typical in indoor settings. II-B Indoor Cooperative Perception Dataset TABLE I: Indoor dataset summary (Inf: Infrastructure, Auto: Automated, GT: Ground truth) Dataset Mounting Sensors Annotation KTH [ 9 ] Robot RGB-D, 2D Lidar Auto L-CAS [ 10 ] Robot 3D LiDAR Manual MuSoHu [ 11 ] Wearable RGB/Stereo camera, 3D LiDAR N/A Central station [ 4 ] Inf Camera Auto ATC [ 7 ] Inf RGB-D, Lidar, Motion tracking Auto Thor [ 8 ] Inf Motion capture GT MVSL (Our) Inf Camera, 3D Lidar GT As shown in Table I , existing indoor datasets typically rely on RGB and depth cameras, LiDARs, and motion capture/tracking systems to obtain the position of each object. Despite their utility, these datasets fail to fully capture the scope of indoor environments and dynamics due to the inherent limitations of the technologies employed. For instance, cameras (RGB-D) and LiDAR sensors installed on mobile robots or wearable devices [ 9 , 10 , 11 ] are limited by their range, FOV, and issues like object truncation and occlusions. III Methodology As shown in Fig. 2 , the proposed delay-aware cooperative system comprises two main components: local perception for sensor nodes and delay-aware global perception on a center node. Each sensor node is equipped with dual cameras, a LiDAR sensor, 5G/wireless communication capabilities, and a Jetson Orin NX for edge computing. These nodes process multi-modal sensory data locally to produce tracked object lists. By integrating edge computing capabilities, we aim to reduce the overall system latency. The center node aggregates and combines the structured perception results of the sensor nodes to generate a holistic view of the dynamic indoor environment. This configuration allows for real-time detection and tracking of dynamic elements across multiple nodes in complex indoor