SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes
Minghao Ning, Yufeng Yang, Shucheng Huang, Jiaming Zhong, Keqi Shu, Chen Sun, Ehsan Hashemi, and Amir Khajepour · 2025
Read SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes by Minghao Ning, Yufeng Yang, Shucheng Huang, Jiaming Zhong, Keqi Shu, Chen Sun, Ehsan Hashemi, and Amir Khajepour (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05727.txt.
Introduction
Minghao Ning and co-authors give indoor robots a shared set of eyes so they can move politely among people, in a 2025 paper. A robot's own sensors see little and lose people behind shelves. Wall and ceiling sensors can fill the gaps. Their frame, SAP-CoPE, blends those fixed views with the robot's own to track each person's pose in 3D.
The planner then builds a personal-space field around each person, shaped by their pose and heading. A control method steers the robot to honor both hard safety and soft comfort. Pose checks can also fold in sparse laser points when they are at hand. Real-world runs showed the stack producing smooth, socially aware paths. The authors aim the work at halls, wards, and other shared indoor spaces.
Catalog record
| CREATOR | Minghao Ning, Yufeng Yang, Shucheng Huang, Jiaming Zhong, Keqi Shu, Chen Sun, Ehsan Hashemi, and Amir Khajepour |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05727
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes Minghao Ning, Yufeng Yang, Shucheng Huang, Jiaming Zhong, Keqi Shu, Chen Sun, Ehsan Hashemi, and Amir Khajepour Abstract Note to Practitioners Indoor autonomous mobile robots operating in public facilities such as hospitals must navigate crowded environments where onboard sensing is often challenged by occlusions and limited fields of view, while also accounting for human comfort in addition to collision avoidance. This paper presents SAP-CoPE, a deployable framework that integrates cooperative infrastructure sensing with social-aware motion planning. Ceiling-mounted sensor nodes provide expanded and occlusion-robust perception through uncertainty-aware 3D human pose estimation from cameras, optionally supplemented by sparse LiDAR data. Leveraging these pose estimates, an MPC-based planner incorporating a human personal space field generates trajectories that balance social comfort, efficiency, and safety while respecting the robot’s kinematic constraints. The proposed architecture is modular and compatible with existing indoor robotic platforms using standard hardware, and real-world experiments demonstrate smoother navigation and improved clearance and travel-time performance compared with conventional baseline methods. ©2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. I Introduction Intelligent indoor mobility systems are gaining increasing attention in healthcare, logistics, and the automotive industry [ 1 , 2 , 3 ] . These systems reduce manual transportation tasks, minimize workplace injuries, and improve operational efficiency [ 4 ] . In restricted environments, mobile robots primarily focus on avoiding collisions with objects and are programmed to stop when humans are nearby. However, this strategy is less effective in public areas like hospitals and shopping malls, where interactions with people are unavoidable. As shown in Fig. 1 , to enable safe and efficient human-robot interaction, such a system requires real-time obstacle detection, human state and motion-cue estimation, and a robust motion planning algorithm for safe and smooth navigation. Fig. 1 : Key challenges: (1) How to reliably detect humans in crowded, occluded scenarios? (2) How to guide the robot for human-comfortable motion? Recent research has improved robot perception using high-performance onboard sensors; however, onboard-only systems remain limited by perception range, field of view (FOV), and occlusions in crowded indoor spaces [ 5 ] . Infrastructure-supported perception can mitigate these limitations by observing shared spaces from complementary viewpoints. However, for human-aware navigation, object-level information such as pedestrian positions or bounding boxes is often insufficient, because it provides limited cues about human body orientation, posture, and short-term motion tendency. Articulated 3D human pose provides richer information for social-aware planning. Human heading and body configuration can indicate whether a pedestrian is facing the robot, crossing its path, or beginning to turn. Such cues are useful for constructing an orientation-aware personal-space (PS) field, rather than treating each human as an isotropic obstacle. This is particularly important for large indoor robotic platforms, whose rectangular footprints and kinematic constraints require the planner to balance human comfort, physical safety, travel efficiency, and motion feasibility. This work builds upon our preliminary studies on infrastructure-supported perception and MPC-based indoor robot navigation [ 6 , 7 ] . Those studies established the foundations for cooperative infrastructure perception and physically safe MPC navigation, but they focused mainly on obstacle-level perception and collision-oriented planning. In contrast, this article develops a unified pose-aware and social-aware navigation framework. The proposed SAP-CoPE framework estimates human 3D pose and heading in the world coordinate system using uncertainty-aware camera projection, joint-coherence constraints, and optional sparse LiDAR associations. The estimated pose and heading are then incorporated into an MPC-based planner through an asymmetric PS field, enabling the robot to generate socially acceptable motion while satisfying its kinematic constraints. The key contributions of this article are as follows. 1. Uncertainty-aware and sparse-measurement-compatible 3D human pose estimation: A lightweight optimization-based pose estimation formulation is proposed to account for uncertainty propagation from camera projection while enforcing human joint coherence. The sparse LiDAR term is activated only when reliable LiDAR-to-joint associations are available, allowing the method to operate in both camera-only and camera-LiDAR configurations. 2. Extension from physically safe MPC to social-aware MPC navigation: A human-orientation-dependent PS field is incorporated into the MPC formulation for a large omnidirectional indoor robot. Instead of treating humans only as generic obstacles, the planner uses estimated human pose and heading to generate socially acceptable motion that balances clearance, comfort, travel efficiency, and robot kinematic feasibility. 3. Practical validation and scope demonstration in human-populated indoor environments: The proposed SAP-CoPE framework is validated through simulation and real-world experiments with human participants. The results demonstrate its applicability to infrastructure-supported indoor autonomous mobility tasks, where the robot must navigate efficiently while maintaining human-comfortable interaction distances. The remainder of this article is organized as follows. Section II reviews related work. Section III presents an overview of the proposed methodology. Section IV details the perception algorithm. Section V formulates the social-aware motion planner. Section VI presents the experimental results. Finally, Section VII concludes the paper and discusses future research directions. II Related Work II-A Perception Infrastructure-Based Perception Systems: Reliable perception is fundamental for intelligent indoor mobility systems. Traditional infrastructure-based approaches, particularly those using vision sensors, have been widely explored. Works such as [ 8 , 9 , 10 , 11 ] employ ceiling or wall-mounted cameras to detect and track pedestrians. However, purely independent camera-driven systems can be sensitive to lighting conditions and occlusions, and they are generally limited in their ability to obtain precise 3D object positions without additional depth information. To overcome these limitations, researchers have explored multi-sensor fusion within the infrastructure. Brščić et al. [ 12 ] integrated RGB-D cameras, LiDAR, and marker-based tracking to enhance spatial accuracy and robustness. Though this approach improves performance, its scalability is hindered by high equipment and installation costs. Similarly, high-frequency motion capture systems [ 13 ] offer ground-truth level accuracy but are restricted to controlled spaces, limiting their practical use in real-world deployments. Overall, these limitations emphasize the demand for indoor perception systems that are both reliable under challenging conditions and feasible for cost-effective, large-scale deployment. Cooperative Perception Systems: Cooperative perception improves environmental awareness by integrating sensory data from multiple sources, including other robots and infrastructure. This collaborative approach effectively addresses challenges such as occlusions and the limited FOV inherent in individual sensors. Recent advancements highlight the significance of Vehicle-to-Everything (V2X) in the autonomous driving domain [ 14 , 15 , 16 , 17 ] . Huang et al. [ 15 ] highlight the critical role of data sharing between agents and infrastructure while identifying real-time performance and communication delay as key bottlenecks. Inspired by this concept, we propose a real-time cooperative perception system utilizing ceiling-mounted ISNs. Each node performs local perception and transmits lightweight observations to a central fusion module in real time. This architecture not only expands FOV and reduces occlusions in crowded indoor spaces but also ensures timely and coherent perception by mitigating latency-induced errors in the fusion pipeline. 3D Human Pose Detection: Accurately estimating human 3D poses is vital for autonomous systems to interpret and predict human behaviour, ensuring safe