End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy
Zifan Wang, Xun Yang, Jianzhuang Zhao, Jiaming Zhou, Teli Ma, Ziyao Gao, Arash Ajoudani, Junwei Liang · 2025
Read End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy by Zifan Wang, Xun Yang, Jianzhuang Zhao, Jiaming Zhou, Teli Ma, Ziyao Gao, Arash Ajoudani, Junwei Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07611.txt.
Introduction
Zifan Wang and seven co-authors teach a humanoid robot to walk safely and politely among people, in a 2025 paper. Real spaces are cluttered and shifting. Blind controllers cannot see obstacles, and camera systems miss 3D depth.
The policy reads raw LiDAR point clouds and outputs motor commands directly. Safety was written into training as hard costs, using barrier function ideas. Extra rewards favor smooth, predictable motion that does not crowd people. The trained policy moved from simulation onto a real humanoid. The robot threaded past still and moving obstacles with ease. The work joins provable safety with social grace in one controller.
Catalog record
| CREATOR | Zifan Wang, Xun Yang, Jianzhuang Zhao, Jiaming Zhou, Teli Ma, Ziyao Gao, Arash Ajoudani, Junwei Liang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07611
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy Zifan Wang, Xun Yang, Jianzhuang Zhao, Jiaming Zhou, Teli Ma, Ziyao Gao, Arash Ajoudani, Junwei Liang Abstract The deployment of humanoid robots in unstructured, human-centric environments requires navigation capabilities that extend beyond simple locomotion to include robust perception, provable safety, and socially aware behavior. Current reinforcement learning approaches are often limited by blind controllers that lack environmental awareness or by vision-based systems that fail to perceive complex 3D obstacles. In this work, we present an end-to-end locomotion policy that directly maps raw, spatio-temporal LiDAR point clouds to motor commands, enabling robust navigation in cluttered dynamic scenes. We formulate the control problem as a Constrained Markov Decision Process (CMDP) to formally separate safety from task objectives. Our key contribution is a novel methodology that translates the principles of Control Barrier Functions (CBFs) into costs within the CMDP, allowing a model-free Penalized Proximal Policy Optimization (P3O)—to enforce safety constraints during training. Furthermore, we introduce a set of comfort-oriented rewards, grounded in human-robot interaction research, to promote motions that are smooth, predictable, and less intrusive. We demonstrate the efficacy of our framework through a successful sim-to-real transfer to a physical humanoid robot, which exhibits agile and safe navigation around both static and dynamic 3D obstacles. Project Page: https://github.com/aCodeDog/SafeHumanoidsPolicy Keywords: Humanoid Robot, Locomotion, Reinforcement Learning, Collision Avoidance, LiDAR Perception I INTRODUCTION The vision of humanoid robots seamlessly coexisting and collaborating with people in everyday environments presents a grand challenge for robotics. A fundamental prerequisite for this vision is the ability to navigate complex, human-centric spaces safely and efficiently. This requires more than just dynamic locomotion; it demands a holistic integration of 3D perception, principled safety, and social awareness, an area where current controllers often fall short. A significant body of state-of-the-art research in legged locomotion has relied on reinforcement learning (RL) to develop controllers that are ”blind,” using only proprioceptive feedback [ 1 , 2 , 3 ] . While these methods have achieved remarkable agility on flat or moderately uneven terrain, they are inherently incapable of navigating environments with obstacles. To overcome this, recent work has incorporated exteroceptive sensing, primarily using depth cameras to generate 2D height maps for local terrain awareness [ 4 , 5 , 6 , 7 ] .Depth cameras are sensitive to lighting conditions and have a limited field of view [ 8 , 9 , 10 ] .And the reduction of 3D sensory information to a 2D elevation map makes the robot blind to any non-ground-level obstacles, such as overhanging clutter or the upper bodies of other agents [ 11 , 12 , 13 ] . This limitation poses a substantial collision risk for a full-body humanoid robot. We argue that for robust navigation in cluttered indoor spaces, a perception modality that is lighting-invariant and provides direct 3D information is essential. LiDAR sensors meet these criteria, yet their integration into end-to-end locomotion policies remains limited [ 14 , 15 ] . Even with robust perception, ensuring safety is a non-trivial challenge. The common approach of shaping reward functions with penalties for collisions is often brittle, difficult to tune, and can lead to overly conservative or still-unsafe behaviors [ 16 , 7 , 17 ] . Furthermore, for robots intended to operate alongside people, merely avoiding collisions is insufficient. The robot’s motion must also be psychologically comfortable—that is, predictable, fluid, and non-threatening—to foster trust and acceptance [ 18 , 19 , 20 , 21 ] . This higher-level, human-centric aspect of motion planning is rarely considered in locomotion policies. In this work, we address these gaps with an integrated, end-to-end framework. Our contributions are: 1. A LiDAR-driven end-to-end policy that processes raw 3D point clouds to navigate complex environments, overcoming the limitations of blind and 2D-vision-based approaches. 2. A principled safety framework based on Constrained Reinforcement Learning (CMDP). We introduce a novel method to translate model-based Control Barrier Function (CBF) principles into costs for a model-free RL algorithm, P3O, enabling robust safety enforcement. 3. A comfort-oriented reward structure explicitly designed to produce socially aware motions by penalizing behaviors known to cause human discomfort, such as high approach speeds and unpredictable movements, as identified in HRI research. 4. Successful real-world deployment on a humanoid robot, demonstrating agile and robust avoidance of diverse static and dynamic 3D obstacles in cluttered environments. II RELATED WORK II-A Legged Robot Locomotion and Perception RL has produced policies capable of remarkable dynamic skills, yet many still lack the environmental perception needed for navigation in cluttered spaces [ 22 , 5 , 23 ] . Recent work has begun to incorporate exteroceptive sensing. Policies using height maps derived from depth cameras or LiDAR have successfully traversed uneven terrain [ 7 , 4 , 24 , 25 , 14 ] . II-B Safe Reinforcement Learning Ensuring safety during RL is critical for real-world deployment [ 26 , 27 ] . A common but often unreliable method is to use negative rewards to penalize unsafe actions. A more structured approach is to formulate the problem as a CMDP, which separates the task objective (reward) from safety specifications (constraints) [ 28 , 29 ] . This allows for the use of specialized algorithms that aim to satisfy constraints while maximizing rewards. Within this domain, Control-Theoretic Methods offer strong safety guarantees. CBFs define a safe region of the state space and can be used to synthesize controllers that are guaranteed to remain within it [ 30 , 31 ] . However, classical CBF approaches typically require an accurate analytical model of the system dynamics, making them difficult to apply directly in model-free RL. Our work bridges this gap. We draw inspiration from CBF theory to define a safety condition but implement it as a cost function within a CMDP inspired by [ 32 , 33 ] . This allows us to use a powerful, model-free constrained policy optimization algorithm, P3O [ 28 ] , to learn a safe policy. P3O is a first-order method, making it more computationally efficient than higher-order alternatives like CPO , and its use of normalized advantage functions improves training stability. By integrating CBF principles into a proven, practical CMDP algorithm, we develop a robust framework for learning safe, perception-driven locomotion. III Definitions III-A Constrained Markov Decision Process The task of learning a safe locomotion policy is formulated as a CMDP which is defined by a tuple (\mathcal{S},\mathcal{A},P,R,\{C_{j}\},\{\epsilon_{j}\},\gamma_{\text{RL}}) , where: • \mathcal{S} is the set of states s_{k} . • \mathcal{A} is the set of actions a_{k} (equivalent to u_{k} ). • P(s_{k+1}|s_{k},a_{k}) is the state transition probability. • R(s_{k},a_{k},s_{k+1}) is the reward function. • C_{j}(s_{k},a_{k},s_{k+1}) is the j -th instantaneous cost function. • \epsilon_{j} is the threshold for the j -th constraint. • \gamma_{\text{RL}}\in[0,1) is the discount factor for reinforcement learning. The objective is to find a policy \pi:\mathcal{S}\rightarrow\mathcal{P(A)} that maximizes the expected discounted sum of rewards, J_{R}(\pi)=\mathbb{E}_{\tau\sim\pi}[\sum_{t=0}^{\infty}\gamma^{t}R(s_{t},a_{t},s_{t+1})] , subject to constraints on the expected discounted sum of costs: \begin{gathered}J_{C_{j}}(\pi)=\mathbb{E}_{\tau\sim\pi}\left[\sum_{t=0}^{\infty}\gamma^{t}C_{j}(s_{t},a_{t},s_{t+1})\right]\leq\epsilon_{j},\\ \quad\forall j\in\{1,...,m\}.\end{gathered} (1) We partition the state space \mathcal{S} into a set of safe states , denoted \mathcal{S}_{\text{safe}} , and a set of unsafe states , denoted \mathcal{S}_{\text{unsafe}} , such that \mathcal{S}=\mathcal{S}_{\text{safe}}\cup\mathcal{S}_{\text{unsafe}} and \mathcal{S}_{\text{safe}}\cap\mathcal{S}_{\text{unsafe}}=\emptyset . The set \mathcal{S}_{\text{safe}} represents configurations where the system can operate without risk,