Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments
Zifan Wang, Teli Ma, Yufei Jia, Xun Yang, Jiaming Zhou, Wenlong Ouyang, Qiang Zhang, Junwei Liang · 2025
Read Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments by Zifan Wang, Teli Ma, Yufei Jia, Xun Yang, Jiaming Zhou, Wenlong Ouyang, Qiang Zhang, Junwei Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19214.txt.
Introduction
Eight authors led by Zifan Wang teach legged robots to dodge hits from all sides with laser sight. Robots on legs must cross rooms and yards full of hanging clutter, rough ground, and moving people. Depth cameras wobble with noise and light, and map steps eat time. Their policy reads raw laser point clouds end to end, with no map in between, and steers the legs straight from that stream.
A new sight module grades risk near and far in one net, from the time flow of laser frames. A high-grade sim rig, with true noise marks and fast ray math, trains the policy at scale. The rig plugs into three known sim worlds for wide tests. Robots trained this way crossed messy scenes with both still and moving blocks in their path. Gains beat camera paths in the team runs.
Catalog record
| CREATOR | Zifan Wang, Teli Ma, Yufei Jia, Xun Yang, Jiaming Zhou, Wenlong Ouyang, Qiang Zhang, Junwei Liang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19214
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments Zifan Wang, Teli Ma, Yufei Jia, Xun Yang, Jiaming Zhou, Wenlong Ouyang, Qiang Zhang, Junwei Liang Abstract Agile locomotion in complex 3D environments requires robust spatial awareness to safely avoid diverse obstacles such as aerial clutter, uneven terrain, and dynamic agents. Depth-based perception approaches often struggle with sensor noise, lighting variability, computational overhead from intermediate representations. In contrast, direct integration of LiDAR sensing into end-to-end learning for legged locomotion remains underexplored. We propose Omni-Perception , an end-to-end locomotion policy that achieves 3D spatial awareness and omnidirectional collision avoidance by directly processing raw LiDAR point clouds. At its core is PD-RiskNet (Proximal-Distal Risk-Aware Hierarchical Network), a novel perception module that interprets spatio-temporal LiDAR data for environmental risk assessment. To facilitate efficient policy learning, we develop a high-fidelity LiDAR simulation toolkit with realistic noise modeling and fast raycasting, compatible with platforms such as Isaac Gym, Genesis, and MuJoCo, enabling scalable training and effective sim-to-real transfer. Learning reactive control policies directly from raw LiDAR data enables the robot to navigate complex environments with static and dynamic obstacles more robustly than approaches relying on intermediate maps or limited sensing. We validate Omni-Perception through real-world experiments and extensive simulation, demonstrating strong omnidirectional avoidance capabilities and superior locomotion performance in highly dynamic environments. Figure 1: Validation scenarios for the Omni-Perception framework. Effective omnidirectional collision avoidance is demonstrated on the left, where the robot reacts to obstacles from various approach vectors. Robustness against diverse environmental features is shown on the right, including successful negotiation of aerial, transparent, slender, and ground obstacles. These results highlight the capacity of the Omni-Perceptio to achieve collision-free locomotion in challenging 3D settings directly from raw LiDAR input. Keywords: Legged Robot, Locomotion, Reinforcement Learning, LiDAR Perception 1 Introduction Legged robots possess the unique potential to navigate complex, human-centric environments where wheeled or tracked systems often fail. Unlocking their full capabilities, however, requires not only agile locomotion but also robust perception and rapid reaction to diverse 3D surroundings [ 1 , 2 ] . Achieving safe, high-speed locomotion among static and dynamic obstacles—ranging from ground clutter and uneven terrain to overhanging structures and moving agents—requires holistic spatial awareness and tightly coupled reactive control [ 3 , 4 ] . Enabling such agile, collision-free movement in unstructured 3D environments remains a fundamental challenge [ 5 , 6 ] . Most state-of-the-art controllers operate blindly using only proprioception, limiting their ability to handle complex terrain or unexpected obstacles [ 7 , 8 , 9 , 2 , 10 , 11 , 12 ] . While methods incorporating exteroception via depth cameras [ 13 , 14 ] have advanced locomotion over uneven terrain, depth-based perception suffers from limitations such as sensitivity to lighting conditions, limited fields of view, noise susceptibility, and the computation overhead of maintaining intermediate representations like elevation maps [ 15 , 16 ] . These representations, in turn, struggle with aerial clutter and complex non-planar obstacles. Alternative approaches that decouple navigation planning from locomotion control [ 17 , 18 ] often lead to conservative behaviors, preventing full exploitation of a robot’s agility. Meanwhile, LiDAR sensors have become instrumental in fields such as autonomous driving, offering dense, lighting-invariant 3D measurements, and manipulation [ 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] . LiDAR delivers rich geometric information ideal for navigating spatially complex environments. Despite its promise, the integration of LiDAR— especially direct point cloud processing —into end-to-end learning for legged locomotion remains underexplored [ 27 , 28 , 29 ] . This gap arises from the challenges of real-time point cloud processing within fast control loops, and from difficulties in accurately modeling LiDAR physics for effective sim-to-real transfer [ 30 ] . This work aims to bridge this critical gap. We posit that directly leveraging raw, spatio-temporal LiDAR point clouds within an end-to-end Reinforcement Learning (RL) policy can unlock robust 3D environmental awareness for legged robots. This enables the robot to achieve omnidirectional collision avoidance and agile navigation without relying on handcrafted intermediate representations or restrictive decoupled planning. We introduce Omni-Perception , a novel framework centered around an end-to-end policy trained using RL in a high-fidelity LiDAR simulator. Our core technical contribution is the Proximal-Distal Risk-Aware Hierarchical Network (PD-RiskNet) , a novel perception architecture that efficiently processes raw spatio-temporal LiDAR streams to assess multi-level environmental risks. To enable scalable and realistic training, we also develop a high-fidelity, cross-platform LiDAR simulation toolkit with realistic noise modeling and efficient parallel raycasting, supporting multiple physics engines including Isaac Gym, Genesis, MuJoCo, and others. By learning directly from 3D spatial data, Omni-Perception enables legged robots to dynamically track velocity commands while avoiding complex, multi-axis threats (e.g., aerial obstacles, ground traps, moving agents), pushing the boundaries of agile and safe navigation in unstructured environments.Our contributions are summarized as follows: 1. End-to-End LiDAR-Driven Locomotion Framework: We present Omni-Perception, the first framework to achieve 3D spatial awareness for legged robots by directly processing raw LiDAR point clouds within an end-to-end RL architecture. 2. Novel LiDAR Perception Network (PD-RiskNet): 3. High-Fidelity LiDAR Simulation Toolkit: We develop a high-performance, cross-platform LiDAR simulation toolkit featuring realistic noise models and fast parallel raycasting, enabling effective zero-shot sim-to-real policy transfer. 4. Demonstration of Robust LiDAR-Driven Agility: We validate Omni-Perception through extensive simulation and real-world experiments, demonstrating strong velocity tracking and omnidirectional avoidance capabilities across diverse and challenging environments containing static and dynamic 3D obstacles. 2 Related Work 2.1 Learning-Based Legged Locomotion Reinforcement Learning (RL) has emerged as a powerful paradigm for developing sophisticated locomotion controllers for legged robots, demonstrating remarkable agility and robustness surpassing traditional model-based methods in many scenarios [ 9 , 31 , 32 ] . RL approaches have successfully generated controllers for high-speed running [ 8 , 11 ] and challenging terrain traversal [ 33 , 7 ] . These policies typically map proprioceptive states—and in some cases, limited exteroceptive information—directly to joint commands. 2.2 Exteroceptive Perception for Locomotion Incorporating environmental perception is crucial for enabling legged robots to move through unknown or dynamic environments. Depth-Based Perception: A significant body of work utilizes depth cameras. Many approaches reconstruct the environment into intermediate representations like 2.5D elevation maps, for foothold planning or policy input [ 34 , 35 ] . However, elevation maps inherently struggle with non-planar obstacles like overhangs or aerial clutter and are sensitive to sensor noise. Other works attempt to use depth data more directly, either by learning end-to-end policies from depth images [ 36 , 37 , 13 , 38 ] or extracting sparse features like ray distances from depth [ 3 ] . Nevertheless, these methods inherit the limitations of depth sensors, including sensitivity to lighting and limited range. Furthermore, sim-to-real transfer often requires extensive domain randomization, particularly noise injection into depth data, to address real-world sensor characteristics [ 37 ] . LiDAR-Based Perception: LiDAR sensors provide direct 3D measurements and are largely invariant to lighting conditions, making them fundamental in domains such as autonomous driving and mobile robot mapping [ 39 , 40 , 41 ] . However, their application within learning-based, end-to-end legged locomotion has been very limited. Existing uses often involve traditional pipelines (SLAM, path planning) rather than direct