Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation
Pascal Roth 1 2 , Jonas Frey 1 3 , Cesar Cadena 1 , and Marco Hutter 1 · 2025
Read Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation by Pascal Roth 1 2 , Jonas Frey 1 3 , Cesar Cadena 1 , and Marco Hutter 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19322.txt.
Introduction
Pascal Roth and co-authors learn a forward dynamics model for safe robot navigation in a 2025 paper. Safe travel in complex terrain needs a live read on which ground the robot can cross, judged against its own abilities. Older methods simplify dynamics and rely on hand tuned cost functions, a tedious process that does not transfer well between settings.
Their learned model predicts the robot future state from surrounding geometry and past body sensor history. It was trained on years of simulated navigation, including risky maneuvers, plus real world runs. Plugged into a sampling based planner, it lets a simple cost function suffice. On a legged robot, position estimates improved by a large margin on average.
Catalog record
| CREATOR | Pascal Roth 1 2 , Jonas Frey 1 3 , Cesar Cadena 1 , and Marco Hutter 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-19322
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation Pascal Roth 1 2 , Jonas Frey 1 3 , Cesar Cadena 1 , and Marco Hutter 1 Abstract Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot’s capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot’s future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data. Code and models are made publicly available under https://github.com/leggedrobotics/fdm . Fig. 1 : Demonstration of the proposed perceptive Forward Dynamics Model for robust navigation in complex environments. The model, trained with real-world and simulation data, predicts the robot’s future states given a sequence of velocity actions. It takes as input the surrounding geometry in the form of a height scan, along with past states and proprioceptive measurements. A sampling-based planner evaluates the integrated paths based on simple reward functions to select the optimal next action in a receding horizon fashion. (A) Ten example paths are visualized and overlaid on the environment image alongside the height map and the downsampled height scan (blue points). Path colors indicate rewards, with the highest reward assigned to the closest collision-free trajectory to the goal. (B–E) Additional planning events are shown, displaying sampled paths and the selected trajectory (green), demonstrating safe planning in rough terrain. I Introduction Understanding robotic system dynamics is essential for ensuring safe and effective control, particularly in complex tasks like motion planning in contact-rich scenarios. The dynamics of a mobile robot navigating within an environment depend on its structure and interactions with the terrain. This results in highly nonlinear behaviors that are challenging to generalize across diverse scenarios [ 1 ] . Forward Dynamics Models (FDM) are typically used to predict such complex dynamics, estimating the robot’s future state conditioned on the applied commands. These models capture the robot-terrain interactions and implicitly provide a terrain traversability estimate. Dynamic models must carefully balance key modeling choices, including state representation, fidelity, time horizon, and modeling frequency. While the dynamics of on- and off-road vehicles have been extensively explored [ 2 , 3 , 4 ] , quadrupedal robots present unique challenges due to their complex system dynamics and difficult-to-model environmental interactions [ 5 ] . Moreover, their learned locomotion policies, which rely on deep neural networks, additionally complicate the modeling of the robot’s behavior. Traditional physics-based models that are derived from first principles and calibrated using system identification often fail to capture the dynamics accurately. They specifically struggle in contact-rich scenarios, which introduce additional non-linearities and require accurate perception [ 6 ] . Further, they can be computationally expensive and sensitive to initial conditions. Consequently, those models face challenges when it comes to accurately modeling system dynamics, which in turn results in biased predictions and persistent modeling errors. To overcome these limitations, data-driven approaches have emerged as a promising alternative to approximate complex dynamics. However, training neural networks to represent robot dynamics often requires substantial amounts of state-action trajectories, motivating the use of synthetic data to mitigate the challenges of collecting extensive real-world datasets [ 5 ] . Further, simulation allows performing dangerous or catastrophic maneuvers that harm the real robot, such as falling or colliding. While the simulator’s complex physics modeling is accurate in rigid-body scenarios, it is computationally expensive and fails to capture scenarios outside of its domain. As a result, it becomes necessary to distill the dynamics into a learned model for sufficient inference speed on a compute-restricted mobile robot [ 5 ] . Moreover, real-world data remains essential for addressing unmodeled effects and bridging the reality gap [ 7 ] . Addressing the gap between physics-informed and learned models, approaches that integrate physics constraints — such as kinematic laws or energy conservation — in the learning setup show strong performance [ 8 , 9 , 10 , 6 ] . However, they remain limited to short control timescales compared to the longer planning timesteps addressed in this work. The first work that employs a learned FDM on a quadrupedal system has been done by [ 5 ] . Combined with their developed trajectory sampling technique, they demonstrate reactive navigation in complex, narrow environments. However, open challenges remain to incorporate 3D perception to target rough environments and the transfer from simulation to the real system. This work introduces a perceptive, n -step Forward Dynamics Model framework. The proposed approach combines pre-training with synthetic data generated using a state-of-the-art simulator and fine-tuning with real-world data. This hybrid strategy leverages the safety and flexibility of simulation while capturing real-world dynamics. The FDM is designed for both legged and wheel-legged systems, marking the first application of its kind in rough terrain environments. Our novel framework extends the capabilities of sampling-based planner methods by reducing the need for extensive parameter tuning and providing a flexible solution for non-task-specific planning. This enables zero-shot adaptation to new environments without requiring additional learning steps. Additionally, the perception capabilities of the model represent a significant step forward, offering an attractive alternative to explicitly modeling the environment’s traversability. The main contributions of this work are as follows: • The first application of a rough-terrain Forward Dynamics Model trained in simulation and deployed on a quadrupedal robot. The model demonstrates reliable sim-to-real transfer capabilities and robust performance in rough terrain. • A hybrid training strategy using real-world data to effectively capture the full system’s dynamics beyond rigid-body simulation while leveraging synthetic data for pre-training to safely account for high-risk scenarios. • A simplified cost formulation for MPPI-based planning that integrates the platform-specific FDM to enable safe and reliable trajectory generation. The approach supports zero-shot adaptation to new environments by cost-term adjustments without the need for additional training. II Related Work II-A Dynamics Modeling The field of dynamics learning has predominantly focused on data-driven solutions [ 2 , 3 , 11 , 7 , 12 , 4 , 1 ] , as models derived from first principles and calibrated via system identification often oversimplify or misrepresent system dynamics, leading to bias and persistent modeling errors [ 6 ] . In contrast, learned models can approximate complex, nonlinear dynamics from large datasets [ 13 , 14 ] while capturing uncertainties using probabilistic neural