HACL: History-Aware Curriculum Learning for Fast Locomotion
Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao, Dinesh Manocha · 2025
Read HACL: History-Aware Curriculum Learning for Fast Locomotion by Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao, Dinesh Manocha (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18429.txt.
Introduction
Prakhar Mishra and co-authors teach legged robots to run fast without falling, using a training method that remembers its own past commands. Sprinting is a prized skill for four-legged and two-legged robots, but speed tends to break balance. Their 2025 method feeds the history of joint speed commands and past rewards into the lesson plan.
A memory network learns how forward and turning commands map to rewards over time. That history shapes the order in which harder tasks are introduced. In simulation on three robot models, and on a real robot dog, the method reached a peak speed near the top command. It beat past locomotion methods by about twenty percent, the authors report.
Catalog record
| CREATOR | Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao, Dinesh Manocha |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18429
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HACL: History-Aware Curriculum Learning for Fast Locomotion Prakhar Mishra, Amir Hossain Raj, Xuesu Xiao, Dinesh Manocha Abstract We address the problem of agile and rapid locomotion, a key characteristic of quadrupedal and bipedal robots. We present a new algorithm that maintains stability and generates high-speed trajectories by considering the temporal aspect of locomotion. Our formulation takes into account past information based on a novel history-aware curriculum Learning (HACL) algorithm. We model the history of joint velocity commands with respect to the observed linear and angular rewards using a recurrent neural net (RNN). The hidden state helps the curriculum learn the relationship between the forward linear velocity and angular velocity commands and the rewards over a given time-step. We validate our approach on the MIT Mini Cheetah, Unitree Go1, and Go2 robots in a simulated environment and on a Unitree Go1 robot in real-world scenarios. In practice, HACL achieves peak forward velocity of 6.7 m/s for a given command velocity of 7m/s and outperforms prior locomotion algorithms by nearly 20%. I Introduction One key capability of bipedal and quadrupedal robots is to maneuver rapidly and with agility in different scenarios. Moving fast allows the robot to traverse large distances quickly and navigate diverse terrains effectively and efficiently. However, performing fast locomotion on diverse terrains gives rise to many challenges because we do not have accurate information about the real world characteristics around the robot, e.g., dynamics parameters like friction, obstacles, uneven terrain, slippery or inclined surfaces, etc. One possibility is to model the environment parameters [ 47 ] , [ 11 ] . However, designing robust techniques based on model-based control requires a significant amount of expertise and creativity on the part of the human designer. Another possibility is to infer these parameters [ 22 ] , [ 16 ] . The current paradigm is to train the robot in simulation and use state-of-the-art techniques [ 38 ] to deploy it on the real robot in the physical world. These methods achieve either high speed or stability, but not both. Although rapid locomotion is highly desirable, it should not compromise the stability and efficiency of the robot. Operation in the real world can result in many challenges in the form of terrain diversity, such as sudden slopes, slippery surfaces, icy patches, or obstacles. The major issues in terms of using learning methods include sparse rewards distribution, the exploration-exploitation dilemma, the sim-to-real gap, and the non-Markovian nature of locomotion [ 6 ] , [ 46 ] . Fig. 1 : Testing and parallelized training: Application of HACL in the real-world on the Go1 robot (above 2 rows). Parallelized training of MIT Mini Cheetah, Go1, and Go2 quadrupeds with HACL in Nvidia IsaacGym using 4000 environments (bottom row). One way to overcome such challenges is to use a curriculum [ 28 ] , [ 22 ] , [ 29 ] , [ 18 ] , [ 16 ] . These methods first train the robot on smaller velocity ranges. In the process, they gradually increase the difficulty if the robot succeeds in performing easier tasks, decrease difficulty if the robot performs worse, and go to a random difficulty level if the agent performs well on even the most difficult tasks. The objective of curriculum learning is to improve training efficiency and final performance, but it can result in overfitting [ 5 ] or suboptimal exploration [ 22 ] , [ 12 ] . Main Results: We present a new approach for fast and stable locomotion that takes into account temporal dependencies by incorporating history in the curriculum. Many approaches for locomotion perform sequential decision-making and have inherent temporal dependencies due to their non-Markovian nature [ 6 ] , [ 46 ] , [ 44 ] . We present a novel History-Aware Curriculum Learning (HACL) algorithm, which addresses issues of temporal dependencies and suboptimal exploration, and improves the forward linear velocity computation and stability. The novel components of our approach include: • We present history-aware curriculum learning (HACL), a curriculum approach that schedules the next episode bins based on past observed training rewards for linear and angular velocities ( r_{lin} , r_{ang} ). Our formulation also models temporal dependencies. • We show that by capturing historical information, we can bridge the gap between non-Markovian dynamics and curriculum learning by incorporating past state information ( h_{t-1} ). This leads to improved convergence and learning. • We have conducted an extensive evaluation of our method in simulated environments on MIT Mini Cheeta, Unitree Go1 and Go2 robots in NVIDIA IsaacGym. We also highlight the benefitson a Unitree Go1 robot operating in the real world. Our results show that HACL achieves higher speed, stability, and improved efficiency. HACL achieves velocity of 6.7m/s with a standard deviation of 1.73 m/s, which is significantly better than [ 22 ] . The stability score is around 2000 and energy efficiency is -4000 , which is much better compared to the current RL methods. HACL consumes less energy for the higher velocity output compared to fixed curricula, bandit-based task sampling including UCB [ 3 ] , Thompson [ 7 ] ), and non-history neural networks such as MLP [ 30 ] and CNN [ 17 ] . II Related Work II-A Curriculum learning in Robotics Bengio et al. 2009 propose a method similar to continuation methods where training happens in a progression, starting with easier examples and gradually increasing the difficulty. Wang et al. 2021 discuss how to implement/design a predefined curriculum or an automatic curriculum. Their work categorizes automatic curriculum learning (CL) into 4 categories: Self-paced Learning, Transfer Teacher, RL Teacher Matiisen et al. 2019 , and Other Automatic CL. While [ 2 ] employs a multi-terrain curriculum approach and similarly environment progression based by curriculum [ 16 ] for learning locomotion skills. And [ 8 ] , [ 19 ] discuss hindsight curriculum learning, Wang et al. 2024 arm wheel curriculum, and Tidd et al. 2020 guided curriculum for terrains. While [ 22 ] , employed a fixed rule based curriculum update method for learning the locomotion control policy, absence of history information was common across all these works. II-B Model-based control and Reinforcement Learning for Locomotion Some of the earliest attempt to implement model-based control can be dated back to DARPA’s learning research in locomotion [ 48 ] , [ 15 ] , [ 24 ] . However, A key limitations of these methods regarding fast locomotion is sim-to-real gap. To reduce the gap between sim-to-real, several methods have been proposed like domain randomization [ 38 ] . Domain randomization gives robust behavior, but it can result in a conservative policy [ 20 ] . Alternatively, one can design simulation environment as close as possible to the real world scenarios [ 11 ] , [ 33 ] . But they would require extensive amount of real world data and need to be setup for different configurations. One way to avoid these issues is to infer the data during deployment [ 16 ] , [ 22 ] . II-C Non-Markovian nature of locomotion As [ 6 , 46 , 44 ] have pointed out, locomotion has a non-Markovian nature, where the state is not only dependent on the immediate