PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers 276 Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Journal: TOG Conference: Special Interest Group on
Michael Xu, Yi Shi, KangKang Yin, Xue Bin Peng · 2025
Read PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers 276 Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Journal: TOG Conference: Special Interest Group on by Michael Xu, Yi Shi, KangKang Yin, Xue Bin Peng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04002.txt.
Introduction
Michael Xu and co-authors build a training loop in 2025 that teaches animated characters parkour-style movement. Real motion capture for climbing walls and leaping gaps is scarce and costly, so characters trained on small sets stay clumsy on new ground. Their framework, named PARC in the paper, grows its own training data round by round.
A motion maker first learns core skills, then drafts moves for new terrain. Those drafts often have flaws like wrong contacts, so a physics-based tracker replays them in simulation and fixes them. The cleaned moves rejoin the training set for the next round. Round by round, both parts grow more agile and more versatile on complex ground.
Catalog record
| CREATOR | Michael Xu, Yi Shi, KangKang Yin, Xue Bin Peng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04002
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers 276 Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers ; August 10–14, 2025; Vancouver, BC, Canada Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’25), August 10–14, 2025, Vancouver, BC, Canada DOI: 10.1145/3721238.3730616 ISBN: 979-8-4007-1540-2/2025/08 CCS: Computing methodologies Procedural animation CCS: Computing methodologies Physical simulation Michael Xu, Yi Shi, KangKang Yin, Xue Bin Peng Abstract . Humans excel in navigating diverse, complex environments with agile motor skills, exemplified by parkour practitioners performing dynamic maneuvers, such as climbing up walls and jumping across gaps. Reproducing these agile movements with simulated characters remains challenging, in part due to the scarcity of motion capture data for agile terrain traversal behaviors and the high cost of acquiring such data. In this work, we introduce PARC ( P hysics-based A ugmentation with R einforcement Learning for Character C ontrollers), a framework that leverages machine learning and physics-based simulation to iteratively augment motion datasets and expand the capabilities of terrain traversal controllers. PARC begins by training a motion generator on a small dataset consisting of core terrain traversal skills. The motion generator is then used to produce synthetic data for traversing new terrains. However, these generated motions often exhibit artifacts, such as incorrect contacts or discontinuities. To correct these artifacts, we train a physics-based tracking controller to imitate the motions in simulation. The corrected motions are then added to the dataset, which is used to continue training the motion generator in the next iteration. PARC’s iterative process jointly expands the capabilities of the motion generator and tracker, creating agile and versatile models for interacting with complex environments. PARC provides an effective approach to develop controllers for agile terrain traversal, which bridges the gap between the scarcity of motion data and the need for versatile character controllers. Figure 1. A visualization of a physically simulated character traversing a complex terrain using different agile parkour skills. Our framework starts with a small dataset of terrain traversal motions, and then performs an iterative dataset augmentation loop that progressively expands the capabilities of a motion generation model and a physics-based motion tracking controller. Heightmap parkour agent. 1. Introduction Humans possess the remarkable ability to navigate through diverse and complex environments by employing a broad range of agile motor skills. A prime example of human agility can be seen in parkour practitioners, who regularly demonstrate extraordinary athleticism by stylishly traversing obstacles using dynamic combinations of maneuvers, such as vaulting, climbing, and jumping. Developing simulated characters that can achieve comparable versatility remains a significant challenge. Current state-of-the-art methods for training physics-based controllers predominantly rely on motion capture data, using imitation objectives to guide the learning process towards natural and human-like behaviors. Due to the challenges of capturing highly athletic interactions with complex environments, there is little data available of human athletes traversing diverse terrains with agile motor skills. Moreover, acquiring large quantities of high quality motion data for such athletic behaviors can be exorbitantly expensive. Human motion capture data is important for training character controllers that can move in a natural and life-like manner. While recording a large volume of data from human athletes can be costly, it is feasibly economical to capture a narrow set of high quality motion data depicting core terrain-traversal skills. A simple data augmentation method leveraging generative models may be to first train a motion generator on this small initial dataset, then use the generative model to automatically generate an expanded synthetic dataset of more diverse behaviors. This augmented dataset can be used to further train the motion generator to improve its generation capabilities for traversing more difficult and diverse environments. This self-consuming process can be performed iteratively to create a sufficiently large dataset suitable for training a general and effective terrain traversal controller. However, a motion generator trained on a small dataset often produces low-quality, physically implausible motions, especially in unfamiliar scenarios. These artifacts—such as incorrect contacts, floating, sliding, or discontinuities—can degrade the generator’s performance over the iterations. To mitigate this progressive degradation, we leverage physics simulation by training a motion imitation controller to correct these artifacts. Instead of using the raw generated motions, the physically corrected motions are added to the dataset, improving the physical realism of the synthetic data and stabilizing the training process throughout the iterative process. Building on this idea, we introduce PARC (Physics-based Augmentation with Reinforcement Learning for Character Controllers), a framework that takes a small initial motion dataset as input and outputs a motion tracking controller for traversing complex terrains. PARC iteratively trains a motion generator and motion tracker to augment a motion dataset, progressively expanding the capabilities of both models. The motion generator synthesizes new terrain traversal motions, while the motion tracker refines them to ensure physical plausibility before adding the motion clips to the dataset. This expanded dataset is used to continuously train both the generator and tracker, enhancing their versatility. Through multiple iterations, PARC develops an expressive motion generator and an agile motion-tracking controller. Together, these components enable precise control of a simulated character, allowing it to agilely navigate complex obstacle-filled environments. The code and data used to train PARC, as well as the models and data generated by PARC, can be found at https://github.com/mshoe/PARC . 2. Related Work Recent advances in machine learning have led to a surge of techniques that are capable of automatically producing high-fidelity human motions, which span both kinematics-based methods ( Holden et al. 2017 ; Starke et al. 2019 ; Rempe et al. 2021 ) and physics-based methods ( Peng et al. 2018 ; Peng et al. 2021 ; Peng et al. 2022 ; Fussell et al. 2021 ; Yao et al. 2022 ) . However, generating highly dynamic character-scene interaction behaviors remains a persistent challenge due to limited data availability. The constraints imposed by complex scenes also place more stringent demands on motion quality, as they are susceptible to more pronounced artifacts, such as floating and terrain collisions. In this section, we review the most relevant prior work on generating interactive motion under imposed terrain constraints. 2.1. Kinematic Motion Generation Given abundant, high-quality motion data, kinematic motion generation models can effectively synthesize complex human behaviors. Holden et al. 2017 ; Holden et al. 2020 introduced a phase-based auto-regressive model that is able to generate locomotion behaviors on irregular terrain. Yi et al. 2024 ; Li et al. 2024 leveraged the expressiveness of diffusion models to synthesize scene-aware motions. While these works have shown promising results, they often struggle to generalize to new scenarios not captured in the original training dataset. Furthermore, their lack of physics-based simulation often leads to artifacts such as floating, ground penetration, and self-collision, compromising the realism of the synthesized motions. 2.2. Physics-based Character Control Physics-based character simulation has been explored as a means to procedurally generate novel behaviors in scenarios where motion data may be scarce. For example, to model human athletic skills, existing works have developed physics-based controllers capable of replicating a wide range of dynamic sports, including parkour ( Liu