Learning Robust Motion Skills via Critical Adversarial Attacks for Humanoid Robots

Yang Zhang, Zhanxiang Cao, Buqing Nie, Haoyang Li, Yue Gao · 2025

Read Learning Robust Motion Skills via Critical Adversarial Attacks for Humanoid Robots by Yang Zhang, Zhanxiang Cao, Buqing Nie, Haoyang Li, Yue Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08303.txt.

Introduction

Yang Zhang and co-authors toughen humanoid robot motion skills by attacking them at their weakest points during training, in a 2025 paper. Learned motion policies often lose balance over long runs or under noise and knocks. Random disturbance training can miss the true weak spots or make robots overly timid.

A learned adversary picks the most fragile states and actions to perturb, within a set attack budget. Policy and adversary train in turns, each forcing the other to improve. On a commercial humanoid robot, trained policies crossed rough terrain with forty percent more success and cut tracking error by about a third. Long runs held up well. The authors back targeted attacks as a driver of sturdy skills.

Catalog record

CREATORYang Zhang, Zhanxiang Cao, Buqing Nie, Haoyang Li, Yue Gao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08303

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Robust Motion Skills via Critical Adversarial Attacks for Humanoid Robots Yang Zhang, Zhanxiang Cao, Buqing Nie, Haoyang Li, Yue Gao Abstract Humanoid robots show significant potential in daily tasks. However, reinforcement learning-based motion policies often suffer from robustness degradation due to the sim-to-real dynamics gap, thereby affecting the agility of real robots. In this work, we propose a novel robust adversarial training paradigm designed to enhance the robustness of humanoid motion policies in real worlds. The paradigm introduces a learnable adversarial attack network that precisely identifies vulnerabilities in motion policies and applies targeted perturbations, forcing the motion policy to enhance its robustness against perturbations through dynamic adversarial training. We conduct experiments on the Unitree G1 humanoid robot for both perceptive locomotion and whole-body control tasks. The results demonstrate that our proposed method significantly enhances the robot’s motion robustness in real world environments, enabling successful traversal of challenging terrains and highly agile whole-body trajectory tracking. I Introduction Humanoid robots, with their human-like morphology and versatile mobility [ 1 ] , have the potential to replace humans in performing various tasks in daily life [ 2 ] , demonstrating significant potential in fields such as domestic service [ 3 ] , industrial production [ 4 ] , and healthcare [ 5 ] . Robust motion control of humanoid robots is the basis for achieving other complex tasks [ 6 ] . In scenarios such as carrying objects [ 7 ] and performing high-precision operations [ 1 ] , robots must have the ability to move autonomously and maintain body balance in dynamic unknown terrain environments [ 8 ] . Furthermore, robots need to adapt to external disturbances during interactions with the environment [ 9 ] , such as unexpected collisions or slippage of feet, to prevent falls and ensure task safety [ 10 ] . However, the high Degrees of Freedom (DoF) and the complex kinematic coupling between the upper-body and lower-body make the development of flexible and stable locomotion control systems highly challenging [ 11 ] , limiting their broader deployment in real world applications [ 12 ] . Fig. 1: Snapshot of the humanoid robot executing whole-body trajectory tracking. PL-CAP can track challenging dynamic trajectories over an extended duration, demonstrating that adversarial training based on critical attacks significantly improves the robustness of motion policies. Traditional model-based control methods are based on precise dynamic modeling and complex parameter tuning [ 13 ] . However, these methods struggle to comprehensively capture the diversity and complexity of humanoid robot interactions with the environment [ 14 ] , leading to degraded control performance or even failure in complex terrain environments [ 1 ] . Recently, with the rapid development of simulation technology and computing power, Reinforcement Learning (RL)-based locomotion control has emerged as a promising alternative [ 15 , 16 , 17 ] . By learning the optimal policy through interaction with the environment in simulation, the robot can autonomously adapt and optimize its motion behavior in complex environments [ 18 , 12 ] . The trained policy is then transferred from the simulation to the real robot to achieve flexible motion capabilities [ 19 , 20 ] . However, due to the discrepancies between the simulation environment and the real environment [ 21 ] , including environmental variations [ 22 ] , sensor noise [ 23 ] , and external disturbances [ 24 ] , neural controllers are likely to encounter domain distribution shifts [ 22 ] , leading to the sim-to-real transfer problem of motion policies [ 25 ] . Additionally, the inherent sensitivity of RL-based neural controllers to perturbations further exacerbates the instability of deploying these policies in real world applications [ 26 ] . To enhance the robustness of motion policies on real robots, prior works have employed domain randomization [ 11 ] to reduce the distribution gap between simulated and real world data, thereby improving the robustness of the policy to environmental uncertainty [ 15 ] . However, domain randomization fails to precisely identify specific vulnerability of neural policies and is difficult to generate high-quality perturbations samples to assist policy training [ 27 , 18 ] . Furthermore, excessive domain randomization can significantly degrade policy performance [ 28 ] . In contrast, adversarial attacks introduce carefully designed perturbations that severely degrade neural network performance [ 25 ] , effectively identifying network vulnerabilities and evaluating the robustness of motion policies [ 29 ] . Additionally, adversarial training incorporates adversarial samples into the training data to enhance the ability of motion policies to resist perturbations and improve the robustness of motion policies in real environments [ 30 , 31 , 32 ] . In this paper, we propose a novel robust adversarial training paradigm to enhance the robustness of humanoid motion policies in real world environments. This paradigm introduces a learnable adversary network to identify potential vulnerabilities in motion policies and assist in optimizing the policy to improve the robot’s ability to resist perturbations. The attacker is designed to identify vulnerabilities in humanoid motion states and apply targeted perturbation attacks, aiming to destabilize the robot with minimal attack cost. Through a non-zero-sum adversarial training framework, we alternately optimize attack policies and motion policies, enhancing the attacker’s ability to generate effective perturbations while progressively improving the motion policy’s robustness against such perturbations. We conduct extensive experiments on the Unitree G1 humanoid robot [ 33 ] in various challenging terrains and agile trajectory tracking. The results demonstrate that our approach significantly improves the motion performance of humanoid robots in real world environments. In summary, the contributions of this work are as follows: • A novel robust adversarial training paradigm for humanoid robots is proposed. By introducing adversarial attack policies to identify vulnerabilities of motion policies, effective adversarial samples are generated to enhance the robustness of motion policies. • The critical attack policy is proposed by constraining attack costs. It improves the vulnerability mitigation of motion policies rather than conservative degradation. • Extensive experimental results demonstrate that our method effectively enhances the robustness of motion policies and significantly improves the motion performance of the real robot. Fig. 2: Overview of the adversarial training framework. The CAP identifies vulnerabilities in motion states and generates adversarial samples by applying perturbations in both state space and action space. Through alternating adversarial training under non-zero-sum game, the motion policy continuously addresses its own vulnerabilities using adversarial samples, thereby enhancing its robustness against perturbations. During deployment, the robust motion skills are deployed to real-world robots without requiring attack policies, enabling robust whole-body motion control for humanoid robots II Related Work II-A Learning-Based Humanoid Motion Control In recent years, RL-based approaches have achieved impressive results in humanoid robot motion control [ 34 , 11 , 16 , 35 , 6 ] , especially in complex terrain traversal [ 17 , 18 ] and whole-body trajectory imitation [ 36 , 9 ] . These methods narrow the gap between simulation and reality through reward shaping [ 15 , 19 ] , reference trajectory guidance [ 37 , 21 ] , and robustness constraints [ 10 ] , thereby improving the performance of motion policies when applied to real robots. For instance, [ 35 , 6 ] and [ 38 ]

Read the complete text