MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots
Yusen Feng, Xiang Wang, Heyuan Yao, Zixi KangXinyu Huo, Boyang Yu, Pengyun Qiu, Ruijie Zhao, Baoquan Chen, Libin Liu · 2026
Read MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots by Yusen Feng, Xiang Wang, Heyuan Yao, Zixi KangXinyu Huo, Boyang Yu, Pengyun Qiu, Ruijie Zhao, Baoquan Chen, Libin Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24592.txt.
Introduction
Yusen Feng and colleagues present MuGen, a 2026 data-driven framework for teaching humanoid robots many locomotion skills from example motions. Vector-quantized autoencoders trained with model-based reinforcement learning build a generative representation of locomotion from hours of varied human data. A teacher-student setup with a policy distillation strategy turns that representation into a deployable student policy. The robot then tracks and mimics unseen human motions, and can put the learned latent space to work on other tasks.
Catalog record
| CREATOR | Yusen Feng, Xiang Wang, Heyuan Yao, Zixi KangXinyu Huo, Boyang Yu, Pengyun Qiu, Ruijie Zhao, Baoquan Chen, Libin Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49412 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24592
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots Yusen Feng, Xiang Wang, Heyuan Yao, Zixi KangXinyu Huo, Boyang Yu, Pengyun Qiu, Ruijie Zhao, Baoquan Chen, Libin Liu Abstract This paper presents MuGen, a data-driven framework for learning and deploying multi-skill locomotion on humanoid robots. MuGen enables a robot to perform expressive motions like humans under the guidance of example motion sequences. To achieve this, we employ vector-quantized autoencoders (VQ-VAEs) trained with model-based reinforcement learning, resulting in a generative representation of locomotion that captures key patterns of human motion from hours of heterogeneous human performance data. We employ a teacher-student learning framework and develop a new policy distillation strategy to enable a deployable student policy learning this efficient latent representation. This policy allows the robot to track and mimic unseen human motions and further enables the robot to reuse the learned latent space for other tasks. We demonstrate the effectiveness of our framework through a diverse set of motions and accurate execution. † † papernotice: Peking University Fig. 1 : MuGen enables multi-skill humanoid locomotion by learning a generative controller. (a-d): A simulated humanoid tracks (a) a long walking motion manually crafted from motion data and (b–d) dance, run, and crouching walk motions selected from the motion dataset. (e-f): A real Unitree G1 robot tracks short segments of (e) a straight walk and (f) a crouching walk from the motion dataset. Details are provided in Section IV-A . I Introduction Humanoid robots have achieved significant advancements in bipedal locomotion and dynamic maneuvers (e.g., running, jumping, getting up, acrobatics) through deep reinforcement learning (RL) frameworks. However, careful task-specific reward shaping and tuning of RL algorithms are typically required for these tasks, which is often challenging and not generalizable across tasks. By leveraging the morphological similarity between humanoid robots and humans, along with the increasing availability of human motion performance datasets, imitating human performance has emerged as a promising approach for humanoid robots [ 14 , 75 , 31 , 43 ] . This approach alleviates the burden of precise reward shaping and enables robots to leverage the control strategies embedded in human motion to produce agile and expressive behaviors. Existing methods for motion imitation typically train policies to take reference kinematic trajectories as part of the input and compute actions that control the robot to follow these trajectories. A kinematic motion generation module may also be employed to generate motions based on task goals, producing new reference trajectories that are then fed to the policy for tracking [ 31 ] . However, this paradigm has two main limitations. First, naive tracking policies cannot effectively capture the structure of the action space under the kinematic and dynamic constraints of the robot, and they often struggle to extract distinctive motion patterns from large-scale datasets and reuse them in downstream tasks. Second, the separation of motion generation and robot control cannot guarantee the dynamic reliability of the generated trajectories. Even if the trajectories are kinematically plausible, the tracking policy may be unable to reproduce them, leading to suboptimal results or even failures. In this paper, we address these fundamental limitations by learning a generative control system, MuGen , from a large human motion dataset with various locomotion performances. MuGen learns a structured latent space for motion skills and actions directly from physical simulation, ensuring that the skill embeddings are consistent with the kinematic and dynamic constraints of the robot. By projecting kinematic trajectories into this latent space or generating new trajectories from it, MuGen can achieve robust tracking on unseen motions and generation of new motions within the capabilities of the robot. We employ vector-quantized variational autoencoders (VQ-VAE) as the generative model, with the goal of reconstructing input motions from discrete latent codes. To achieve effective training, we draw inspiration from recent work in physics-based motion control [ 80 , 81 ] and adopt a model-based reinforcement learning (MBRL) approach, in which a differentiable dynamics model is trained to bridge the gap between kinematic references and the actions predicted by the policy. To further facilitate training, we adopt a two-stage training strategy. First, a teacher generative policy is learned using privileged information, and then the policy and skill embeddings are distilled into a deployable student policy. To this end, we develop a novel DAgger [ 62 ] -style distillation strategy with a progressive scheduling mechanism. We evaluate MuGen on a diverse set of locomotion tasks and validate its performance in robust tracking and the generation of new motions on a Unitree G1 humanoid robot. Our approach significantly outperforms baseline methods in terms of tracking accuracy and robustness when trained on a small set of human motions and generalizes to unseen motions. We also demonstrate that MuGen can generate robust control of new motions using the latent embeddings without requiring reference trajectories. II Related Works II-A Legged Robot Locomotion Legged locomotion has long been a fundamental topic in robotics, with early approaches primarily centered on model‑based control and trajectory optimization. Classical methods, ranging from the pioneering WABOT [ 35 ] and early dynamic walking studies [ 45 ] to the landmark Honda humanoids [ 23 ] rely heavily on accurate analytic models [ 28 , 59 , 46 , 36 , 60 ] . Subsequent work introduced hybrid‑zero dynamics [ 76 ] , whole‑body task hierarchies [ 8 ] , and the 3‑D LIPM for pattern generation [ 34 ] . Model‑based techniques remain state‑of‑the‑art for many humanoid and quadruped platforms, including ANYmal [ 27 ] , geometric retargeting [ 9 ] , multimode or tele‑operated frameworks [ 49 , 58 , 29 , 48 ] , whole‑body MPC [ 47 , 61 , 13 , 12 ] , and recent acrobatic or synchronized human–robot studies [ 6 , 7 ] . Despite their effectiveness, these methods demand extensive expert tuning and often degrade under modeling error or unstructured terrain. II-B Reinforcement Learning-based Methods for Locomotion Recent advances in reinforcement learning (RL) have shown remarkable robustness and adaptability, as policies are learned directly from interaction without an exact dynamics model. For quadrupeds, model‑free RL enables agile motion imitation [ 52 ] , rapid motor adaptation [ 37 ] , energy‑efficient gait discovery [ 15 ] , vision‑based blind stair climbing [ 66 ] , and parkour‑level versatility [ 84 , 5 , 73 ] . RL further supports loco‑manipulation [ 4 , 30 , 63 ] and dynamic ball‑dribbling [ 32 ] . For bipeds and humanoids, policies have progressed from simulation‑only agility [ 50 , 54 , 53 , 68 ] to real‑world deployment [ 38 , 40 , 56 ] , RL exoskeleton‑cockpit teleoperation [ 2 ] , and versatile jump/stand‑up controllers [ 39 , 25 ] . Nevertheless, low sample efficiency and the sim‑to‑real gap remain key obstacles. Some works [ 33 , 19 ] combine online system identification with policy learning, while extensive domain randomization is still common practice [ 1 , 79 , 10 ] . However, these tasks often require fine-grained reward design with respect to the goal. In this work, we address these limitations by proposing a differentiable physical simulator, transforming humanoid behaviors learning into a model-based approach. Our method reduces the need for extensive reward engineering and enables