PhyGile : Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking

Jiacheng Bao, Haoran Yang, Yucheng Xin, Junhong Liu, Yuecheng Xu, Han Liang, Pengfei Han, Xiaoguang Ma, Dong Wang, Bin Zhao · 2026

Read PhyGile : Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking by Jiacheng Bao, Haoran Yang, Yucheng Xin, Junhong Liu, Yuecheng Xu, Han Liang, Pengfei Han, Xiaoguang Ma, Dong Wang, Bin Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19305.txt.

Introduction

Bao and co-authors make humanoid robots move with agility while staying within physics limits. Motion models trained on human data often yield moves a robot cannot safely do. Their PhyGile system makes robot-native motion at test time, guided by physics-based prefixes. A tracking controller is first trained with a staged mix of experts. It is then tuned on the generated moves under those same prefixes. Tests off and on real robots show fast, stable motion. The loop cuts the gap between made and done moves.

Catalog record

CREATORJiacheng Bao, Haoran Yang, Yucheng Xin, Junhong Liu, Yuecheng Xu, Han Liang, Pengfei Han, Xiaoguang Ma, Dong Wang, Bin Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54969
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19305

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PhyGile : Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking Jiacheng Bao, Haoran Yang, Yucheng Xin, Junhong Liu, Yuecheng Xu, Han Liang, Pengfei Han, Xiaoguang Ma, Dong Wang, Bin Zhao Abstract Humanoid robots are expected to execute agile and expressive whole-body motions in real-world settings. Existing text-to-motion generation models are predominantly trained on captured human motion datasets, whose priors assume human biomechanics, actuation, mass distribution, and contact strategies. When such motions are directly retargeted to humanoid robots, the resulting trajectories may satisfy geometric constraints (e.g., joint limits, pose continuity) and appear kinematically reasonable. However, they frequently violate the physical feasibility required for real-world execution. To address these issues, we present PhyGile, a unified framework that closes the loop between robot-native motion generation and General Motion Tracking (GMT). PhyGile performs Physics-prefix-Guided robot-native motion generation at inference time, directly generating robot-native motions in a 262-dimensional skeletal space with physics-guided prefix, thereby eliminating inference-time retargeting artifacts and reducing generation–execution discrepancies. Before physics-prefix adaptation, we train the GMT controller with a curriculum-based mixture-of-experts scheme, followed by post-training on unlabeled motion data, to improve robustness over large-scale robot motions. During physics-prefix adaptation, the GMT controller is further fine-tuned with generated objectives under physics-derived prefixes, enabling agile and stable execution of complex motions on real robots. Extensive offline and real-robot experiments demonstrate that our PhyGile expands the frontier of text-driven humanoid control, enabling stable tracking of agile, highly-difficult whole-body motions that go well beyond walking and low-dynamic motions typically achieved by prior methods. I Introduction Humanoid control is undergoing a paradigm shift from task-specific locomotion toward scalable and general-purpose motion generation. Recent advances in general motion tracking (GMT) show that a single policy can imitate large collections of reference motions and reliably transfer them to real hardware [ 4 , 35 , 33 ] . In parallel, text-driven motion generation has progressed rapidly, with diffusion-based models producing semantically rich and diverse human motions conditioned on natural language [ 30 , 39 , 3 ] . Recently, these two paradigms have begun to converge, where text-to-motion models provide high-level motion priors and GMT policies serve as low-level executors, forming a hierarchical pipeline for autonomous humanoid control [ 34 , 11 , 26 ] . Despite the above-mentioned successes, motion generation and physical execution remain fundamentally misaligned in practice. Most text-to-motion models are developed in the human domain and synthesize motions in standardized representations such as SMPL [ 20 ] . Running these motions on humanoid robots requires a retargeting stage [ 1 , 21 ] that maps human joint trajectories to robot morphologies. While forward-kinematics-based retargeting can approximately preserve pose structure, it does not enforce coupled physical constraints, including torque limits, contact consistency, and dynamic balance. Consequently, retargeted motions often contain physically inconsistent segments that appear kinematically plausible yet are dynamically unstable [ 24 ] . In large-scale training pipelines, such corrupted references degrade policy learning and often necessitate filtering mechanisms to discard invalid samples, reducing data efficiency and robustness. Beyond retargeting artifacts, GMT is further constrained by severe data imbalance. Large motion datasets exhibit pronounced long-tail distributions. For example, HumanML3D [ 7 ] , one of the most widely used benchmarks for text-driven motion modeling, contains abundant simple motions such as walking and running, whereas complex and agile motions are comparatively rare. When training a general tracking policy on such skewed distributions, optimization naturally prioritizes frequent and low-difficulty motions. Rare and high-difficulty skills that require tight coordination and dynamic stability remain undertrained, leading to fragile performance when robots attempt agile motions. Recent humanoid motion generation approaches attempt to reduce reliance on human-first retargeting by aligning language with robot-controllable embeddings [ 11 , 31 ] or directly modeling robot-native motion data [ 34 ] . While these methods strengthen the connection between semantics and embodiment, generation and tracking are typically optimized as loosely coupled modules: a generator proposes trajectories, and a tracker attempts to follow them. When generated motions exceed the feasible region of the current controller, execution usually fails unless a mechanism is available to reconcile generation with control. In this work, we propose PhyGile, a physics-prefix-guided framework that couples robot-native motion generation with agile GMT, enabling high-performance tracking and generation within a single, physically grounded pipeline. With the motion generator frozen, PhyGile leverages physics-validated prefixes as a shared interface to improve motion quality and GMT agility, leading to more physically consistent and task-effective motions at execution time. On the generation side, we introduce a diffusion-based motion model operating in a 262D robot-skeleton space, directly synthesizing robot-native joint trajectories. To enhance compositional language grounding, we incorporate a Token-level Parameter-mixing Mixture-of-Experts (TP-MoE), enabling fine-grained alignment between individual text tokens and the motion timeline, so that temporally localized semantic cues correspond to appropriate motion segments. On the control side, we address the long-tail challenge of agile motions through a curriculum mixture-of-experts training scheme. In the first stage, we perform explicit curriculum learning on difficulty-annotated motion data, where motions are stratified by complexity and experts specialize across skill levels. Error-aware adaptive sampling progressively increases exposure to high-difficulty motions, improving robustness on rare and highly dynamic skills. In the second stage, we conduct global soft-moe post-training on large-scale unlabeled motions to enhance overall generalization while preserving the agility acquired during curriculum learning. To bridge generation and execution, we further introduce a physics-prefix-guided fine-tuning stage. Dynamically feasible motion segments extracted from the tracking policy are injected as conditioning prefixes into the diffusion process, anchoring the initial denoising states to executable regions of the robot’s motion manifold. During this stage, the diffusion model remains frozen, while the GMT controller is fine-tuned under prefix-conditioned motion generation to improve tracking performance on generated motion distributions, thereby tightening the coupling between generation and execution. By integrating difficulty-aware tracking, robot-native diffusion generation, and physics-prefix-guided alignment into a unified pipeline, PhyGile enables the generation and execution of agile humanoid motions that are both semantically aligned and physically realizable. Extensive offline and real-robot experiments demonstrate that PhyGile effectively resolves the semantics–physics trade-off in generation, establishing a highly robust and unified tracking policy. Notably, for highly dynamic agile motions where prior pipelines fail, our framework rapidly unlocks stable hardware execution through lightweight fine-tuning, demonstrating strong scalability toward complex whole-body skills. In summary, PhyGile ’s main contributions are as follows: • We propose PhyGile , a physics-prefix-guided framework for general humanoid motion tracking that couples robot-native diffusion-based motion generation with GMT. By closing the loop via physics-consistent prefixes, PhyGile mitigates the mismatch between text-driven motion synthesis and real-world control, enabling agile and dynamically feasible humanoid motions. • We develop three key components: (i) a curriculum-based MOE training strategy for agile GMT with motion-difficulty stratification; (ii) a 262D robot-skeleton representation with TP-MoE-enhanced motion generation for fine-grained text conditioning; and (iii) a physics-prefix-guided fine-tuning stage that aligns generated motions with executable tracking policies. • Through extensive offline benchmarks and real-robot experiments, we demonstrate that PhyGile effectively resolves the semantics–physics trade-off in generation. Furthermore, our robust, unified tracking policy achieves stable hardware execution of highly dynamic humanoid motions, successfully unlocking complex agile behaviors that prior pipelines fail to realize even with additional tuning. II Related Works II-A Text-driven Human Motion Generation Text-driven motion generation aims to learn the conditional distribution of

Read the complete text