FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control
Tan Jing, Shiting Chen, Yangfan Li, Weisheng Xu, Renjing Xu · 2025
Read FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control by Tan Jing, Shiting Chen, Yangfan Li, Weisheng Xu, Renjing Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19926.txt.
Introduction
Tan Jing and co-authors present a 2025 robotics paper on teaching humanoid machines to copy fast, explosive human motion. Controllers that handle gentle daily movement well still fail on jumps and spins, which blocks real use. Their framework, called FARM, attacks the gap from three sides at once.
First, training clips are stretched so the model sees bigger jumps between frames. Second, a steady base controller handles calm motion. Third, a team of expert modules adds capacity only for hard, fast actions. The team also built a public test set of 3,593 high-dynamic clips. On it, their system cut tracking failures by about forty-three percent against the baseline.
Catalog record
| CREATOR | Tan Jing, Shiting Chen, Yangfan Li, Weisheng Xu, Renjing Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-19926
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control Tan Jing, Shiting Chen, Yangfan Li, Weisheng Xu, Renjing Xu Abstract Unified physics-based humanoid controllers are pivotal for robotics and character animation, yet models that excel on gentle, everyday motions still stumble on explosive actions, hampering real-world deployment. We bridge this gap with FARM (Frame-Accelerated Augmentation and Residual Mixture-of-Experts), an end-to-end framework composed of frame-accelerated augmentation, a robust base controller, and a residual mixture-of-experts (MoE). Frame-accelerated augmentation exposes the model to high-velocity pose changes by widening inter-frame gaps. The base controller reliably tracks everyday low-dynamic motions, while the residual MoE adaptively allocates additional network capacity to handle challenging high-dynamic actions, significantly enhancing tracking accuracy. In the absence of a public benchmark, we curate the High-Dynamic Humanoid Motion (HDHM) dataset, comprising 3593 physically plausible clips. On HDHM, FARM reduces the tracking failure rate by 42.8% and lowers global mean per-joint position error by 14.6% relative to the baseline, while preserving near-perfect accuracy on low-dynamic motions. These results establish FARM as a new baseline for high-dynamic humanoid control and introduce the first open benchmark dedicated to this challenge. The code and dataset will be released at https://github.com/Colin-Jing/FARM. 1 Introduction Physics-based humanoid motion imitation combines the visual fidelity of motion-capture data with the strict dynamics of rigid-body simulation, underpinning advanced robotics, digital characters, and immersive AR/VR experiences. In practice, however, each new target sequence still demands days of reward redesign and hyperparameter tuning. This heavy engineering overhead hampers large-scale adoption and highlights the need for a robust universal controller that can track a wide range of motions without per-task retraining. The Universal Humanoid Controller (UHC) ( Luo et al. 2021 ) first explored the challenge of universal motion tracking, achieving a 97% tracking success rate on the 10k-sequence AMASS ( Mahmood et al. 2019 ) dataset by incorporating a hand-crafted residual force controller (RFC) ( Yuan and Kitani 2020 ) . Perpetual Humanoid Control (PHC) ( Luo et al. 2023 ) advanced this by removing the RFC and introducing a progressive multiplicative control architecture, elevating the tracking success rate to 98.9%. Its subsequent refinement, PHC+ ( Luo et al. 2024 ) , further improved performance through data filtering and training optimizations, achieving near-100% success. However, the incremental learning approach inherent in PHC+ complicates the training process and hampers scalability. Addressing this complexity, the Fully Constrained (FC) Controller proposed in MaskedMimic ( Tessler et al. 2024 ) adopts a unified network architecture, simplifying training and yielding superior generalization capabilities. Despite these methodological advances, all aforementioned controllers share a critical limitation in that they are trained primarily on the AMASS dataset, dominated by simple, low-dynamic everyday motions. This restricts their ability to generalize to highly dynamic motions, leaving the robust control of explosive actions an open challenge in universal humanoid control. Building upon this limitation, we initially considered a straightforward intuition: since the AMASS dataset predominantly comprises low-dynamic actions with small pose-to-pose variations, artificially accelerating motion clips through uniform frame dropping might enlarge the pose differences between successive frames, mimicking the abrupt transitions seen in highly dynamic motions. However, experiments (Section 4.4 ) revealed that fine-tuning the FC controller directly on these frame-accelerated clips provided minimal improvement for high-dynamic actions, and even degraded its previously near-perfect accuracy on low-dynamic sequences. This prompted a second key insight inspired by human motor attention: routine, everyday movements demand minimal cognitive resources, whereas explosive, high-dynamic actions require heightened attention and additional processing capacity. Analogously, we reasoned that effective high-dynamic control necessitates dynamically adjusting network capacity to match motion intensity, ensuring high accuracy without compromising low-dynamic performance. Figure 1: Comparison between FARM and the baseline FC on two high-dynamic motions. FARM accurately completes both motions (green check), while FC falls or loses balance (red cross). Frames are shown in playback order from left to right. Motivated by these insights, we propose the FARM framework, which consists of three components: frame-accelerated augmentation, a base controller, and a residual mixture-of-experts (MoE). Frame-accelerated augmentation widens the pose gap between successive frames to expose the model to the abrupt transitions found in highly dynamic actions. The base controller delivers dependable baseline motion tracking for everyday movements. Building on this foundation, the residual MoE tackles the remaining high-dynamic errors. Its speed-aware router (SAR) stratifies motions by dynamic intensity and directs each band to a dedicated subset of experts, while the dynamic expert-assignment (DEA) adaptively scales capacity, activating only the required experts for low-dynamic segments and progressively more for highly dynamic ones. Together, these components enable FARM to master challenging high-dynamic motions while preserving strong performance on low-dynamic tasks. Figure 1 illustrates FARM’s ability to accurately complete complex high-dynamic motions, where the baseline FC model fails. Because no public dataset exists specifically for high-dynamic humanoid motions, we curate the High-Dynamic Humanoid Motion (HDHM) benchmark for controlled evaluation. HDHM comprises 3593 physics-plausible clips, each manually screened to ensure kinematic validity. On this benchmark, our FARM framework cuts the tracking failure rate by 42.8% and lowers the global mean per-joint position error (MPJPE g ) by 14.6% relative to the baseline, while preserving the near-perfect accuracy of the baseline on low-dynamic motions. 1. HDHM dataset. We release the first open, manually curated benchmark dedicated to high-dynamic humanoid control, comprising 3593 physics-plausible clips. 2. FARM framework. We propose a frame-accelerated residual MoE framework. Frame-accelerated augmentation allows the model to acquire high-dynamic skills from predominantly low-dynamic data, and the residual MoE adaptively allocates computational capacity according to motion intensity. 3. Comprehensive experiments. Extensive evaluations show that FARM significantly outperforms strong baselines on challenging high-dynamic motions while preserving the near-perfect accuracy on low-dynamic motions. 2 Related Works 2.1 Physics-based motion imitation Physics-based motion imitation learns control policies to track mocap references in simulation using reinforcement learning ( Mnih et al. 2016 ) . DeepMimic ( Peng et al. 2018 ) , AMP ( Peng et al. 2021 ) , and PHC ( Luo et al. 2023 ) progressively improved motion diversity and generalization, with PHC+ ( Luo et al. 2024 ) achieving near-perfect tracking on large-scale datasets like AMASS. MaskedMimic ( Tessler et al. 2024 ) further unified diverse conditioning forms into a single controller via masked modeling. While effective on standard mocap data, these methods struggle with highly dynamic motions due to limited exposure to fast transitions. Our approach addresses this by explicitly targeting high-dynamic regimes through data and architectural adaptations. 2.2 Mixture-of-experts in motion control Several recent works apply MoE to improve specialization and scalability in humanoid or legged control. GMT ( Chen et al. 2025 ) employs MoE alongside adaptive sampling to track diverse whole-body motions with a unified policy. MoRE ( Wang et al. 2025 ) uses a mixture of latent residual experts to learn human-like gaits across complex terrains. MoE‑Loco ( Huang et al. 2025 ) addresses multitask locomotion with dynamic selection of experts to mitigate gradient conflicts and support varied gait and terrain combinations. These methods demonstrate the benefits of MoE-based specialization. Unlike fixed expert usage in prior work, our residual MoE adapts the expert count based on motion intensity for better capacity allocation. 3 Method Figure 2: Overview of the FARM pipeline. Failure cases are mined by applying the FC model to the