ULC : A U nified and Fine-Grained C ontroller for Humanoid L oco-Manipulation

Wandong Sun, Luying Feng, Baoshi Cao, Yang Liu, Yaochu Jin, Zongwu Xie · 2025

Read ULC : A U nified and Fine-Grained C ontroller for Humanoid L oco-Manipulation by Wandong Sun, Luying Feng, Baoshi Cao, Yang Liu, Yaochu Jin, Zongwu Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06905.txt.

Introduction

Wandong Sun and co-authors present a 2025 robot control system called ULC for humanoid robots that walk and use their arms at the same time. Most past systems split the job, with one policy for the legs and one for the upper body. That split makes training simpler, but it limits how well the parts move as one whole.

ULC uses a single policy to track root speed, root height, torso turn, and both arms at once. The team tested it on a Unitree G1 robot with a three degree waist. It tracked targets better than strong split systems and covered a larger work space. The robot could also handle loads on its arms while keeping whole body control steady.

Catalog record

CREATORWandong Sun, Luying Feng, Baoshi Cao, Yang Liu, Yaochu Jin, Zongwu Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-06905

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ULC : A U nified and Fine-Grained C ontroller for Humanoid L oco-Manipulation Wandong Sun, Luying Feng, Baoshi Cao, Yang Liu, Yaochu Jin, Zongwu Xie Abstract Loco-Manipulation for humanoid robots aims to enable robots to integrate mobility with upper-body tracking capabilities. Most existing approaches adopt hierarchical architectures that decompose control into isolated upper-body (manipulation) and lower-body (locomotion) policies. While this decomposition reduces training complexity, it inherently limits coordination between subsystems and contradicts the unified whole-body control exhibited by humans. We demonstrate that a single unified policy can achieve a combination of tracking accuracy, large workspace, and robustness for humanoid loco-manipulation. We propose the U nified L oco-Manipulation C ontroller ( ULC ), a single-policy framework that simultaneously tracks root velocity, root height, torso rotation, and dual-arm joint positions in an end-to-end manner, proving the feasibility of unified control without sacrificing performance. We achieve this unified control through key technologies: sequence skill acquisition for progressive learning complexity, residual action modeling for fine-grained control adjustments, command polynomial interpolation for smooth motion transitions, random delay release for robustness to deploy variations, load randomization for generalization to external disturbances, and center-of-gravity tracking for providing explicit policy gradients to maintain stability. We validate our method on the Unitree G1 humanoid robot with 3-DOF (degrees-of-freedom) waist. Compared with strong baselines, ULC shows better tracking performance to disentangled methods and demonstrating larger workspace coverage. The unified dual-arm tracking enables precise manipulation under external loads while maintaining coordinated whole-body control for complex loco-manipulation tasks. The code and videos are available on our project website at https://ulc-humanoid.github.io . I INTRODUCTION Humanoid robots, with their human-like morphology, represent a promising paradigm for versatile robotic systems capable of operating in human-designed environments. Recent years have witnessed remarkable advances in locomotion [ 1 , 2 , 3 , 4 , 5 , 6 ] and autonomous manipulation [ 7 , 8 , 9 , 10 , 11 ] capabilities for humanoid platforms. These achievements are enabled by the synergistic integration of high-level decision-making layers, powered by Imitation Learning (IL) models [ 12 , 13 ] or Vision-Language-Action (VLA) models [ 14 , 15 , 16 ] , with sophisticated Loco-Manipulation Controllers (LMCs) [ 17 , 18 , 19 , 20 , 21 , 22 ] that translate high-level commands into precise whole-body motions for autonomous locomotion and dexterous dual-arm manipulation. Fig. 1 : Diverse loco-manipulation capabilities enabled by ULC . The humanoid robot demonstrates various coordinated whole-body actions including: picking up bread from a table and placing it in a refrigerator, pushing a cart with coordinated locomotion, squatting to shovel sand from the ground, lifting boxes from the floor to table height with dual-arm coordination, picking up dolls from the ground with hand switching and placing them on a sofa, sitting and playing ukulele with fine motor control, placing items in a microwave with precise manipulation, cleaning kitchen surfaces with wiping motions, erasing blackboards with arm coordination, and performing torso rotation in outdoor environments. An ideal Loco-Manipulation Controller (LMC) should seamlessly translate whole-body motion commands into precise joint-level actions, minimizing the discrepancy between commanded and executed motions while guaranteeing dynamic stability. However, designing effective LMCs involves several critical architectural decisions that significantly impact performance: • Command Space Selection . Robot actions can be parameterized through various representations [ 20 ] , including joint positions, Cartesian pose targets, root velocity, and root height. An effective command space should eliminate potential conflicts between different command modalities while enabling full exploration of the robot’s kinematic and dynamic capabilities. • Unified vs. Decoupled Control Architecture . Whole-body controllers [ 23 , 20 , 24 , 25 , 21 ] can theoretically achieve superior performance by coordinating all degrees of freedom simultaneously, but are commonly perceived as more challenging to train effectively compared to specialized controllers. Alternatively, some approaches decouple the LMC into separate upper and lower body controllers [ 19 , 22 , 18 , 17 ] , which accelerates learning but may compromise performance in scenarios requiring tight coordination between locomotion and manipulation. • Motion Capture vs. Procedural Training Data . Motion capture data provides physically plausible whole-body movement patterns, but inherent noise and kinematic infeasibilities can significantly degrade tracking accuracy [ 23 , 20 , 24 , 25 , 26 ] . Additionally, distribution bias in motion capture datasets poses risks when encountering out-of-distribution movements during deployment. Procedurally sampled command spaces can mitigate distribution bias but are primarily limited to upper-body motions, as the inherent instability of humanoid platforms precludes obtaining stable leg references through random sampling [ 17 , 18 , 19 , 21 ] . These design choices present fundamental trade-offs that significantly impact the practical deployment of humanoid loco-manipulation systems. Each decision involves balancing competing objectives: command space design must reconcile expressiveness with feasibility, control architecture must weigh coordination benefits against training complexity, and data generation must balance physical plausibility with distribution coverage. The challenge lies not in optimizing any single aspect, but in finding the optimal combination of design decisions that collectively enable robust, versatile, and deployable loco-manipulation capabilities. I-A Objective of this Work The objective of this work is to design a practical and versatile loco-manipulation controller that balances design decisions to best fit real-world scenarios. Specifically, we aim to develop a command space that comprehensively covers the majority of human mobility and manipulation scenarios while maintaining physical feasibility and eliminating command conflicts. We pursue a unified control architecture that coordinates the entire body simultaneously to maximize workspace coverage and enable tight coupling between locomotion and manipulation, challenging the conventional wisdom that decoupled approaches are necessary for practical deployment. To ensure robustness and generalization, we adopt procedural command generation rather than motion capture data, developing novel training methodologies that mitigate the inherent challenges of random sampling for humanoid locomotion while preserving the distribution coverage advantages. Our goal is to demonstrate that unified whole-body control can achieve both the precision of specialized controllers and the coordination benefits of integrated approaches, creating a system that is not only theoretically sound but practically deployable in real-world scenarios. I-B Contributions To achieve these objectives, we propose the U nified L oco-Manipulation C ontroller ( ULC ), a unified control framework that employs massively parallel reinforcement learning (RL) to accurately track procedurally sampled commands including root velocity, root height, torso orientation, and arm joint positions. This design choice deliberately simplifies leg commands compared to full motion capture approaches, but enables comprehensive coverage of the feasible command space through principled random sampling while preserving the coordination benefits of unified control. To realize these theoretical advantages in practice, we identify and address three fundamental technical challenges in developing ULC , presenting novel solutions for each: • Multi-Task Learning in Unified Control . Single-model multi-task tracking for humanoid robots often suffers from reduced single-task performance due to potential conflicts between heterogeneous command modalities and gradient interference across tasks [ 27 ] . We address these issues through: (1) careful command space design to ensure physical feasibility of command combinations, (2) progressive command curriculum learning from simple to difficult to enable systematic capability exploration, (3) residual action modeling [ 28 , 29 , 5 ] for both arms to enhance tracking precision, and (4) sequential skill acquisition [

Read the complete text