HoMeR : Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control
Priya Sundaresan 1 , Rhea Malhotra 1 , Phillip Miao 1 , Jingyun Yang 1 , Jimmy Wu 1 , Hengyuan Hu 1 , Rika Antonova 1,2 , Francis Engelmann 1 , Dorsa Sadigh 1 , Jeannette Bohg 1 · 2025
Read HoMeR : Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control by Priya Sundaresan 1 , Rhea Malhotra 1 , Phillip Miao 1 , Jingyun Yang 1 , Jimmy Wu 1 , Hengyuan Hu 1 , Rika Antonova 1,2 , Francis Engelmann 1 , Dorsa Sadigh 1 , Jeannette Bohg 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01185.txt.
Introduction
Home robots need to move through a room and use their arms at the same time, a skill called mobile manipulation. Priya Sundaresan and co-authors present HoMeR, a learning framework for this in real homes. A fast controller based on body geometry turns a desired hand position into coordinated motion of the base and arm. Learning then works in that simpler hand space. The system switches between two prediction styles: absolute positions for long reaches, and small relative steps for delicate work.
HoMeR was tested on a wheeled robot with a seven joint arm, in both simulation and a real home, on tasks like opening cabinets, sweeping trash, and moving pillows. With only twenty demonstrations per task it succeeded about four times out of five, far ahead of the next best method. It can also borrow knowledge from vision and language models to handle new objects and clutter.
Catalog record
| CREATOR | Priya Sundaresan 1 , Rhea Malhotra 1 , Phillip Miao 1 , Jingyun Yang 1 , Jimmy Wu 1 , Hengyuan Hu 1 , Rika Antonova 1,2 , Francis Engelmann 1 , Dorsa Sadigh 1 , Jeannette Bohg 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01185
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HoMeR : Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control Priya Sundaresan 1 , Rhea Malhotra 1 , Phillip Miao 1 , Jingyun Yang 1 , Jimmy Wu 1 , Hengyuan Hu 1 , Rika Antonova 1,2 , Francis Engelmann 1 , Dorsa Sadigh 1 , Jeannette Bohg 1 Abstract We introduce HoMeR , an imitation learning framework for mobile manipulation that combines whole-body control with hybrid action modes that handle both long-range and fine-grained motion, enabling effective performance on realistic in-the-wild tasks. At its core is a fast, kinematics-based whole-body controller that maps desired end-effector poses to coordinated motion across the mobile base and arm. Within this reduced end-effector action space, HoMeR learns to switch between absolute pose predictions for long-range movement and relative pose predictions for fine-grained manipulation, offloading low-level coordination to the controller and focusing learning on task-level decisions. We deploy HoMeR on a holonomic mobile manipulator with a 7-DoF arm in a real home. We compare HoMeR to baselines without hybrid actions or whole-body control across 3 simulated and 3 real household tasks such as opening cabinets, sweeping trash, and rearranging pillows. Across tasks, HoMeR achieves an overall success rate of 79.17 % \mathbf{79.17\%} using just 20 demonstrations per task, outperforming the next best baseline by 29.17 % \mathbf{29.17\%} on average. HoMeR is also compatible with vision-language models and can leverage their internet-scale priors to better generalize to novel object appearances, layouts, and cluttered scenes. In summary, HoMeR moves beyond tabletop settings and demonstrates a scalable path toward sample-efficient, generalizable manipulation in everyday indoor spaces. Code, videos, and supplementary material are available at: http://homer-manip.github.io . Keywords: Imitation Learning, Mobile Manipulation, Whole-Body Control Figure 1: HoMeR . Left: A demonstrator uses whole-body iPhone teleoperation to collect data with a mobile manipulator in a real home. Right: From these collected demonstrations, HoMeR learns a hybrid imitation learning policy that switches between absolute actions for reaching, and relative actions for fine manipulation. A whole-body controller maps these end-effector commands to arm and base joint commands for execution. 1 Introduction To unlock the full potential of robots, we must move beyond controlled lab spaces and into the diverse, unstructured environments of everyday life. Unlike stationary tabletop robots, which lack the mobility to perform the wide range of tasks found in homes, offices, and warehouses, mobile manipulators are capable of navigating and interacting in these human-centric spaces. Tasks like watering a row of plants, wiping a spill on a long table, or moving to open a cabinet ( Fig. 1 ) require not only precision, but also manipulation and mobility working hand-in-hand. Modern mobile embodiments such as wheeled base-arm platforms [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] , humanoids [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ] , and quadrupeds [ 18 , 19 , 20 , 21 , 22 ] enable robots to operate beyond fixed workspaces. However, these embodiments introduce significant control complexity. In particular, they require careful coordination between the base and arm in wheeled embodiments, or the limbs and torso in legged systems. One common approach to managing this complexity is to use whole-body controllers (WBCs), which map high-level end-effector commands to coordinated whole-body motion using analytical or learning-based approaches. Prior work on WBCs has focused primarily on quadrupeds [ 23 , 24 , 25 , 26 ] , where achieving balance and stability requires learning WBCs from scratch with extensive reward shaping and sim-to-real transfer techniques, often tailored to the dynamics of legged robots. Furthermore, real-world tasks are naturally multi-phase, combining long-range movements (e.g., reaching or repositioning) with fine-grained local manipulation of objects. On the policy learning side, recent imitation learning (IL) methods [ 27 , 28 , 29 ] have shown the benefits of hybrid action spaces, although their application has been restricted to tabletop domains. These policies typically learn to alternate between predicting (1) keypose actions (6 DoF absolute end-effector poses, equivalently referred to as waypoint [ 27 , 28 , 29 ] or keyframe [ 30 , 31 ] actions) for long-range movement and (2) dense actions ( i.e. , delta end-effector actions) for fine-grained interactions. While these approaches achieve strong performance and generalization from limited demonstrations in tabletop settings, they have not been introduced in the mobile manipulation space with greater embodiment complexity, larger workspaces, and longer task horizons. Our key insight is that scalable mobile manipulation requires not only an effective strategy for managing control complexity, but also a means of generalizing to novel scenarios. As mobile robots move through diverse human environments, they are exposed to far greater variability in objects, spatial configurations, and task conditions than static arms. To address both challenges, we propose HoMeR : H ybrid wh o le-body policies for M obil e R obots. HoMeR ( Fig. 1 ) combines a fast, kinematics-based whole-body controller (which maps end-effector actions to coordinated base-arm motion) with a hybrid IL policy that switches between keypose predictions for long-range movement and dense delta actions for fine-grained manipulation. This structured approach addresses high-dimensional control and multi-phase execution. Additionally, we show that HoMeR is modular enough to incorporate task-relevant keypoints derived from vision-language models (VLMs), providing a path towards improved generalization in unfamiliar environments. We deploy HoMeR in a real home environment and evaluate it on a suite of challenging mobile manipulation tasks that reflect everyday household demands. Overall, our contributions are: 1. A sample-efficient imitation learning framework for mobile manipulation that leverages WBC and hybrid action representations to outperform strong non-hybrid and non-WBC baselines using only 20 demonstrations per task. 2. A modular policy architecture that can be conditioned on VLM keypoints , enabling generalization to novel object geometries, appearances, and cluttered environments. 3. A practical whole-body controller that supports intuitive teleoperation , facilitating efficient demonstration collection in real household settings. 2 Related Work Mobile Manipulation and Control. Legged platforms typically focus on navigation across diverse terrains ( e.g. , with Boston Dynamics Spot [ 21 ] and ANYmal [ 32 ] quadrupeds). A few recent works equip quadrupeds with lightweight arms and develop whole-body controllers, utilizing either model-based motion planning ( e.g. , RoLoMa [ 33 ] ) or learning-based manipulation policies ( e.g. , DeepWBC [ 24 ] , Visual WBC [ 25 ] , UMI-on-Legs [ 23 ] ). Our work draws inspiration from these works, but our framework is agnostic to the exact implementation of the WBC ( e.g. , inverse kinematics (IK) or learning-based). In our work, we use a task-agnostic IK-based WBC that is reusable across many scenarios. Furthermore, in contrast to these prior works, we learn policies with hybrid action modes to encourage both spatial generalization and precision. More recently, humanoid research has advanced rapidly, with most efforts centered on whole-body control for expressive behaviors such as dancing, walking, or jumping [ 10 , 11 ] . Although some humanoid systems perform manipulation, they often rely on using high-dimensional joint-space actions ( e.g. , 50+ DoF for HumanPlus [ 12 ] ) for imitation learning. In contrast, our work adopts a whole-body control strategy with hybrid action modes to enable