MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation

Zhenyu Wu, Angyuan Ma, Xiuwei Xu, Hang Yin, Yinan Liang, Ziwei Wang, Jiwen Lu, Haibin Yan · 2025

Read MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation by Zhenyu Wu, Angyuan Ma, Xiuwei Xu, Hang Yin, Yinan Liang, Ziwei Wang, Jiwen Lu, Haibin Yan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01658.txt.

Introduction

Zhenyu Wu and seven co-authors present MoTo, an add-on module that lets a fixed robot arm work while its base moves. Fixed arm systems can handle many tasks but cannot travel across a room. MoTo adds travel skill without new training data drawn from moving robots. Vision and language models help the system find where the robot should park.

The method picks docking points near a target object and plans paths for both the base and the arm. Key points on the object and the arm guide the motion so the arm stays in reach. Tests on a standard benchmark and with a real robot beat prior methods on success rate. The gain comes with no extra training examples.

Catalog record

CREATORZhenyu Wu, Angyuan Ma, Xiuwei Xu, Hang Yin, Yinan Liang, Ziwei Wang, Jiwen Lu, Haibin Yan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01658

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation Zhenyu Wu, Angyuan Ma, Xiuwei Xu, Hang Yin, Yinan Liang, Ziwei Wang, Jiwen Lu, Haibin Yan Abstract Mobile manipulation stands as a core challenge in robotics, enabling robots to assist humans across varied tasks and dynamic daily environments. Conventional mobile manipulation approaches often struggle to generalize across different tasks and environments due to the lack of large-scale training. However, recent advances in manipulation foundation models demonstrate impressive generalization capability on a wide range of fixed-base manipulation tasks, which are still limited to a fixed setting. Therefore, we devise a plug-in module named MoTo, which can be combined with any off-the-shelf manipulation foundation model to empower them with mobile manipulation ability. Specifically, we propose an interaction-aware navigation policy to generate robot docking points for generalized mobile manipulation. To enable zero-shot ability, we propose an interaction keypoints framework via vision-language models (VLM) under multi-view consistency for both target object and robotic arm following instructions, where fixed-base manipulation foundation models can be employed. We further propose motion planning objectives for the mobile base and robot arm, which minimize the distance between the two keypoints and maintain the physical feasibility of trajectories. In this way, MoTo guides the robot to move to the docking points where fixed-base manipulation can be successfully performed, and leverages VLM generation and trajectory optimization to achieve mobile manipulation in a zero-shot manner, without any requirement on mobile manipulation expert data. Extensive experimental results on OVMM and real-world demonstrate that MoTo achieves success rates of 2.68% and 16.67% higher than the state-of-the-art mobile manipulation methods, respectively, without requiring additional training data. Our project homepage can be found at here . Keywords: Mobile Manipulation, Vision Language Action Models 1 Introduction Mobile manipulation allows robots to operate effectively in expansive environments, where solving complex tasks requires the joint coordination of the robotic arm and mobile base [ 1 , 2 , 3 ] . The rapid proliferation of intelligent robotic systems has created an urgent demand for mobile manipulation in diverse application domains, including household services [ 4 , 5 ] , manufacturing [ 6 ] , and logistics [ 7 ] , where robots must autonomously execute cross-space manipulation tasks. However, enabling robots to operate across diverse tasks in unstructured environments (e.g., human daily-life assistance) remains highly challenging. Therefore, designing a general mobile manipulation framework for real-world deployments is desirable. Existing mobile manipulation approaches [ 8 , 9 ] propose end-to-end frameworks to jointly search the navigation and manipulation action spaces to improve performance. However, the end-to-end mobile manipulation policy models lack training on large-scale mobile manipulation datasets due to extremely expensive collection costs for mobile manipulation demonstration, which results in low generalization across tasks and environments. On the other side, the emerging manipulation foundation models [ 10 , 11 , 12 ] have demonstrated high generalization capabilities across different manipulation tasks and diverse deployment scenes. However, these models are not able to predict the movement of the base and thus are limited to fixed-base manipulation tasks. A natural idea is to build a mobile manipulation framework by directly patching together the LLM or VLM with the manipulation foundation models. Conventional mobile manipulation frameworks employ navigation [ 13 , 14 ] and fixed-base manipulation [ 15 , 16 ] modules separately to accomplish mobile manipulation tasks. However, naive combining navigation and manipulation results in compounding errors since the large gap between the goals of navigation and manipulation [ 17 ] . Finding suitable navigation docking points for interaction remains a huge challenge in mobile manipulation tasks. Figure 1: MoTo can be plugged into any fixed-base manipulation model and transferred to mobile manipulation tasks in a zero-shot manner, enabling generalized mobile manipulation. In this paper, we propose to solve the problem of mobile manipulation with an interaction-aware navigation policy, namely Mo ve and To uch (MoTo). MoTo is a zero-shot plug-in that can be combined with any off-the-shelf fixed-base manipulation model and empower it with mobile manipulation ability. The proposed interaction-aware navigation policy generates suitable base docking points for the robot to successfully perform manipulation, which are inspired by the fact that the robot can perform subsequent fixed-base manipulation actions on the docking points where its arms can reach the target object interaction region. We prompt the vision-language model (VLM) to generate task-related keypoints on the target object and robotic arm to enable zero-shot mobile manipulation. We can control the robot by solving an optimization problem that minimizes the distance between the two keypoints and considers several additional constraints, including collision avoidance, smoothness, and arm margin. Figure 1 illustrates the proposed approaches. MoTo efficiently transfers manipulation foundation models to diverse mobile manipulation tasks, where the mobile base and the robot arm coordinately perform actions with physically feasible trajectories. Extensive experimental results on OVMM [ 18 ] and real-world demonstrate that MoTo achieves 2.68% and 16.67% higher success rates compared to state-of-the-art mobile manipulation techniques, respectively. 2 Related Works Mobile Manipulation Frameworks: Existing mobile manipulation frameworks can be categorized into end-to-end and modular based on the model structure. The end-to-end approaches [ 19 , 20 , 21 , 22 ] directly map visual observations to the mobile manipulation action space. M 2 Diffuser [ 23 ] guides the diffusion process with various energy terms to ensure that the trajectory of the mobile manipulation follows the physical constraints. However, end-to-end approaches require extensive training data leading to high costs [ 24 ] . The modular mobile manipulation framework [ 18 , 25 , 26 ] benefits from off-the-shelf navigation and manipulation frameworks exhibiting outstanding data efficiency. IALP [ 27 ] leverages LLM reasoning capability to generate manipulation actions in a closed-loop interactive manner. MoManipVLA [ 3 ] achieves data-efficient generalized mobile manipulation via optimization of the base waypoints to ensure that the VLA prediction trajectories are feasible. AMR [ 28 ] and LIMP [ 29 ] leverage language instructions to guide robots to target poses, demonstrating the potential of vision-language navigation for mobile manipulation. However, naively combining navigation and manipulation complicates instructions and leads to compounding errors, as humans may not know optimal base positions for manipulation. Trajectory Optimization: Trajectory optimization is important for robots to perform precise manipulation actions. Early research [ 30 , 31 , 32 ] due to the hand‐crafted cost terms restricts deployment in diverse and complex environments. To improve scalability, researchers focus on data-driven learnable methods. Contact-GraspNet [ 33 ] and O2O-Afford [ 34 ] generate trajectories with learned grasping poses and object affordances, respectively. Diffusion policy-based methods [ 35 , 36 ] learn collision-free trajectories directly from expert episodes and achieve promising performance. More recently, foundation‑model–based frameworks like VoxPoser [ 37 ] and ReKep [ 38 ] leverage pretrained priors to infer physical constraints, which significantly improve generalization. Inspired by ReKep, we propose a multi-view voting strategy to generate scene-level interaction keypoints to fine-grain guide mobile manipulation trajectory generation. 3 Problem Statement Our goal is to enable robots to perform long-horizon mobile manipulation tasks with strong generalization ability to unseen environments and goals. The task is described by a free-form language instruction \mathcal{T} . For example, “I need to drink water”, the robot should first pick up a

Read the complete text