Where to Touch, How to Contact: Hierarchical RL–MPC Framework for Geometry-Aware Long-Horizon Dexterous Manipulation
Zhixian Xie, Yu Xiang, Michael Posa, Wanxin Jin · 2026
Read Where to Touch, How to Contact: Hierarchical RL–MPC Framework for Geometry-Aware Long-Horizon Dexterous Manipulation by Zhixian Xie, Yu Xiang, Michael Posa, Wanxin Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10930.txt.
Introduction
Zhixian Xie, Yu Xiang, Michael Posa, and Wanxin Jin build a framework for dexterous robot manipulation over long horizons. Contact-rich tasks demand joint reasoning about shape, joint limits, and rough contact dynamics. End-to-end learned policies skip that structure. They often need huge data, transfer poorly from simulation, and generalize weakly across tasks and robot bodies.
The authors combine reinforcement learning with model predictive control in a hierarchy. Their frame is geometry-aware: where to touch, and how to make contact, are decided as linked questions. The hierarchy lets learning handle what models miss, and models keep learning grounded. The work targets hands that must work in the real world, not just in simulators.
Catalog record
| CREATOR | Zhixian Xie, Yu Xiang, Michael Posa, Wanxin Jin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-10930
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Where to Touch, How to Contact: Hierarchical RL–MPC Framework for Geometry-Aware Long-Horizon Dexterous Manipulation Zhixian Xie, Yu Xiang, Michael Posa, Wanxin Jin Abstract A key challenge in contact-rich dexterous manipulation is the need to jointly reason over geometry, kinematic constraints, and intricate, nonsmooth contact dynamics. End-to-end visuomotor policies bypass this structure, but often require large amounts of data, transfer poorly from simulation to reality, and generalize weakly across tasks/embodiments. We address those limitations by leveraging a simple insight: dexterous manipulation is inherently hierarchical—at a high level, a robot decides where to touch (geometry) and move the object (kinematics); at a low level it determines how to realize that plan through contact dynamics. Building on this insight, we propose a hierarchical RL–MPC framework in which a high-level reinforcement learning (RL) policy predicts a contact intention, a novel object-centric interface that specifies (i) an object-surface contact location and (ii) a post-contact object-level subgoal pose. Conditioned on this contact intention, a low-level contact-implicit model predictive control (MPC) optimizes local contact modes and replans with contact dynamics to generate robot actions that robustly drive the object toward each subgoal. We evaluate the framework on non-prehensile tasks, including geometry-generalized pushing and object 3D reorientation. It achieves near-100% success with substantially reduced data (10 × \times less than end-to-end baselines), highly robust performance, and zero-shot sim-to-real transfer. I Introduction Dexterous manipulation is a hallmark skill of intelligent robots. Recent research has made significant progress by training robot policies either from demonstrations, e.g., imitation learning [ 65 , 8 ] and vision language action model [ 4 , 26 , 15 ] , or from robot experience, e.g., reinforcement learning [ 42 , 63 ] . These approaches typically learn a end-to-end policy from robot observations to low-level motor commands. While end-to-end methods can achieve impressive performance, such as emergent generalization [ 51 ] and robustness [ 30 ] , they often face practical challenges, such as data scarcity, sim-to-real gap, and lack of task or embodiment generalizability. Yet, human manipulation does not necessarily use a monolithic end-to-end motor policy [ 37 , 17 ] . For instance, when picking up a hammer to strike a nail,we do not directly infer low-level finger–joint commands from raw perception. Instead, we reason hierarchically: at a higher level, we decide where to contact (e.g., grasping the handle) and what object-level outcome to achieve (e.g., aligning the head with the nail); at lower level, we determine how to realize this plan guided by our intuitive physics, leveraging interaction strategies such as rolling, sliding, sticking, and disengagement. This hierarchical decomposition enables humans to combine semantic and geometric understanding and physics intuition, yielding efficient and robust manipulation control. In this work, we propose an analogous hierarchical reasoning framework for robotic contact-rich manipulation. The hierarchical framework includes • Higher level: geometric-kinematic reasoning. Given a scene observation, the higher-level learning-based policy determines where on the object surface the robot should establish contact (geometry) and where the object should move as a result (kinematics). To formalize this decision, we introduce a novel representation, termed Contact Intention , which specifies (i) contact locations on the object surface and (ii) a post-contact object subgoal for the desired object-level motion progress. We train this higher-level policy using reinforcement learning • Lower level: contact dynamics reasoning. Conditioned on the generated contact intention, the dynamic contact planner generates joint-level commands by optimizing with contact dynamics. Leveraging recent advances in real-time contact-implicit model predictive control, this level synthesizes local contact modes, including rolling, sticking, sliding, and purposeful disengagement, at the suggested contact location, robustly moving the object to the planned subgoal. The two levels forms a hierarchical RL–MPC framework that offers the following benefits: (I) Significant data efficiency for geometry-aware long-horizon manipulation. By delegating contact dynamics reasoning to MPC optimization, the framework avoids inefficient learning over non-smooth contact dynamics. This allows the learning policy to focus on geometry and kinematics, enabling more efficient learning of long-horizon contact behaviors. (II) Robust performance enabled by contact implicit MPC. The low-level MPC replans at high frequency ( \sim 100 Hz) with contact dynamics, continuously correcting for disturbances and modeling errors. This reduces sensitivity to simulation imperfections and improves robustness in sim-to-real deployment. (III) Contact intention interface facilitating sim-to-real transfer. The contact-intention interface decouples the geometric-kinematic reasoning from low-level contact dynamics. This allows training a policy in simulator where geometry/kinematics transfer more reliably than contact dynamics. It makes the learned higher-level RL policy inherently more robust for sim-to-real deployment than an end-to-end policy that must account for hard-to-model contact dynamics. In sum, we highlight our following contributions: (1) We introduce contact intention , a hierarchical policy interface that decomposes dexterous manipulation into geometric–kinematic reasoning (where to touch) and contact dynamics reasoning (how to contact). It enables data-efficient, geometry-aware manipulation with robust sim-to-real transfer. (2) We develop a hierarchical RL–MPC framework built on the contact intention interface. The higher-level RL policy predicts the contact intention, which then parameterizes a lower-level MPC for contact dynamics reasoning. This framework demonstrated state-of-the-art performance in data efficiency, success rate, robustness for geometry-aware long-horizon non-prehensile manipulation. (3) We introduce multiple design novelties for higher-level RL policy training, including (i) a tri-component object-centric representation space that jointly encodes object geometry, task progress, and collision feasibility, and (ii) a dual-branch policy network architecture that captures both local geometry and global kinematic context. These designs enable a unified policy to learn non-prehensile manipulation across diverse objects and to generalize to unseen geometries. II Related Works II-A RL-MPC Hierarchy in Robot Manipulation The use of RL–MPC architectures is extensive in robotic locomotion [ 61 , 64 , 6 , 23 , 19 ] , where the primary objective is to achieve agile, adaptive, and diverse motor behaviors by combining learned high-level decision making with model-based control. This paradigm has more recently been transferred to robotic manipulation, where adaptability and generalization are crucial. For instance, in [ 47 , 33 ] , reinforcement learning is employed to infer the latent dynamics of deformable objects, which are subsequently incorporated into MPC or adaptive control for accurate manipulation. Other works [ 40 , 34 ] leverage RL to assist with learning contact-rich dynamics models and to provide a policy warm-start for sampling-based MPC solvers. In another line of research [ 3 , 68 ] , RL predicts intermediate end-effector goals that are tracked by a low-level MPC controller, thereby ensuring feasibility and safety during contact-rich tasks. Unlike prior uses of RL as auxiliary guidance or trajectory prediction, we introduce a tightly coupled RL–MPC hierarchy enabled by a novel contact intention interface for contact-rich manipulation. This design allows RL to reason over object geometry and kinematics while MPC handles high-frequency dynamic contact strategies, together enabling robust and dexterous manipulation. II-B Non-Prehensile Manipulation Since the proposed method is primarily applied to non-prehensile manipulation, we review the relevant prior work below. Non-prehensile manipulation [ 36 ] is defined as manipulating objects without explicitly grasping them. Classic work in non-prehensile manipulation studied the mechanics and control of the manipulator-object motion primitives [ 35 , 1 ] . However, motion primitives face scalability issue in the open-ended world. Recent effort attacks the problem through contact-implicit planning [ 38 , 39 , 62 , 55