Object-Informed Model Predictive Path Integral Control for Non-Prehensile Robot Manipulation

Nikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov · 2026

Read Object-Informed Model Predictive Path Integral Control for Non-Prehensile Robot Manipulation by Nikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30778.txt.

Introduction

Raicevic and colleagues present a closed-loop object-informed method. Sampling-based control handles contact well but struggles in long-horizon planning. Their method splits the problem into object planning and robot planning selecting actions for goal poses. Path integral control solves the subproblems, coupled through consensus on object poses. The object plan is revised toward realizable trajectories, and the robot plan aligns with poses. In pushing tasks with obstacles, success rises by 35 percent in simulation and 43 percent on hardware. The comparison uses standard control with the same budget.

Catalog record

CREATORNikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37029
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30778

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Object-Informed Model Predictive Path Integral Control for Non-Prehensile Robot Manipulation Nikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov Abstract Long-horizon planning for non-prehensile robot manipulation is challenging due to underactuated and discontinuous interactions. We propose a hierarchical formulation of model predictive path integral (MPPI) control that guides robot-level planning with a separately computed object-level plan to achieve efficient long-horizon prediction. We first solve a simplified object-only problem, assuming the object can be actuated directly, and use the planned object trajectory as a reference in solving the joint robot-object planning problem. We evaluate our method in both simulation and hardware using a 6-DoF xArm6 manipulator to perform object pushing tasks in which the target object must reach a goal while avoiding static obstacles, necessitating non-myopic reasoning. Our object-informed MPPI increases task success by 40% with a 26% faster control frequency in simulation, and by 20% in real experiments with similar computation as regular MPPI. I Introduction Non-prehensile manipulation expands the capabilities of robot systems by allowing pushing interactions beyond stable grasping, as shown in Fig. 1 , which may be advantageous in certain scenarios. Recent advances in modeling and simulation of multi-body contact dynamics [ 13 , 19 ] have enabled object motion prediction under intermittent contact in parallel and at faster-than-real-time speeds on modern hardware. Furthermore, sampling-based model predictive control (MPC) algorithms such as model predictive path integral (MPPI) control [ 22 ] can utilize such fast parallelized simulators to achieve real-time control given a suitable cost description of the manipulation task [ 15 ] . However, long-horizon planning for non-prehensile manipulation remains challenging for two reasons. First, contact dynamics are discontinuous, so small changes in robot motion can produce qualitatively different object behaviors depending on how contact is established or broken. This renders large portions of the robot’s configuration space not useful for task progress. Second, the object is actuated only implicitly, and the robot must select its own motion to produce the desired object displacement through contact. This dramatically expands the search space, as the planner must simultaneously reason about robot kinematics, contact geometry, and object response. Fig. 1 : A manipulator pushes the orange letter I to form a complete phrase ”IROS” while avoiding the letters R , O , and S using the proposed object-informed MPPI approach. To address these challenges, we propose a hierarchical formulation of MPPI that guides robot-level planning with a separately computed object-level plan. The key idea is to decompose the manipulation task into two coupled optimal control problems operating at different abstraction levels, rather than planning directly in the joint robot-object state space, as illustrated in Fig. 2 . First, we solve an object-level optimal control problem using MPPI by treating the object as directly actuated through virtual control inputs, as shown in Fig. 2 (b). Second, we solve a robot-level control problem using MPPI, where the robot samples control trajectories that align with the planned object trajectory from the object-level problem, as shown in Fig. 2 (c). This hierarchical decomposition reduces the effective planning horizon at the robot level, while allowing the object-level planner to reason at a longer horizon in the smaller configuration space of object motions. We explore two variants: sequential object informed (SOI) MPPI , which performs a single object-planning stage in the beginning, and closed-loop object-informed (CLOI) MPPI , which re-computes the object trajectory in closed-loop given the latest object position, as illustrated in Fig. 2 . Fig. 2 : Object-Informed hierarchical planning. Starting from an initial configuration (a), the object-level planner generates a desired object trajectory x_{t,0:H^{o}}^{o*} (b), which is passed to the robot-level planner for tracking (c). After execution, the updated object state x_{t+1}^{o} is fed back to the object-level planner for replanning (d), forming a closed loop. The dashed red region represents SOI, which performs a single object-planning stage, while the dashed black region represents CLOI, which performs object-planning in a closed loop. Our contributions are summarized as follows. • We introduce two hierarchical MPPI variants, CLOI and SOI, that decompose non-prehensile manipulation into object-level and robot-level planning problems. • We demonstrate in simulated and real experiments using a 6-DoF robotic manipulator that our object-informed MPPI algorithms enable long-horizon non-prehensile manipulation with significantly improved success rate, by 40\% in simulation and 20\% on hardware, compared to standard MPPI. II Related Work Non-prehensile manipulation is a challenging robotic task that demands reasoning about making and breaking contact with objects of interest. Contact forces can be modeled using complementarity constraints [ 18 ] or an equivalent second-order cone program (SOCP) [ 1 ] . Recent work focuses on directly optimizing through such analytical models of contact dynamics by forming local approximations of the complementarity constraints [ 16 , 11 , 6 ] , smoothing the SOCP formulation [ 14 ] , or constructing an explicit SOCP solution [ 8 ] . Although these analytical approaches are physically grounded, they require carefully designed differentiable models of contact, which may be difficult to obtain for arbitrary objects. Sampling-based MPC relies on parallel forward simulation in a multi-body physics simulation engine without gradient information. Notable methods include the cross entropy method (CEM) [ 5 ] , MPPI [ 22 ] , and diffusion-inspired annealing (DIAL) MPC [ 23 ] . With appropriate cost definitions and control parameterization, sampling-based MPC has proven useful in manipulation tasks [ 2 ] and benefits from GPU-accelerated, high-fidelity simulation [ 15 ] . These methods iteratively update a distribution over control sequences based on the cumulative cost of online interactions with the simulation environment resulting from control sequences sampled from the distribution [ 21 ] . Our work is inspired by the success of sampling-based MPC in handling complicated dynamics via parallel forward simulation, and seeks to capture the hierarchical robot-object structure present in non-prehensile manipulation. Some prior approaches offer insight into imposing structure for non-prehensile manipulation. Recently, consensus complementarity control (C3) [ 20 ] applied the alternating direction method of multipliers (ADMM) to solve non-prehensile manipulation via consensus between two iterative steps: the first solving for control actions without complementarity constraints, and the second projecting the solution to satisfy complementarity constraints. This approach isolates the non-convexity of the complementarity constraints and exploits their temporal independence. In related works, explicit choice of contact modes has proven to be beneficial for performance and computational efficiency [ 10 , 3 , 7 ] . Closely related to our approach are the object-centric planning methods [ 17 , 9 ] , which first plan a full object-only trajectory that achieves the goal, and subsequently solve for a robot motion that achieves the object trajectory. In comparison, our approach can also create object plans in a receding horizon to close the loop with object pose estimates. We also contribute a clear mathematical formulation of this decomposition between robot- and object-level planning problems based on a general dynamic model. III Problem Statement Consider a robot manipulator modeled in discrete-time input-affine form: x_{t+1}^{r}\;=\;f_{r}(x_{t}^{r})\;+\;g_{r}(x_{t}^{r})\,u_{t}^{r}\;+\;h_{r}(x_{t}^{r})\,w_{t}^{r}, (1) where x_{t}^{r}\in\mathbb{R}^{n_{r}} is the robot’s configuration at time t , u_{t}^{r}\in\mathbb{R}^{m_{r}} denotes the control input (e.g., joint torque or desired velocity), and w_{t}^{r}\in\mathbb{R}^{p_{r}} represents external forces induced by contact with the object or the environment. The robot’s task is to

Read the complete text