Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments
Chrisantus Eze, Ryan C Julian, Christopher Crick, Oklahoma State University, Google DeepMind · 2026
Read Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments by Chrisantus Eze, Ryan C Julian, Christopher Crick, Oklahoma State University, Google DeepMind (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02511.txt.
Introduction
Chrisantus Eze, Ryan Julian, and Christopher Crick retrieve objects from dense clutter with a robot in this 2026 paper. Large end-to-end models can manipulate objects but learn slowly and resist inspection. The authors argue for specialized, decoupled systems instead. Their framework, Unveiler, splits spatial reasoning from action execution.
A lightweight transformer encoder scans the clutter and picks the most critical obstacle. That choice passes to a rotation-invariant decoder that carries out the removal. Training starts by imitating a heuristic, then reinforcement fine-tuning discovers better strategies. Success reached high levels even when targets were fully hidden in simulation. The reasoning module moved to real scenes with no retraining. A physical robot validated the full system after only geometric calibration.
Catalog record
| CREATOR | Chrisantus Eze, Ryan C Julian, Christopher Crick, Oklahoma State University, Google DeepMind |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36991 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.02511
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments Chrisantus Eze, Ryan C Julian, Christopher Crick, Oklahoma State University, Google DeepMind Abstract Robotic manipulation in cluttered environments presents a critical challenge for automation. Recent large-scale, end-to-end models demonstrate impressive capabilities but often lack the data efficiency and modularity required for retrieving objects in dense clutter. In this work, we argue for a paradigm of specialized, decoupled systems and present Unveiler, a framework that explicitly separates high-level spatial reasoning from low-level action execution. Unveiler’s core is a lightweight, transformer-based Spatial Relationship Encoder (SRE) that sequentially identifies the most critical obstacle for removal. This discrete decision is then passed to a rotation-invariant Action Decoder for execution. We demonstrate that this decoupled architecture is not only more computationally efficient in terms of parameter count and inference time, but also significantly outperforms both classic end-to-end policies and modern, large-model-based baselines in retrieving targets from dense clutter. The SRE is trained in two stages: imitation learning from heuristic demonstrations provides sample-efficient initialization, after which PPO fine-tuning enables the policy to discover removal strategies that surpass the heuristic in dense clutter. Our results, achieving up to 97.6% success in partially occluded and 90.0% in fully occluded scenarios in simulation, make a case for the power of specialized, object-centric reasoning in complex manipulation tasks. Additionally, we demonstrate that the SRE’s spatial reasoning transfers zero-shot to real scenes, and validate the full system on a physical robot requiring only geometric workspace calibration; no learned components are retrained. I Introduction Fig. 1: Unveiling occluded targets through strategic obstacle removal. Given a cluttered scene with objects of similar shapes and colors, our approach identifies which obstacles must be removed and in what order to access the target object (blue object), solving the spatial reasoning challenge before action execution. Retrieving an occluded object from dense clutter requires solving two distinct sub-problems: which objects to remove and in what order, then how physically to remove each one. Methods that conflate these, mapping pixels directly to grasp actions [ 23 , 29 , 12 , 13 ] , force a single model to solve object selection and execution jointly, which scales poorly as scene complexity grows. Large-scale end-to-end and vision-language-action (VLA) approaches [ 9 , 2 , 1 ] offer impressive generality, but at either a prohibitive cost or with long inference latencies exceeding 6,000ms per decision (see Table I ), neither of which fits the tight budgets of real manipulation systems. We argue that explicit decomposition is the right inductive bias for this problem. By separating a lightweight spatial reasoning module, which identifies the next obstacle, from a dedicated execution module, which removes it, each component learns a strictly simpler problem, failures become independently diagnosable, and the system remains deployable on resource-constrained hardware. The Unveiler’s Spatial Relationship Encoder (SRE) and Action Decoder utilize only 83.03M parameters yet achieve 97.6% success in partially occluded scenarios, outperforming models with significantly larger parameter counts, such as [ 27 , 18 , 17 ] , by over 20% (see Tables I and II ). Furthermore, we show that VLAs can reason spatially in simple scenes but degrade in dense clutter. Our primary contributions are: • A decomposed architecture for cluttered manipulation that separates spatial reasoning from action execution, enabling the SRE to focus on obstacle dependency modeling, while the Action Decoder handles robust push-grasp execution. This decomposition yields 15-40% higher success rates than monolithic approaches across varying clutter densities. • A transformer-based spatial relationship encoder that learns to identify optimal obstacle removal sequences from heuristic demonstrations, achieving up to 97.6% accuracy in densely cluttered scenes in simulation. • A two-stage SRE training pipeline in which imitation learning from heuristic demonstrations initializes the policy sample-efficiently, and PPO fine-tuning then enables the SRE to discover removal sequences that surpass the heuristic in dense, fully occluded scenes. We train our framework entirely using simulated demonstrations in PyBullet [ 4 ] and evaluate it across multiple scene complexities (ranging from 2 - 12 objects), levels of target occlusion, and both simulated and real-world settings. Our results show that Unveiler significantly outperforms strong baselines in terms of task completion and planning efficiency. By focusing on the specific problem of “unveiling” a target, our work makes a case for the continued relevance of purpose-built systems in an era of large-scale models, which can serve as a complement. II Related Work Our work builds upon and extends existing work in robotic manipulation, particularly in the areas of push-grasping synergy, object-centric representation learning, and foundation models for robotics. II-A Push-Grasping Synergy The concept of coordinating pushing and grasping actions to manage clutter and improve grasping success is a well-established area of research [ 30 , 23 , 5 , 28 ] . Early work, such as [ 30 ] , introduced the idea of using self-supervised deep RL to learn the synergistic relationship between pushing and grasping. The TransporterNet architecture [ 29 ] , a significant advancement, uses a fully convolutional network to learn visuomotor policies for both pushing and grasping, demonstrating impressive performance in cluttered environments. However, TransporterNet and its successors often rely on a two-stage process, first pushing to clear space and then grasping, without a more integrated approach to manipulation. More recent methods have sought to improve upon this two-stage paradigm [ 12 , 13 ] . MPGNet [ 13 ] , for example, introduces a network that learns to synergize “move”, “push”, and “grasp” actions for more efficient target-oriented grasping in occluded scenes. This work, along with others, highlights a trend toward more integrated and dynamic action selection. Our work contributes to this line of research by proposing a novel object-centric representation that allows for more nuanced spatial reasoning and a more flexible approach to sequential manipulation. Unlike methods that primarily focus on clearing clutter, our approach reasons about the spatial relationships between objects to inform a sequence of actions, leading to more efficient and successful manipulation in highly cluttered scenes. II-B Object-centric Representation Learning A central challenge in robotic manipulation is learning representations that capture object properties and their relations. Early approaches such as VIMA [ 10 ] and VIOLA [ 33 ] employed imitation learning with language prompts and object proposals, but their effectiveness is constrained by the diversity and quality of demonstrations. More recent work emphasizes object-centric world models to improve generalization and exploration. For instance, [ 32 ] constructs 3D object-centric representations that enable manipulation policies robust to viewpoint changes and novel objects, while FOCUS [ 7 ] learns an object-centric world model to guide exploration and planning. Our approach departs from these by tailoring object-centric reasoning specifically to sequential manipulation in dense clutter. Instead of focusing primarily on generalization, we explicitly encode long-horizon spatial dependencies and occlusion relationships, yielding interpretable decision-making. This design not only facilitates trust and validation since the selection probabilities allow inspection of which object the model prioritized and why, but also enhances robustness in multi-step planning, such as clearing obstacles before grasping a target. II-C Vision-Language Models for Robotic Manipulation The integration of large language models (LLMs) and vision-language models (VLMs) [ 6 ] has opened new frontiers in robotics. Models like ThinkGrasp [ 17 ] , OVGNet [ 16