OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation

Yushan Liu 1 Peibo Sun 2 Shoujie Li 3 Yifan Xie 1 Lingfeng Zhang 1 Xintao Chao 1 Shiyuan Dong 2 Fang Chen 2 Xiao-Ping Zhang 1 Wenbo Ding 1 1 Tsinghua University 2 Shanghai Jiao Tong University 3 Nanyang Technological University · 2026

Read OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation by Yushan Liu 1 Peibo Sun 2 Shoujie Li 3 Yifan Xie 1 Lingfeng Zhang 1 Xintao Chao 1 Shiyuan Dong 2 Fang Chen 2 Xiao-Ping Zhang 1 Wenbo Ding 1 1 Tsinghua University 2 Shanghai Jiao Tong University 3 Nanyang Technological University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06481.txt.

Introduction

Yushan Liu and nine co-authors propose OA-WAM, a world action model that represents a robot's scene as addressable objects rather than one whole image. World action models predict how a scene will change along with the robot's actions. Holistic images make it hard for an action decoder to find the specific object an instruction names, especially after scene shifts.

Each frame splits into one robot slot and several object slots, each with a fixed address vector and a changing content vector. Attention routes through addresses only, keeping identity separate from appearance. The model matches strong baselines on LIBERO and SimplerEnv and stays robust under geometric changes and object swaps.

Catalog record

CREATORYushan Liu 1 Peibo Sun 2 Shoujie Li 3 Yifan Xie 1 Lingfeng Zhang 1 Xintao Chao 1 Shiyuan Dong 2 Fang Chen 2 Xiao-Ping Zhang 1 Wenbo Ding 1 1 Tsinghua University 2 Shanghai Jiao Tong University 3 Nanyang Technological University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06481

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation Yushan Liu 1 Peibo Sun 2 Shoujie Li 3 Yifan Xie 1 Lingfeng Zhang 1 Xintao Chao 1 Shiyuan Dong 2 Fang Chen 2 Xiao-Ping Zhang 1 Wenbo Ding 1 1 Tsinghua University 2 Shanghai Jiao Tong University 3 Nanyang Technological University Abstract World Action Models (WAM) enhance Vision-Language-Action (VLA) policies by jointly predicting scene evolution and robot actions, but most existing methods represent the predicted world as holistic images, video-token streams, or global latents—making it hard for the action decoder to locate a specific target object when an instruction refers to it: under novel scene shifts the relevant object often remains visible while holistic tokens entangle its identity with surrounding context. We formulate this as a lack of object addressability and propose OA-WAM, an Object-Addressable World Action Model for robust robot manipulation. Each frame is decomposed into N + 1 N{+}1 per-slot state tokens ( 1 1 robot slot, N N object slots), each concatenating an identity vector addr k \mathrm{addr}_{k} and a time-varying content vector content k t \mathrm{content}_{k}^{t} , and fused with text, image, proprioception, and past-action tokens into a block-causal sequence. A world head regresses next-frame slot state and a flow-matching action head decodes a 16 16 -step continuous action chunk, both in a single forward pass. Object addressability is enforced architecturally by routing cross-slot attention on addr \mathrm{addr} alone at every transformer layer—an addr-only key projection together with a per-layer reset of the addr slice in the residual stream. This separates "which object to act on" from "what that object currently is" at the tensor level, providing an intervenable object-level interface without introducing extra tokens. OA-WAM matches the SOTA VLA and WAM baselines on LIBERO ( 97.8 % 97.8\% ) and SimplerEnv ( 79.3 % 79.3\% ); on LIBERO-Plus, OA-WAM sets a new SOTA on the geometric axes most aligned with the hypothesis, while remaining competitive with π 0.5 \pi_{0.5} on the seven-axis aggregate ( 83.9 % 83.9\% vs 85.7 % 85.7\% ). A causal slot-intervention test gives OA-WAM a swap-binding cosine of 0.87 0.87 versus ≤ 0.09 \leq 0.09 for all holistic baselines. Our results show that explicitly organizing a world model as addressable object states is an effective path toward more robust robot manipulation under scene perturbations. Figure 1: Overview of OA-WAM. Under scene perturbations (left, six typical axes), holistic WAMs entangle target identity with context in global tokens and drift to wrong actions (top right). Our OA-WAM (bottom right) decomposes each frame into N{+}1 addressable object slots whose cross-slot attention key reads only the identity address subvector, keeping robust manipulation. 1 Introduction Robot manipulation naturally unfolds at the object level. An instruction like “put the red mug on the green tray” is not a request to replay a training trajectory; it requires the robot to identify the language-named objects in the current scene, reason about their spatial and functional relations to surrounding distractors, and generate correct, robust closed-loop actions. Vision-Language-Action (VLA) policies have made rapid progress on standard manipulation benchmarks by scaling vision-language backbones, robot data, and action decoders [ 102 , 34 , 60 ] ; World Action Models (WAMs) build on this by jointly predicting actions and future world states, supplying extra temporal and causal supervision [ 9 , 90 , 89 ] . However, robustness benchmarks show that high standard-benchmark scores do not imply reliable scene understanding: under modest perturbations of object layout, camera viewpoint, robot initial state, background, lighting, or sensor noise, policies often collapse from near-saturated success rates [ 19 , 61 , 23 ] , and remain insensitive to paraphrased or even meaningless instruction tokens [ 19 , 99 ] —both signs that target selection is bound to training layouts and visual context rather than to the language-named objects. Most existing WAMs still represent the predicted world as full-frame observations, image/video token streams, or shared global vision-action latents [ 101 , 10 , 95 ] ; such representations capture how scenes evolve but provide the action decoder no stable interface for deciding which object to act on. When camera, layout, or robot pose changes, the target object often remains visible yet becomes entangled with background and neighboring content inside holistic tokens, causing action selection to drift. Object-centric work attempts to fill this gap: slot-based dynamics models improve sample efficiency and relational reasoning [ 20 , 29 ] , and object-state grounding turns object representations into operational interfaces between language and action [ 13 , 26 , 88 ] . Yet many such methods rely on Slot-Attention-style discovery from scratch [ 50 , 81 , 82 ] , which is sensitive to fixed slot counts, occlusion, and cluttered scenes; and even with stable slots, each slot still mixes identity, appearance, pose, and local context, so the action decoder may still drift to the wrong instance under perturbation. Based on this diagnosis, we propose OA-WAM, an Object-Addressable WAM. Each frame is decomposed into N{+}1 slots whose vectors are split into a frozen identity address \mathrm{addr}_{k} —computed once per episode from the language label and the initial DINOv3 feature, and never updated thereafter—and a time-varying content \mathrm{cnt}_{k}^{t} refreshed every step from SAM 3 + DINOv3 masks. Slots are fused with text, image-VQ, proprioception, and past-action streams into a block-causal sequence processed by a 7B Chameleon-style trunk, from which a world head regresses next-frame per-slot (\hat{\mathrm{cnt}},\hat{\mathrm{pose}}) and a flow-matching action head decodes a 16-step continuous action chunk in a single forward pass. The OA constraint is enforced as two parameter-free tensor-level operations applied at each of the 32 transformer layers: the cross-slot key projection reads only the \mathrm{addr} subvector, and a forward hook resets the residual \mathrm{addr} slice back to the cached identity. Because \mathrm{addr} is frozen per episode and gradients into it are blocked by the reset, slot-routing keys at every transformer layer are architecturally constrained to depend only on the frozen identity address. Conditional on correct upstream slot extraction , this removes a specific failure mode of holistic backbones—scene-context shifts can no longer rewrite which slot the action head reads from—while content, pose, and language remain free to flow through the residual stream and value projections; the constraint is on key routing rather than an end-to-end semantic guarantee, and inherits any errors made by the SAM 3 + Qwen3-VL slot pipeline. Across three benchmarks, OA-WAM validates this design where the hypothesis predicts. On in-distribution LIBERO and SimplerEnv WidowX (Visual Matching) it reaches 97.8 and 79.3 avg, leading all published VLA and WAM baselines (Table 1 ). On the LIBERO-Plus robustness benchmark (Table 2 ) it sets a new SOTA on the geometric axes most aligned with the hypothesis—camera 80.5 ( +4.7 % over Cosmos-Policy), robot-initial-state 89.6 (within 0.1 % of X-VLA), and a Camera/Robot/Layout Geo Avg of 84.3 ( +4.8 % above \pi_{0.5} )—and is competitive on the seven-axis overall aggregate ( 83.9 vs \pi_{0.5} ’s 85.7 ), with the residual gap concentrated on Sensor Noise where photometric corruption disrupts slot extraction at the perception stage rather than the action policy itself. A causal slot-intervention test gives OA-WAM a swap-binding cosine of 0.87 versus \leq 0.09 for all eight holistic baselines—direct behavioural evidence that target selection is grounded in

Read the complete text