When to Trust Imagination: Adaptive Action Execution for World Action Models

Rui Wang, Yue Zhang, Jiehong Lin, Kuncheng Luo, Jianan Wang, Zhongrui Wang, Xiaojuan Qi · 2026

Read When to Trust Imagination: Adaptive Action Execution for World Action Models by Rui Wang, Yue Zhang, Jiehong Lin, Kuncheng Luo, Jianan Wang, Zhongrui Wang, Xiaojuan Qi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06222.txt.

Introduction

Rui Wang and five co-authors teach world action models to check their own imagined futures before acting further. Such models predict both future views and actions, yet usually execute a fixed number of actions before replanning. That habit ignores whether the imagined future still matches reality, which can let a stale plan run on too long or waste replanning on a sound one.

Their verifier reasons jointly over predicted actions, predicted visual change, real observations, and instruction meaning. Execution continues while the plan verifies and replans when its validity fades. Training across mixed horizons improves long task coverage. On a simulation benchmark and on six real tasks with a humanoid robot, the system raised success while cutting planning calls sharply.

Catalog record

CREATORRui Wang, Yue Zhang, Jiehong Lin, Kuncheng Luo, Jianan Wang, Zhongrui Wang, Xiaojuan Qi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06222

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When to Trust Imagination: Adaptive Action Execution for World Action Models Rui Wang, Yue Zhang, Jiehong Lin, Kuncheng Luo, Jianan Wang, Zhongrui Wang, Xiaojuan Qi Abstract World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation by jointly predicting future visual observations and future actions. However, current WAMs typically execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the actual physical rollout. In this work, we formulate adaptive WAM execution as a future–reality verification problem: the robot should execute longer when the WAM-predicted future remains reliable, and replan earlier when reality deviates from imagination. To this end, we propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that jointly reasons over predicted future actions, predicted visual dynamics, real observations, and language instructions to estimate whether the remaining action rollout can still be trusted. FFDC enables adaptive action chunk sizes as an emergent consequence of prediction–observation consistency, preserving the efficiency of long-horizon execution while restoring responsiveness in contact-rich or difficult phases. We further introduce Mixture-of-Horizon Training to improve long-horizon trajectory coverage for adaptive execution. Experiments on the RoboTwin benchmark and in the real world demonstrate that our method achieves a strong robustness–efficiency trade-off: on RoboTwin, it reduces WAM forward passes by 69.10% and execution time by 34.02%, while improving success rate by 2.54% over the short-chunk baseline; in real-world experiments, it improves success rate by 35%. 1 Introduction Humans do not execute actions by blindly committing to a fixed future plan. Instead, we constantly predict how the world should evolve under our actions and compare this internal prediction with what we actually observe. When the predicted future remains consistent with reality, we can act smoothly over a long horizon; when the prediction deviates from observation, we immediately slow down, correct, or replan. A familiar example is missing a stair step: the body has already predicted the expected sensory feedback, and the sudden mismatch between expectation and reality creates an immediate warning signal. This prediction–observation comparison is central to robust physical interaction, especially when the world becomes uncertain, contact-rich, or difficult to model. Recent World Action Models (WAMs) provide a promising computational analogue of this mechanism. Unlike conventional vision-language-action policies [ 22 , 3 , 2 , 17 ] that mainly generate actions from the current observation and instruction, WAMs jointly predict future visual observations and future actions [ 19 , 14 , 12 , 5 , 30 , 7 ] . Through large-scale video-action pretraining, WAMs acquire spatiotemporal priors and physical dynamics knowledge, enabling stronger generalization to novel environments, unseen tasks, and new motion patterns. Recent studies [ 1 , 13 , 33 , 28 , 29 , 31 , 23 ] have demonstrated strong performance in zero-shot generalization, cross-environment transfer, and cross-embodiment learning. However, despite their ability to imagine how the world will evolve, current WAMs typically use their predicted future only to generate an action chunk, while the execution process itself remains largely blind to whether the imagined future is still consistent with the physical rollout. This reveals a fundamental limitation in current WAM execution. At each inference step, a WAM predicts a chunk of future actions [ 32 ] and the robot executes a fixed number of them before querying the model again. Such fixed-size execution ignores the fact that the reliability of WAM imagination varies across tasks and across phases within a task. For simple and predictable dynamics, such as approaching or grasping a rigid cup, the WAM prediction may remain accurate over a long horizon; in this case, repeatedly calling the WAM after only a few actions wastes computation. In contrast, for deformable, contact-rich, or stochastic interactions, such as folding cloth or manipulating objects with complex contact, the predicted future can quickly become unreliable; in this case, blindly executing a long action chunk can cause failure. Therefore, the key challenge is not merely choosing a better chunk size, but deciding when the WAM’s imagined future should still be trusted during physical execution . Existing adaptive execution methods for diffusion policies or VLA models mainly adjust action chunk length based on action uncertainty, entropy, or policy-side confidence [ 9 , 25 , 15 , 26 , 27 ] . However, these methods do not exploit the defining property of WAMs: the model predicts not only what action to take, but also what future visual observations should occur if the action rollout remains valid. This creates a new form of self-verification. During execution, the robot can compare the real observation with the WAM-predicted observation at the corresponding timestep and jointly reason over them with the action sequence to assess whether the remaining rollout is still compatible with reality. If the predicted visual dynamics, the real observation, and the planned actions remain causally consistent, the robot can continue executing the current chunk and avoid expensive WAM inference. Otherwise, the inconsistency becomes an early warning signal, and the robot should stop the current rollout and replan from the latest observation. Figure 1: FFDC enables adaptive trust in WAM imagination. (a) A WAM predicts future visual dynamics and an action chunk, and FFDC verifies during rollout whether the imagined future remains consistent with the real observation, planned actions, and instruction. Consistent predictions allow continued execution, while mismatches trigger replanning. (b) Fixed-size execution is inefficient in predictable phases and unreliable in difficult, contact-rich, or uncertain phases. (c) Scatter plot of success rate and task completion time on the RoboTwin benchmark. FFDC adapts execution length based on future–reality consistency, achieving a better robustness–efficiency trade-off. Based on this insight, we propose an adaptive WAM execution framework that explicitly compares WAM imagination with physical rollout. The core module is Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that estimates whether the remaining WAM-predicted action segment is still reliable. As shown in Fig. 1 (a), the WAM predicts future visual dynamics and an action chunk, while FFDC verifies during execution whether the imagined future remains consistent with the real observation, planned actions, and language instruction. FFDC uses a structured attention mechanism to model the interaction between predicted vision and action, allowing it to detect task-critical mismatches and decide whether the remaining rollout can still be trusted. To equip FFDC with the ability to distinguish reliable imagined futures from deviations that require replanning, we construct a binary verification dataset using valid segments from demonstrations and successful rollouts, together with failure-prone segments from failed rollouts and synthetic action corruptions, and train it to predict the executability of the remaining action segment. This design turns WAM execution from fixed open-loop rollout into adaptive future-aware control. In stable phases, FFDC allows the robot to trust the WAM’s long-horizon imagination and execute more actions per inference, substantially reducing computation. In difficult phases, FFDC detects when the imagined future becomes unreliable and triggers replanning, improving robustness. As a result, the effective action chunk size is no longer a manually fixed hyperparameter, but an emergent consequence of future–reality consistency. The robot executes long when the world is predictable and short when reality deviates. As shown in Fig. 1 (c), FFDC achieves the highest success rate while significantly reducing task completion time. Our contributions

Read the complete text