DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA
Yi Chen, Yuying Ge, Hui Zhou, Mingyu Ding, Yixiao Ge, Xihui Liu · 2026
Read DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA by Yi Chen, Yuying Ge, Hui Zhou, Mingyu Ding, Yixiao Ge, Xihui Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29844.txt.
Introduction
Chen and co-authors present DIAL, a robot control frame that splits intent from action. Most vision-language-action models map inputs straight to motor commands. That wastes the model's high-level judgment and can erode its semantics. DIAL lets a slow system predict latent visual futures as an intent bottleneck. A light policy then turns intent plus current sight into actions. Two-stage training keeps optimization stable. On RoboCasa tasks, DIAL tops prior work with tenfold fewer demonstrations.
Catalog record
| CREATOR | Yi Chen, Yuying Ge, Hui Zhou, Mingyu Ding, Yixiao Ge, Xihui Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49037 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.29844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA Yi Chen, Yuying Ge, Hui Zhou, Mingyu Ding, Yixiao Ge, Xihui Liu Abstract The development of Vision-Language-Action (VLA) models has been significantly accelerated by pre-trained Vision-Language Models (VLMs). However, most existing end-to-end VLAs treat the VLM primarily as a multimodal encoder, directly mapping vision-language features to low-level actions. This paradigm underutilizes the VLM’s potential role in high-level decision making and introduces training instability, frequently causing degradation of its rich semantic representations. To address these limitations, we introduce DIAL ( D ecoupling I ntent and A ction via L atent World Modeling), a framework that bridges high-level decision making and low-level motor execution through a differentiable latent intent bottleneck. Specifically, a VLM-based System-2 performs latent world modeling by synthesizing the latent visual foresight within the native feature space of the VLM’s vision encoder; this foresight explicitly encodes the VLM’s intent and serves as the structural bottleneck. A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions via latent inverse dynamics. To ensure optimization stability, we employ a two-stage training paradigm: a decoupled warmup phase in which System-2 learns to predict latent futures while System-1 learns motor control under ground-truth future guidance within a unified feature space, followed by seamless end-to-end joint optimization. This design enables action-aware gradients to refine the VLM backbone in a controlled manner while preserving its pre-trained knowledge. Extensive experiments on the RoboCasa GR1 Tabletop benchmark demonstrate that DIAL establishes a new state of the art, achieving superior performance with 10 × \times fewer demonstrations than prior methods. Furthermore, by leveraging heterogeneous human demonstrations, DIAL learns physically grounded manipulation priors and exhibits robust zero-shot generalization to unseen objects and novel configurations during real-world deployment on a humanoid robot. † † footnotetext: \dagger Corresponding authors. Figure 1: Overview of the DIAL Framework. DIAL bridges high-level decision making and low-level motor control through a differentiable latent intent bottleneck . (Left) System-2 (VLM) performs latent world modeling (LWM) to synthesize latent visual foresight within its native ViT feature space. This foresight serves as a structural bottleneck to convey the VLM’s intent, which System-1 (Policy) then decodes into actions via latent inverse dynamics. A decoupled-to-unified training paradigm ensures stability, leveraging initial alignment in a consistent latent space to facilitate subsequent end-to-end refinement via action-aware gradients. (Right) Powered by this structural grounding, DIAL scales across heterogeneous human-robot data, achieving SOTA performance with 10\times higher data efficiency and robust zero-shot generalization to unseen real-world configurations. 1 Introduction The development of generalist embodied agents has been significantly accelerated by pre-trained Vision-Language Models (VLMs) [ 4 , 10 , 3 , 2 ] . By internalizing massive semantic knowledge from the internet, VLMs provide a unified cognitive foundation capable of handling diverse multimodal tasks. Consequently, using these pre-trained models as cognitive backbones for robot policies has become a dominant trend [ 6 , 16 , 22 ] . However, effectively translating the abstract, high-level intent of a VLM into high-frequency, precise motor control remains a major challenge. As summarized in Figure 2 , existing approaches that leverage VLMs for robotic action generation face critical limitations. Hierarchical Planners [ 1 , 32 , 19 ] prompt the foundation model to generate high-level plans, typically text subtasks or code, to guide a separate low-level controller. While being interpretable and generalizable, this creates a non-differentiable wall that incurs high latency and prevents downstream action gradients from refining the VLM’s physical understanding. In contrast, End-to-End VLAs [ 6 , 28 , 20 ] directly predict continuous actions. In practice, however, these approaches often treat the VLM primarily as a large multimodal encoder that extracts vision-language features, rather than allowing it to serve as a high-level decision maker that explicitly represents task intent. As a result, training under low-level action supervision can become unstable, frequently causing the VLM’s semantic representations to collapse and overfit to spurious action patterns. Although auxiliary world modeling objectives [ 44 , 43 , 35 ] help by instilling physical dynamics and foresight, the absence of a strict structural bottleneck still permits the policy to rely on superficial correlations rather than truly translating the VLM’s intent into precise motor commands. This leaves the field in a structural dilemma: How can we design an end-to-end VLA that strictly grounds the policy in the VLM’s intent, seamlessly unifying cognitive generalization with execution precision? To resolve this dilemma, we present DIAL ( D ecoupling I ntent and A ction via L atent-world-modeling), as shown in Figure 1 . Inspired by the cognitive distinction between deliberate reasoning (System-2) and reflexive motor control (System-1), DIAL introduces latent visual foresight as a fully differentiable structural bottleneck between reasoning and execution. Rather than generating text or raw pixels, DIAL tasks the VLM (System-2) with predicting the future subgoal state entirely within the native feature space of its Vision Transformer (ViT) encoder, elevating it from a passive feature encoder to an active decision maker whose goal-directed foresight explicitly encodes the VLM’s latent intent and structurally governs downstream execution. A lightweight flow-matching policy (System-1) then operates as a latent inverse dynamics model: it compares the current visual observation against this predicted intent to deduce the precise high-frequency motor commands needed to reach the anticipated goal. This architectural decoupling yields three key advantages. First, it naturally supports a stable decoupled warmup : the VLM learns physical dynamics from diverse, action-free data while the policy independently masters sensorimotor control under ground-truth future guidance, preventing the gradient interference and representation collapse typical of naive joint training. Second, because the latent intent is continuous and provides a consistent interface between both systems, DIAL transitions smoothly into end-to-end synergy : action gradients flow back through the latent intent into the VLM, regularized by the same foresight reconstruction loss, encouraging the predicted intent to evolve into an actively "action-aware" representation without disrupting the VLM’s pretrained knowledge. Third, DIAL enforces strict structural grounding . Unlike prior works that loosely append future features as auxiliary context, our inverse-dynamics design imposes a hard bottleneck: System-1 must resolve the discrepancy between current and predicted latent states to generate actions, effectively mitigating the shortcut learning that commonly afflicts existing VLAs. We conduct extensive experiments to validate DIAL across both comprehensive simulations and real-world deployments. Latent visualizations further confirm that DIAL successfully grounds abstract linguistic instructions into a coherent, structurally aligned “visual roadmap” for the policy. In summary, our core contributions are: • Novel VLA Architecture: We propose DIAL, an end-to-end framework that structurally bridges the cognitive generalization of VLMs with the execution precision of low-level policies. By utilizing latent visual foresight as a differentiable bottleneck, we ensure the generated actions are strictly grounded in the VLM’s reasoning intent. • Decoupled-to-Unified Training Paradigm: To ensure stable optimization, we introduce a targeted dual-warmup strategy. Using action-free data, the VLM warmup shift its abstract semantic knowledge toward physical world dynamics. Simultaneously, the policy independently learns to map low-level perception and specific visual goals into precise motor actions. This distinct dual-initialization prevents representation collapse and paves the way for seamless end-to-end fine-tuning. • State-of-the-Art Performance & Scalability: DIAL achieves the highest reported performance on the RoboCasa GR1 Tabletop benchmark while using