VITA: Vision-to-Action Flow Matching Policy
Dechen Gao 1 , Boqi Zhao 1 , Andrew Lee 1 , Ian Chuang 2 , Hanchu Zhou 3 , Hang Wang 3 , Zhe Zhao 1 , Junshan Zhang 3 , Iman Soltani 4 1 Department of Computer Science, University of California, Davis 2 Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 3 · 2025
Read VITA: Vision-to-Action Flow Matching Policy by Dechen Gao 1 , Boqi Zhao 1 , Andrew Lee 1 , Ian Chuang 2 , Hanchu Zhou 3 , Hang Wang 3 , Zhe Zhao 1 , Junshan Zhang 3 , Iman Soltani 4 1 Department of Computer Science, University of California, Davis 2 Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13231.txt.
Introduction
Dechen Gao and co-authors present VITA, a 2025 robot policy that turns sight straight into action. Standard robot policies start from random noise and clean it up step by step, checking the camera view at every step. That loop costs time and memory.
VITA skips the noise and flows directly from visual features to action features. An action encoder first maps raw moves into a space shaped like the visual one, and a decoding trick keeps that space from collapsing during training. On nine simulated and five real tasks, VITA ran one and a half to two times faster than standard policies while matching or beating their success.
Catalog record
| CREATOR | Dechen Gao 1 , Boqi Zhao 1 , Andrew Lee 1 , Ian Chuang 2 , Hanchu Zhou 3 , Hang Wang 3 , Zhe Zhao 1 , Junshan Zhang 3 , Iman Soltani 4 1 Department of Computer Science, University of California, Davis 2 Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 3 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-13231
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VITA: Vision-to-Action Flow Matching Policy Dechen Gao 1 , Boqi Zhao 1 , Andrew Lee 1 , Ian Chuang 2 , Hanchu Zhou 3 , Hang Wang 3 , Zhe Zhao 1 , Junshan Zhang 3 , Iman Soltani 4 1 Department of Computer Science, University of California, Davis 2 Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 3 Department of Electrical and Computer Engineering, University of California, Davis 4 Department of Mechanical and Aerospace Engineering, University of California, Davis [email protected], [email protected], [email protected], [email protected] [email protected], [email protected], [email protected], [email protected], [email protected] Abstract We present VITA , a VI sion- T o- A ction flow matching policy that evolves latent visual representations into latent actions via flow matching for visuomotor control. Traditional flow matching and diffusion policies face a fundamental inefficiency: they sample from standard source distributions (e.g., Gaussian noise) and then require additional conditioning mechanisms, such as cross-attention, to condition action generation on visual information, incurring time and space overheads. We propose VITA , a novel paradigm that treats latent images as the source of the flow, and learns an inherent mapping from vision to action. VITA eliminates the need for separate conditioning modules, while preserving the generative modeling capabilities. However, learning the flow between fundamentally different modalities, such as vision and action, can be intrinsically challenging. Action data is sparser and lacks semantic structures. Additionally, visual representations are typically much higher-dimensional than raw actions, while flow matching requires the source and target to have identical shapes. To resolve this, we propose to create a structured action latent space via an autoencoder as the target for flow matching, and up-sample raw actions to latent actions to match visual representation shapes. We also show it is crucial to supervise flow matching not only with encoder targets, but also with the final action outputs. We propose flow latent decoding, to backpropagate action reconstruction loss through sequential flow matching ODE (ordinary differential equations) solving steps, enabling effective end-to-end learning. We implement VITA as simple MLP layers, and evaluate VITA on challenging bi-manual manipulation tasks on the ALOHA platform, including 5 simulation and 2 real-world tasks. Despite its simplicity, MLP-only VITA outperforms or matches state-of-the-art generative policies, while reducing inference latency by 50% to 130% compared to conventional flow matching policies requiring different conditioning mechanisms or complex architectures. To our knowledge, VITA is the first MLP-only flow matching policy capable of solving complex bi-manual manipulation tasks like those in ALOHA benchmarks. Project page: VITA . K eywords Imitation Learning \cdot Flow Matching \cdot Robotics Manipulation \cdot Policy Learning 1 Introduction Figure 1: An overview of VITA —a noise-free, conditioning-free policy learning framework. Unlike conventional flow matching and diffusion policies that rely on sampling from standard distributions and require extra conditioning modules (e.g., cross-attention), VITA learns a continuous flow from latent visual representations to latent actions. By leveraging compact 1D visual and action latent spaces, and eliminating additional conditioning modules, VITA enables a highly efficient MLP-only architecture, achieving strong performance and inference efficiency across both simulation and real-world visuomotor tasks. Flow matching, a generalization of diffusion, has demonstrated remarkable success across a wide range of cross-modal generation tasks, from text-to-image generation stablediffusion ; dit ; sit ; crossflow ; flowtok ; conddiffusion , text-to-video generation video-diff ; li2023videogen ; jin2024pyramidal , to visuomotor (vision-to-action) policy chi2023diffusion ; ren2024diffusion ; gao2025ril ; su2025dense ; fmp-3d-consist ; fmp-multisupport ; fmp-riemannian ; black2410pi0 . Conventional flow matching and diffusion methods generate samples in the target modality by starting from a simple source distribution (often Gaussian) and progressively “denoising” them into the target modality distribution lipman2023flowmatchinggenerativemodeling ; sohl2015deep . To condition generation processes based on source modality inputs, conventional flow matching or diffusion necessitate additional conditioning mechanisms, e.g., cross-attention stablediffusion ; conddiffusion ; chi2023diffusion ; dit-policy , to inject source modality inputs at each denoising step, adding further complexity to both implementation and runtime overheads crossflow ; flowtok . In robot learning contexts specifically, minimizing policy inference latency and memory footprint is critical for effective real-time control. To overcome the computational inefficiencies inherent to conventional flow matching and diffusion policies, we introduce VITA ( VI sion- T o- A ction flow matching), a noise-free policy learning framework that directly connects the visual and action latent distributions. Unlike conventional approaches that rely on Gaussian noise priors and additional conditioning mechanisms, VITA leverages the flexibility of flow matching—which places no constraints on the source distribution—to treat latent image representations as the source distribution for the flow. These latent images inherently encode the visual context required for visuomotor tasks, thus eliminating the need for additional conditioning modules. VITA significantly reduces inference latency and memory overhead, simplifying the overall network architecture. Learning vision-to-action flow matching presents several unique challenges. Bridging two distinct modalities is inherently difficult crossflow ; flowtok , particularly in robot learning contexts where action data is often limited, unstructured, and sparse, whereas the visual representations feature rich structures and semantics, and have far more dimensions. Additionally, flow matching demands the source and target distributions to have identical shapes. To address these issues, we propose two crucial designs in VITA . First, we introduce a structured latent action space, learned via an action autoencoder, that allows action representations to match the dimensionality of visual representations and serves as a structured and tractable target distribution for flow matching. Specifically, the action encoder transforms raw actions into latent action targets for flow matching, while the decoder translates these latent actions back into raw actions. However, unlike latent diffusion models for image generation stablediffusion —which leverage abundant image data to pre-train latent image spaces and subsequently freeze them as fixed flow matching targets—a fixed, pre-trained latent action space yields poor performance for vision-to-action flow, due to the inherent sparsity and limited amount of action data available to train a reliable latent action representation. To overcome this limitation, our second key design enables end-to-end joint learning of both the flow matching and latent action space by backpropagating gradients through the ordinary differential equations (ODE) solving steps. This differentiates VITA fundamentally from existing cross-modal generation approaches, e.g., latent diffusion or flow matching models for image generation stablediffusion ; flowtok ; crossflow ; liu2023discovering , which fix the target distribution when training flow matching or diffusion models. We found that for vision-to-action flow matching learning, fixed targets yield poor performance, and enabling gradient updates through ODE solving for fully end-to-end training is crucial for achieving superior performance with VITA . We evaluate VITA on both real-world and simulated tasks using the ALOHA chuang2024active ; fu2024mobile ; zhao2024aloha platform. Our experiments demonstrate that VITA consistently outperforms or matches state-of-the-art generative policies. We instantiate VITA with only MLP layers for the flow matching network and action decoder, complemented by 1D latent representations. Compared to state-of-the-art flow matching and diffusion policies that predominantly leverage complex architecture such as transformers dit ; sit or U-Net unet ; chi2023diffusion , and introduce conditioning modules dit-policy ; chi2023diffusion ; zhao2023learning ; fmp-3d-consist , the MLP-only and noise-free VITA policy achieves, e.g., 50% to 130% reduction in inference latency compared to conventional flow matching. To our knowledge, VITA is the first MLP-only flow matching policy to succeed on tasks as challenging as ALOHA