EdgeVLA: Efficient Vision-Language-Action Models

Paweł Budzianowski, Wesley Maa, Matthew Freed, Jingxiang Mo, Winston Hsiao · 2025

Read EdgeVLA: Efficient Vision-Language-Action Models by Paweł Budzianowski, Wesley Maa, Matthew Freed, Jingxiang Mo, Winston Hsiao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-14049.txt.

Introduction

Paweł Budzianowski and co-authors present EdgeVLA in 2025, a leaner design for robot control models that see, read, and act. Big models of this kind can guide many robot tasks, but they run too slowly on small robots with tight power and memory. The team wanted real-time speed without losing skill.

Their first change drops the step-by-step guessing of the robot hand's target spot, which sped up responses about sevenfold. Their second change swaps the big language core for a small one, which trained just as well in early tests. The result matched a leading open model in training behavior while using far less memory. The team released the model and code for others to build on.

Catalog record

CREATORPaweł Budzianowski, Wesley Maa, Matthew Freed, Jingxiang Mo, Winston Hsiao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE12 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-14049

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EdgeVLA: Efficient Vision-Language-Action Models Thanks: Paweł Budzianowski, Wesley Maa, Matthew Freed, Winston Hsiao, Aaron Xie, Tomasz Młoduchowski, Viraj Tipnis, Benjamin Bolte are with K-Scale Labs. Email correspondence to [email protected] Thanks: Jingxiang Mo is with McGill University. Paweł Budzianowski, Wesley Maa, Matthew Freed, Jingxiang Mo, Winston Hsiao Abstract Vision-Language Models (VLMs) have emerged as a promising approach to address the data scarcity challenge in robotics, enabling the development of generalizable visuomotor control policies. While models like OpenVLA showcase the potential of this paradigm, deploying large-scale VLMs on resource-constrained mobile manipulation systems remains a significant hurdle. This paper introduces Edge VLA (EVLA), a novel approach designed to significantly enhance the inference speed of Vision-Language-Action (VLA) models. EVLA maintains the representational power of these models while enabling real-time performance on edge devices. We achieve this through two key innovations: 1) Eliminating the autoregressive requirement for end-effector position prediction, leading to a 7x speedup in inference, and 2) Leveraging the efficiency of Small Language Models (SLMs), demonstrating comparable training performance to larger models with significantly reduced computational demands. Our early results demonstrate that EVLA achieves comparable training characteristics to OpenVLA while offering substantial gains in inference speed and memory efficiency. We release our model checkpoints and training codebase to foster further research. I INTRODUCTION The development of robust and generalizable manipulation policies has long been hampered by the limited availability of large-scale, diverse embodied datasets. Recent advancements in Vision-Language Models (VLMs) [ 11 , 8 ] offer a compelling solution to this challenge. By leveraging the vast amount of readily available image-text data, VLMs can learn rich representations of the world and be adapted for visuomotor control tasks. Open-source models like OpenVLA [ 9 ] have demonstrated the effectiveness of this approach, showcasing impressive performance in various robotic manipulation tasks. However, deploying these large-scale VLMs, often exceeding billions of parameters, on resource-constrained mobile platforms with edge devices like the Jetson Nano presents significant challenges. Their high computational and memory requirements hinder real-time performance and limit accessibility for researchers and practitioners. The progress in the mobile manipulation can be effective only if the systems we design are inexpensive and easily deployable without putting too much strain on compute requirements. That is why, this paper introduces Edge VLA (EVLA), a novel VLA architecture designed to address above-mentioned challenges. EVLA offers significant improvements in inference speed and efficiency without compromising foundation models’ representational power. Our approach centers around two key innovations: First, our work focuses on architectural modifications to achieve significant speedups while maintaining model performance. Specifically by eliminating the autoregressive requirement for end-effector prediction and leveraging the efficiency of SLMs. We challenge the autoregressive approach for predicting end-effector positions, demonstrating that joint control, where the entire position is predicted simultaneously, does not diminish the model’s encoding capabilities. This modification yields a 7 -times increase in inference speed, crucial for real-time robotic control on edge devices. Secondly, we explore the potential of recently developed Small Large Language Models (SMLs), such as Qwen2 [ 17 ] , Phi [ 1 ] or Gemma [ 13 ] , which achieve comparable performance to their larger counterparts thanks to scaling laws with significantly reduced computational footprints. Our proposed architecture EVLA comprises of a pretrained language model Qwen2-0.5B fused with two visual encoders SigLIP [ 19 ] and DINOv2 [ 12 ] adding to 1B parameters. EVLA maintains training performance comparable to that of models 7 times larger while significantly reducing hardware requirements. Fig. 1 : The comparison of generation logic between OpenVLA and EVLA. The pretraining phase is identical for both models. In phase two, the EVLA LLM is being retrained to generate end-effector position in an autoregressive fashion. II RELATED WORK Learning-based approaches to mobile manipulation are beginning to reach or exceed the performance of classical model-based control systems [ 6 ] . We can broadly divide these approaches into systems trained from scratch and those fine-tuned on top of the foundation models. The former approach has relied on behavioral cloning where visual observations are typically mapped to either end effector position with orientation or joint positions [ 3 , 10 ] . These models can be enhanced through regularization, planning or multi-task learning pushing the limits of the performance. This approach enables easy deployment with relatively cheap hardware but does not leverage the power of foundation models [ 18 , 6 ] . These systems typically train models from scratch with model sizes from 10 to 100 M parameters which limits their ability to generalize to novel environments [ 20 , 3 ] . The line of work that relies on foundation models incorporates all the aforementioned techniques while aiming for more powerful generalization capabilities. The most extensively explored approach relies on vision-language models [ 11 , 8 ] . The vision component is typically adapted to operate in the same token space as the LLM, allowing for the reuse of different pretrained blocks. Combined with large manipulation datasets such as OpenX [ 4 ] , these models have demonstrated the promise of this paradigm by generalizing to new environments [ 2 , 9 ] . Although these works have highlighted the potential of leveraging large language models (LLMs), they come with substantial computational demands. Efforts to improve efficiency include quantization techniques [ 16 ] and hardware-specific kernels [ 14 ] . Nevertheless, these system achieve speed of only 5 to 10 Hz with stationary compute systems preventing their deployment on edge devices even in laboratory settings. Fig. 2 : The loss (left) and action token accuracy (right) training curves for both OpenVLA and EVLA models during training on the BridgeData V2 dataset. III METHOD III-A Phase 1: VLM Pretraining EVLA is based on a VLM trained using a combination of image-text pairs sourced from diverse captioning datasets and synthetically generated multimodal instruction-tuning examples [ 11 ] . The pre-training dataset comprises 1.2 M text-image pairs, facilitating the learning of robust visual and language representations following the recipe of the PrismaticVLM family of models [ 8 ] . For language processing, we utilize Qwen2 [ 17 ] with 0.5 B parameters as it demonstrates the effectiveness of SLMs in achieving comparable performance to that of larger models. We adopt a two-part visual encoder, employing pretrained SigLIP [ 19 ] and DinoV2 [ 12 ] models, following the architecture of OpenVLA [ 9 ] . A projection layer that maps the visual representation to the language model’s token space is learned jointly with the finetuned visual and language components. III-B Phase 2: Joint Control for End-Effector Prediction The second phase of training utilizes around 1 M of manipulation examples from the OpenX dataset [ 4 ] . Traditional VLAs employ an autoregressive approach to predicting end-effector positions, mimicking the causal nature of language generation. However, we hypothesize that for robotic control, this restriction is not inherently necessary. We propose that predicting the entire end-effector position jointly, rather than sequentially, does not compromise the model’s encoding capabilities while significantly improving inference speed. By removing the causal mask in the LLM and training the model to output the entire end-effector position at once, we eliminate autoregressive requirements, achieving a six-times speedup in inference - a critical improvement for real-time applications on

Read the complete text