ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval
Shahram Najam Syed, Yatharth Ahuja, Arthur Jakobsson, Jeff Ichnowski · 2025
Read ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval by Shahram Najam Syed, Yatharth Ahuja, Arthur Jakobsson, Jeff Ichnowski (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06202.txt.
Introduction
Shahram Najam Syed and co-authors help robot vision-language-action models specialize for one workplace instead of staying general. Broad models like OpenVLA handle many tasks passably but can stumble where one site needs steady high performance. Their method, ExpReS-VLA, adapts a pretrained model on the device itself, quickly and without forgetting earlier skills.
Adaptation stores compact embeddings instead of raw image and action pairs, cutting storage by ninety-seven percent. Similar past experiences are retrieved to enrich each training round, and a new loss learns from failed attempts as well as successes. Scores rose sharply on the LIBERO benchmark, and a real robot hit ninety-eight percent success after adapting in about half a minute.
Catalog record
| CREATOR | Shahram Najam Syed, Yatharth Ahuja, Arthur Jakobsson, Jeff Ichnowski |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-06202
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval Shahram Najam Syed, Yatharth Ahuja, Arthur Jakobsson, Jeff Ichnowski Abstract Vision-Language-Action (VLA) models like OpenVLA demonstrate impressive zero-shot generalization across robotic manipulation tasks but struggle to adapt to specific deployment environments where consistent high performance on a limited set of tasks is more valuable than broad generalization. We present EXPierence replayed, REtrieval augmented, Specialized VLA (ExpReS-VLA), a method that enables rapid on-device adaptation of pre-trained VLAs to target domains while preventing catastrophic forgetting through compressed experience replay and retrieval-augmented generation. Our approach maintains a memory-efficient buffer by storing extracted embeddings from OpenVLA’s frozen vision backbone, reducing storage requirements by 97% compared to raw image-action pairs. During deployment, ExpReS-VLA retrieves the k k most similar past experiences using cosine similarity to augment training batches, while a prioritized experience replay buffer preserves recently successful trajectories. To leverage failed attempts, we introduce Thresholded Hybrid Contrastive Loss (THCL), enabling the model to learn from both successful and unsuccessful demonstrations collected during deployment. Experiments on the LIBERO simulation benchmark show that ExpReS-VLA improves success rates from 82.6% to 93.1% on spatial reasoning tasks and from 61% to 72.3% on long-horizon tasks compared to base OpenVLA. Physical robot experiments across five manipulation tasks demonstrate that our approach achieves 98% success on both in-distribution and out-of-distribution tasks (with unseen backgrounds and objects), improving from 84.7% and 32% respectively for naive fine-tuning. ExpReS-VLA accomplishes this adaptation in 31 seconds using only 12 demonstrations on a single RTX 5090, making it practical for real-world deployment where robots must quickly specialize to their specific operating environment. I Introduction Every deployed robot faces a fundamental paradox: trained on diverse Internet-scale data and robot demonstrations, it must excel at just a handful of tasks in one specific environment. A deployed robot does not require the ability to manipulate all object categories from its 970,000-trajectory training dataset—it requires consistent, high-performance manipulation of the specific objects in its deployment environment. OpenVLA [ 13 ] , a 7B-parameter open-source VLA, exemplifies this tension: achieving 70% success across 29 manipulation tasks, yet struggling to reach the 95%+ reliability users demand for their specific objects and lighting conditions. This specialization challenge reveals the gap between how we train vision-language-action models—for broad generalization —and how we deploy them—for consistent specialization in constrained environments. This specialization challenge manifests as domain shift—subtle differences in lighting, object textures, or spatial layouts that degrade zero-shot performance from acceptable to unusable. While fine-tuning can adapt to specific environments, it can suffer from catastrophic forgetting [ 14 ] , where learning new tasks erases previously acquired skills. Existing solutions either require extensive computational resources (full model fine-tuning on GPUs clusters [ 13 ] ) or fail to leverage failed demonstrations that naturally occur during deployment. Moreover, current approaches treat adaptation as an offline process, incompatible with robots that must improve through daily interaction. We present EXPierence replayed, REtrieval augmented, Specialized VLA (ExpReS-VLA) , a method that makes catastrophic forgetting of previously run tasks structurally impossible through frozen encoders and persistent memory buffers, enabling rapid on-device adaptation of pre-trained VLAs. Our key insight is that successful domain adaptation can benefit three complementary mechanisms: (1) compressed memory to efficiently store experiences, (2) retrieval-augmented generation to leverage relevant past experiences, and (3) contrastive learning to explicitly avoid past failures . By combining these mechanisms, ExpReS-VLA transforms OpenVLA from a generalist that works adequately everywhere into a specialist that excels in its deployment environment. Fig. 1 : ExpReS-VLA takes in the RGB and prompt input, treats it with e coding, then passes that encoding to the buffer, used for retrivals and prioritizing the learning data for the policy. All of this runs on a single edge device, RTX 5090, and is optimized for performance on the Franka 7-DOF arm. ExpReS-VLA addresses three critical challenges in practical VLA deployment. First, we achieve 97% storage reduction for experience replay by storing vision encoder embeddings instead of raw images, enabling efficient memory management for continual learning. Second, we accelerate convergence through retrieval-augmented training that injects contextually similar past experiences into each batch. Third, we introduce Thresholded Hybrid Contrastive Loss (THCL), which adaptively switches between triplet [ 29 ] and InfoNCE [ 33 ] objectives based on failure complexity, transforming unsuccessful attempts into learning signals. We evaluate ExpReS-VLA in simulation and real-world experiments. On LIBERO simulation benchmarks, ExpReS-VLA achieves 92.4% success on spatial tasks and 72% on long-horizon tasks, which are improvements of 10% to 11% over base OpenVLA. Physical robot experiments show ExpReS-VLA improves in-distribution success from 84.7% to 98% and out-of-distribution success from 32% to 98%, with the larger OOD gain demonstrating robust adaptation to unseen variations. ExpReS-VLA completes adaptation in 31 seconds using 12 demonstrations on a single RTX 5090 GPU. This work makes the following contributions: • RAG-augmented robot learning : First integration of retrieval mechanisms into VLA fine-tuning, improving adaptation speed. • Compressed experience replay : A 97% memory reduction technique using frozen vision encoders that maintains semantic fidelity while enabling practical deployment. • THCL for failure exploitation : A novel piecewise loss that prevents repeated mistakes by dynamically selecting appropriate contrastive objectives. • Rigorous empirical evaluation : Systematic ablations across 40 simulation tasks (5 seeds) and 5 physical manipulation tasks (150 total trials) establishing clear component contributions. II Related Work Vision-Language-Action Models (VLAs) Generalist policies like OpenVLA [ 13 ] and RT-2 [ 2 ] , among others [ 17 , 24 , 10 , 34 , 31 , 11 , 23 ] , demonstrate broad capabilities but remain brittle in real-world deployments. They often fail with domain shifts across embodiment changes, and lack the deployed performance for specialized tasks [ 12 , 5 , 18 ] . These works also primarily focused on model development and this creates a trade-off between generalization and specialization, which our framework addresses. Our focus is to help these models adapt and fine-tune to specific cases where performance requirements demand robustness and accuracy, by effectively balancing prior data and new experiences. Fine-tuning and Catastrophic Forgetting Traditionally, domain adaptation relies on fine-tuning approaches that update the entire network on new data to generalize from a source domain to a target domain [ 3 , 2 ] . However, this conventional approach is impractical for real-time, on-device adaptation in dynamic environments, as it requires extensive training paradigm, significant GPU memory, and a large, stationary dataset. This process is further complicated by catastrophic forgetting [ 22 , 1 ] , a fundamental problem in continual learning where a model’s plasticity (acquiring new knowledge) comes at the cost of its stability (retaining old knowledge). Recent state-of-the-art methods have sought to overcome these limitations through more efficient techniques. Previous works have tried to overcome this using Elastic Weight Consolidation through regularizing weight changes [ 15 ] , which work well in static scenarios. Advancing with task-specific columns [ 26 ] helps in preserving previous weights but becomes computationally expensive and difficult. Another way to “pack” multiple tasks in a network is to iteratively prune it and free up parameters, but this approach is computationally expensive and provides difficulty in judging the importance of parameters freed-up [ 21 ] . Long et al. [ 20 ] combined