RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

Ulas Berk Karli, Tesca Fitzgerald · 2026

Read RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models by Ulas Berk Karli, Tesca Fitzgerald (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-23617.txt.

Introduction

Karli and Fitzgerald study how robot policies can keep learning after deployment by asking for help only where they are unsure, in a 2026 paper on robot learning. Vision language action models are usually fine tuned passively: experts record full demonstrations for tasks the robot already fails. That wastes effort on parts the robot does well, gives no guidance on which states need supervision, and starts only after failures.

Their approach, RECALL, collects recovery demonstrations starting from states where the model's uncertainty is high. Experiments confirm that this active collection fine tunes more efficiently than passive demonstrations. Training only on the recovery data, though, triggers catastrophic forgetting, the loss of previously learned skills. The authors test countermeasures such as replay, mixing old data back in, and elastic weight consolidation, which protects important weights. Each trades new learning against retention differently.

Catalog record

CREATORUlas Berk Karli, Tesca Fitzgerald
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.23617

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models Ulas Berk Karli, Tesca Fitzgerald Abstract Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies. Keywords: VLA Models; Active Learning; Continual Learning 1 Introduction Recent Vision-Language-Action (VLA) models such as RT-1 [ 1 ] , RT-2 [ 2 ] , Gemini Robotics [ 3 ] , \pi_{0} -FAST [ 4 ] , and \pi_{0.5} [ 5 ] have shown that large-scale robot learning can produce broadly capable manipulation policies. However, despite their generality, VLAs still require fine-tuning when deployed to new robots, environments, or task distributions. In practice, this adaptation is usually performed through passive imitation learning, where an expert (i) collects additional demonstrations starting from task initial states, (ii) fine-tunes the model, (iii) evaluates performance, and (iv) repeats if the policy remains unreliable. This process is inefficient for several reasons. First, it only reveals that the model needs more data after failures occurred, which can be time-consuming and unsafe. Second, it gives little guidance about how much additional data is needed, which can lead to under- or over-collection of data. Third, passive recollection does not identify where supervision would be most useful. Demonstrators may therefore spend most of their efforts on states that are already well represented in the original training data; yet, failures may arise from a small set of difficult substeps or recovery from distribution-shifted intermediate states. We propose an active, continual learning paradigm for fine-tuning VLAs. Our central hypothesis is that collecting demonstrations from high-uncertainty states will lead to more informative (and thus, efficient) data collection compared to typical passive data collection. We use a state-of-the-art technique for uncertainty quantification [ 6 ] and utilize it to collect data that more directly addresses the failure modes limiting downstream task success. We then evaluate techniques for continual learning, with the aim of fine-tuning a VLA with uncertainty-guided data without catastrophic forgetting of previously-learned tasks. Our contributions are as follows: 1. An active data collection framework for autoregressive VLAs that uses INSIGHT [ 6 ] to select uncertain states to prompt new demonstrations and provide the way to best utilize that data. 2. An evaluation of active vs passive data collection, showing that demonstrations provided from high-uncertainty states lead to better model performance than unguided demonstrations. 3. A comparison of online- and offline-style data collection, showing that collecting data from the first uncertain state in a rollout is just as effective (and thus, more efficient) compared to collecting data from all uncertain states. 4. Application and analysis of continual-learning techniques (regularization- and replay-based) for VLA models, showing that replay-based data mixing is currently the most reliable strategy. 5. Empirical design lessons and open research questions for continual, active learning in VLAs. 2 Related Works Active Learning for Robot Policies. Active learning involves identifying states, trajectories, tasks, or environments where additional demonstrations would be most informative. Interactive imitation learning methods such as DAgger [ 7 ] and HG-DAgger [ 8 ] reduce covariate shift by collecting supervision on states induced by the learned policy. Uncertainty-guided active learning further targets this process by querying the expert only when the policy is uncertain, for example using Monte Carlo dropout to select states for supervision [ 9 ] . However, most uncertainty-aware aggregation methods [ 10 , 11 , 12 , 13 ] have been studied for smaller, task-specific policies, often in driving [ 8 , 12 ] or classical imitation-learning settings [ 7 ] , rather than for large pretrained VLAs. Uncertainty Quantification in VLAs. Prior work on uncertainty quantification (UQ) has explored ensemble methods [ 14 ] , Bayesian approximations [ 15 ] , conformal prediction [ 16 ] , and learned intervention or failure predictors. For example, KnowNo [ 17 ] uses conformal prediction to decide when a VLM planner should ask for help, but operates over high-level plan rather than low-level VLA actions. INSIGHT [ 6 ] is an inference-time introspection method for UQ in autoregressive VLAs that uses token-level probability features to predict when the robot should ask for help. INSIGHT learns a model from token-level uncertainty features extracted during action decoding, including entropy, log-probability, and Dirichlet-based aleatoric and epistemic uncertainty estimates. Continual Learning. Continual learning studies how models acquire new knowledge without catastrophic forgetting , where training on new data degrades performance on earlier tasks or distributions [ 18 ] . This issue has been studied in LLMs [ 19 ] , suggesting that large pretrained sequence models such as VLAs may be similarly vulnerable when fine-tuning on new data. Common solutions include replay-based methods, which mix new data with examples from prior datasets [ 20 , 21 ] , and regularization-based methods, which constrain changes to parameters important for prior behavior. Elastic weight consolidation (EWC) is a representative regularization method that uses a Fisher information matrix to penalize changes to parameters important for previous tasks [ 22 ] . Figure 1: Overview of uncertainty-guided active continual learning. An initial \pi_{0} -FAST policy is rolled out on LIBERO-10 while INSIGHT identifies high-uncertainty states. We compare passive collection with online recovery from the first high-uncertainty state and offline recovery from all high-uncertainty states, then integrate the resulting data using replay, or EWC regularization. 3 Enabling Active, Continual Learning from Uncertainty-Guided Data Active learning in VLAs remains unsolved and under-explored. In this work, we propose an active, continual learning paradigm for fine-tuning VLAs. We focus on autoregressive VLAs, particularly \pi_{0} -FAST [ 4 ] , which tokenizes continuous robot actions and predicts them via a next-token objective. Toward enabling active learning in VLAs, we leverage INSIGHT for uncertainty quantification [ 6 ] and utilize it by interpreting its “help predictions” as a proxy for high-uncertainty states from which to collect new demonstrations. We study whether uncertainty-guided data collection enables effective active learning, leading to our first research question RQ-1: Compared to passive data collection, do demonstrations originating from high-uncertainty states result in better policy performance? We next consider the effect of online vs offline active learning in VLA performance. In online learning, the robot requests a demonstration in real-time, which originates at the first uncertain state within a rollout. In contrast, an offline learning setting involves the robot performing a full rollout, recording

Read the complete text