Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters
Ruiqi Zhu, Endong Sun, Guanhe Huang, Oya Celiktutan · 2025
Read Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters by Ruiqi Zhu, Endong Sun, Guanhe Huang, Oya Celiktutan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18684.txt.
Introduction
Four authors led by Ruiqi Zhu give home robots plug-in adapters that learn from past tasks while they pick up new ones. A house bot comes with a skill set, yet each home asks for fresh tricks. Past adapter work taught each trick in its own box, which wastes kin gain. Their adapters train by a meta rule that steers new tricks with old task sense.
Sim homes and a real bot both host the proof runs of the frame. Gains beat the boxed adapter paths in each test world in their hands. The plug-ins stay light, so base skills stay safe from wash-out. Shared sense across tasks is the core gain they claim. Code and a project page ride with the work.
Catalog record
| CREATOR | Ruiqi Zhu, Endong Sun, Guanhe Huang, Oya Celiktutan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters Ruiqi Zhu, Endong Sun, Guanhe Huang, Oya Celiktutan Abstract Continual adaptation is essential for general autonomous agents. For example, a household robot pretrained with a repertoire of skills must still adapt to unseen tasks specific to each household. Motivated by this, building upon parameter-efficient fine-tuning in language models, prior works have explored lightweight adapters to adapt pretrained policies, which can preserve learned features from the pretraining phase and demonstrate good adaptation performances. However, these approaches treat task learning separately, limiting knowledge transfer between tasks. In this paper, we propose Online Meta-Learned adapters (OMLA). Instead of applying adapters directly, OMLA can facilitate knowledge transfer from previously learned tasks to current learning tasks through a novel meta-learning objective. Extensive experiments in both simulated and real-world environments demonstrate that OMLA can lead to better adaptation performances compared to the baseline methods. The project link: https://ricky-zhu.github.io/OMLA/ . I Introduction In recent years, a series of pretrained models have emerged in the domains of computer vision, natural language processing, and speech recognition [ 1 , 2 , 3 , 4 ] . Pretrained on internet-scale data, these models exhibit strong zero-shot generalization and adapt effectively to downstream tasks with minimal task-specific data, which can be costly to collect [ 5 , 6 ] . Similar practices have also been explored in robotic control applications [ 7 , 8 , 9 ] . However, decision-making tasks in robotics present unique challenges due to the diverse and unstructured nature of real-world environments. Therefore, while these pretrained robotic policy models have shown the capability to extrapolate the behaviours to new initial conditions such as object positions or lightning, they struggle to solve tasks involving novel objects or unseen task instructions [ 7 , 10 ] . To address this, robotic agents must retain the ability to adapt to new tasks that differ from earlier tasks. For example, pretrained household robots should be capable of continually adapting to new tasks that involve novel instructions and previously unseen objects as required in each household. However, the question of how to efficiently and effectively continually adapt pretrained robotic policies to new tasks remains an open challenge. Conventional continual adaptation methods usually struggle with the balance of the issue of catastrophic forgetting and the issue of conservative learning [ 11 , 12 ] . To mitigate these issues in continual adaptations, inspired by parameter-efficient fine-tuning in language models, prior works have leveraged lightweight adapters (e.g. LoRA [ 13 ] , RoboAdapter [ 14 ] ) to adapt pretrained models to new robotic tasks [ 15 , 10 ] . These adapters introduce a relatively small number of parameters compared to the whole model parameters while achieving effective adaptations on new tasks. Crucially, they mitigate catastrophic forgetting, as adapters are task-specific and operate in a plug-and-play fashion. Fig. 1: Our motivation: instead of learning adapters for each task separately, we develop OMLA for transfering the knowledge from previously learned tasks to facilitate new task adaptation. In continual adaptations, beyond preventing catastrophic forgetting, knowledge transfer from previously learned tasks to current learning tasks is also essential [ 16 , 17 , 18 ] . However, these prior works treat the adapter learning across tasks separately as shown in Fig. 1 , limiting the knowledge transfer. This raises the question: How can we enable the knowledge transfer when using adapters to adapt pretrained robotic policies? In this paper, we propose Online Meta-Learned Adapters (OMLA) for the continual adaptation of pretrained robotic policies. Unlike prior works that train adapters for new tasks from scratch, OMLA can leverage data from previous tasks to learn a prior over adapter parameters, facilitating knowledge transfer and improving adaptation to new tasks. We evaluate our method extensively in both simulated and real-world environments, focusing on vision-language robotic policies. In simulated experiments, OMLA consistently outperforms baseline methods across diverse task setups and policy structures, even with limited task-specific demonstrations. In real-world experiments, policies trained with OMLA exhibit robust performances to unseen scenes, demonstrating its effectiveness in continual adaptation. II Related Works Continual Learning. Continual learning in control is a long-studied yet challenging topic [ 19 , 20 ] . Its goal is to enable agents to acquire new tasks while maintaining the performance of previously learned ones. A key desideratum is leveraging prior knowledge or experience to accelerate new task learning [ 17 , 18 ] . Most existing continual learning methods in robotic control require updates to the entire model parameters [ 12 , 21 , 18 ] , making them prone to catastrophic forgetting or overly conservative updates that limit adaptability. In contrast, our method adopts task-specific lightweight adapters that maintain flexibility for new tasks while preventing forgetting previously learned tasks. In addition, rather than applying adapters in a straightforward manner [ 15 ] , we introduce a meta-learning objective that promotes effective knowledge transfer between tasks. Meta-learning in Robotics. Meta-learning aims to discover useful priors from a collection of datasets to facilitate fast adaptation to new tasks [ 22 ] . In robotics, meta-learning has been explored in contexts such as one-shot imitation learning [ 23 ] and adaptation to domain shifts [ 24 , 25 ] . These approaches typically apply meta-learning objectives to entire robotic trajectories, which incurs high GPU memory costs—especially when dealing with high-dimensional observations (e.g., images) or large model architectures. We address this by proposing a more efficient strategy that applies meta-learning objectives selectively to informative data points by comparing the similarities of the data points, significantly reducing memory overhead, detailed in Section IV-D . Adapters for pretrained models. Adapters are a form of Parameter-Efficient Fine-Tuning (PEFT) originally developed for adapting large language models (LLMs) [ 13 , 26 , 27 ] . These methods reduce computational overhead by fine-tuning a small set of additional parameters instead of modifying the entire model. Since adapters are task-specific, they naturally align with a key principle of continual learning: new task acquisition should not degrade performance on previously learned tasks. While prior works primarily employ adapters to mitigate catastrophic forgetting [ 15 ] , we extend their application by investigating how they can actively facilitate knowledge transfer across tasks, thereby enhancing continual learning performance. III Background Transformer. Originally introduced for natural language processing by Vaswani et al. [ 28 ] , the Transformer architecture has since become the foundation of large models across various domains [ 1 , 29 , 30 ] . The attention mechanism enables transformers to capture dependencies between tokens by assigning different weights across an input sequence, making them well-suited for processing long-context data. Transformer is further extended to tackle image data [ 31 ] . The Transformer architecture has also been extended to image data [ 31 ] . In this formulation, images are partitioned into patches, flattened, and encoded as tokens via linear projections. This enables the model to capture spatial relationships in images through attention maps. Moreover, Transformers can model temporal dependencies, making them well-suited for robotic policy learning, where they have achieved impressive performance [ 32 , 33 ] . Meta-learning. Meta-learning aims to bootstrap from a set of tasks to learn faster on a new task [