Collage : Adaptive Fusion-based Retrieval for Augmented Policy Learning

Sateesh Kumar, Shivin Dass, Georgios Pavlakos, Roberto Martín-Martín · 2025

Read Collage : Adaptive Fusion-based Retrieval for Augmented Policy Learning by Sateesh Kumar, Shivin Dass, Georgios Pavlakos, Roberto Martín-Martín (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01131.txt.

Introduction

Kumar and three colleagues present COLLAGE in twenty twenty-five, a way to pick training clips for robots that learn from few examples. A robot may get just a handful of demos for a new task, plus a vast store of past clips. Picking by one trait alone, such as looks, can pull in clips from the wrong task. Their method blends picks from many traits.

Each trait based subset gets a weight based on how well a policy trained on it predicts the target demos. Training then draws more from high weight subsets. Tests beat past methods in simulation across ten tasks. Gains were larger in real world tests drawn from a big public robot data set.

Catalog record

CREATORSateesh Kumar, Shivin Dass, Georgios Pavlakos, Roberto Martín-Martín
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01131

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Collage : Adaptive Fusion-based Retrieval for Augmented Policy Learning Sateesh Kumar, Shivin Dass, Georgios Pavlakos, Roberto Martín-Martín Abstract In this work, we study the problem of data retrieval for few-shot imitation learning: select data from a large dataset to train a performant policy for a specific task, given only a few target demonstrations. Prior methods retrieve data using a single-feature distance heuristic, assuming that the best demonstrations are those that most closely resemble the target examples in visual, semantic, or motion space. However, this approach captures only a subset of the relevant information and is prone to introducing detrimental demonstrations, e.g., retrieving data from unrelated tasks due to similar scene layouts, or selecting similar motions from tasks with divergent goals. We present Collage , a method for coll ective data ag gr e gation in few-shot imitation learning that uses an adaptive late fusion mechanism to guide the selection of relevant demonstrations based on a task-specific combination of multiple cues. Collage follows a simple, but flexible and efficient data aggregation recipe: it assigns weights to subsets of the dataset that are pre-selected using a single feature (e.g., appearance, shape, or language similarity), based on their task relevance, measured by how well a policy trained on each subset predicts actions in the few target demonstrations. These weights are then used during policy training to perform importance sampling over the aggregated dataset, sampling data more densely or sparsely, according to their estimated relevance. This weighted aggregation strategy is general and feature-agnostic, allowing Collage to combine and leverage any number of subsets selected by any retrieval heuristic or method, and to identify which subset provides the most benefit for the target task. In extensive experiments, Collage outperforms state-of-the-art retrieval and multi-task learning approaches, achieving a 5.1 % 5.1\% improvement over the best baseline in simulation across 10 tasks, and a 16.6 % 16.6\% improvement in the real world across 6 tasks. For our real world experiments, we include data selection from the large-scale, real-world DROID dataset, significantly improving few-shot imitation policy training. More information at our website . Keywords: Imitation Learning, Few-Shot Learning, Data Retrieval 1 Introduction Figure 1: Different tasks benefit from retrieval based on different modalities. For the target task “Open the book” (left), retrieval based on visual similarity tends to return relevant demonstrations where a robot opens a book, whereas retrieval based on motion similarity often returns demonstrations with similar motions but different semantics, such as opening a cloth. On the other hand, for the target task “Stir the bowl” (right), retrieval based on motion similarity is more effective, returning demonstrations that involve stirring motions, while visual similarity tends to retrieve examples that feature a bowl but involve different actions, such as placing an object inside the bowl. Imitation Learning (IL) [ 1 , 2 , 3 ] has emerged as a powerful framework for training robot policies by mimicking expert demonstrations. With the rise of transformer- and diffusion-based models, single task IL approaches have achieved impressive performance on complex robotic manipulation domains [ 4 , 5 , 6 ] . However, these successes often depend on the availability of extensive demonstration data on the specific task of interest. In real-world environments, such as homes, where object configurations and task requirements frequently change, collecting sufficient demonstrations for every possible task and variation is costly, time-consuming, and ultimately impractical. As a solution to overcome the need for task-specific demonstrations, the community has explored training generalist multi-task policies [ 7 , 8 , 9 ] on large-scale datasets [ 10 , 11 ] , which cover hundreds of tasks and environments and even different robot morphologies. The hope is that policies trained on a diverse array of experiences will generalize to unseen tasks with minimal adaptation. However, in practice, generalist policies often underperform on specific tasks when compared to expert policies trained on a sufficiently large number of demonstrations. This seems to be largely caused by a negative transfer effect [ 12 ] , where irrelevant or conflicting demonstrations degrade the policy’s ability to focus on task-relevant behavior. To address this issue, a recently explored alternative for task-specific imitation policy learning leverages the same existing large demonstration datasets in a novel way: to retrieve data and augment a small dataset with a few target-task-specific demonstrations [ 13 , 14 , 15 , 16 ] . Prior approaches in retrieval-augmented few-shot imitation learning rely on different types of similarity to retrieve relevant trajectories, including visual features [ 13 , 15 ] , optical flow [ 14 ] , or language embeddings [ 17 ] , i.e., they assume that the most helpful trajectories to train for a task are those that look, move, or are called similarly in the large dataset. However, while each of these assumptions is true in a broad, statistical sense, they can easily fail for specific cases (see Fig. 1 ), rendering methods relying on only one of these single-modality heuristics for retrieval highly variable and brittle. Motivated by the limitations of single-modality retrieval, we ask: Can we design a strategy to automatically combine data retrieved using different similarity measures for higher performance in retrieval-augmented few-shot policy learning? We present Collage , a method that demonstrates that the above is indeed possible by coll ectively ag gr e gating subsets of the dataset preselected using single-modality heuristics in a synergistic manner. For aggregation, Collage proposes associating weights with each single-modality subset based on an estimate of their task-training relevance. To compute this estimation, Collage employs a rollout-free mechanism: it trains reference policies on each retrieval subset and estimates the policy task-relevance by evaluating the log-likelihood of the few target demonstrations. We can consider conceptually these weights to be an approximation of the probability of a subset to be helpful to train a performant policy for the given task, and use them to guide an importance sampling mechanism during policy training: more relevant subsets of data are sampled more densely compared to less relevant subsets during the learning process. As a result, Collage is agnostic to the type of retrieval features, enabling flexible integration of different similarity modalities. We evaluate Collage in both simulated and real-world settings. In simulation, we use the LIBERO benchmark [ 18 ] , and demonstrate that our approach significantly outperforms both single-feature retrieval and generalist policy learning baselines by 5.1\% and 14.8\% respectively. For our real-world evaluation, we retrieve from the DROID dataset [ 10 ] , and demonstrate that our method improves few-shot imitation learning performance by 16.6\% over existing state-of-the-art, achieving robust retrieval directly from large, diverse offline datasets without requiring manual curation, even in the presence of substantial visual and task domain shifts. 2 Related Work Collage is a novel methodology for collective data aggregation in few-shot imitation learning settings. We begin by reviewing prior work in this area, followed by related work in zero-shot manipulation and multi-modal representation learning. Data retrieval for few-shot imitation learning methods assume access to a small number of target demonstrations that are augmented with data from a large dataset to improve policy training performance [ 16 , 13 , 14 , 15 , 17 , 19 , 20 , 21 , 22 ] . These methods

Read the complete text