Efficient Few-Shot Continual Learning in Vision-Language Models

Aristeidis Panos, Rahaf Aljundi, Daniel Olmeda Reino, Richard E. Turner · 2025

Read Efficient Few-Shot Continual Learning in Vision-Language Models by Aristeidis Panos, Rahaf Aljundi, Daniel Olmeda Reino, Richard E. Turner (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04098.txt.

Description

Aristeidis Panos, Rahaf Aljundi, and their co-authors present a 2025 method for teaching vision language models new tasks over time without retraining the whole model. These models answer questions about images and write captions, but their fixed image encoders can misread images and drag results down. New data also arrive in small batches in real use.

Their method, called LoRSU, updates only a small, chosen set of parts inside the image encoder, using low rank changes placed where errors occur. Tests on visual question tasks with few examples at each step showed the method cut computing cost by more than twenty five fold compared with full updates, while keeping scores. It also left the model's broad skills largely intact.

Catalog record

CREATORAristeidis Panos, Rahaf Aljundi, Daniel Olmeda Reino, Richard E. Turner
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE134 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-04098

Browse the full library · About the Tower of Babel