Shared LoRA Subspaces for almost Strict Continual Learning

Prakhar Kaushik, Ankit Vaidya, Shravan Chaudhari, Rama Chellappa, Alan Yuille · 2026

Read Shared LoRA Subspaces for almost Strict Continual Learning by Prakhar Kaushik, Ankit Vaidya, Shravan Chaudhari, Rama Chellappa, Alan Yuille (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06043.txt.

Description

Teaching a big pretrained model new tasks one after another often erases what it learned before. Prakhar Kaushik and 4 co-authors propose Share, a method that keeps all tasks inside one shared low-rank space. LoRA, a popular cheap tuning method, normally needs a fresh adapter for each task.

Share builds one common space from early tasks, then folds each new task into it. Only the most useful directions are kept, which limits clashes between old and new skills. The team reports far fewer added weights and far less memory than standard LoRA setups. Tests span images, language, 3D poses, and picture making.

Catalog record

CREATORPrakhar Kaushik, Ankit Vaidya, Shravan Chaudhari, Rama Chellappa, Alan Yuille
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06043

Browse the full library · About the Tower of Babel