Scalable Multi-Task Low-Rank Model Adaptation
Zichen Tian, Antoine Ledent, Qianru Sun · 2026
Read Scalable Multi-Task Low-Rank Model Adaptation by Zichen Tian, Antoine Ledent, Qianru Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-01526.txt.
Description
Adapting one large model to many tasks at once often works at small scale, then collapses as tasks pile up. Zichen Tian, Antoine Ledent, and Qianru Sun diagnose that collapse in this 2026 study of low-rank adaptation. They trace it to clashes between task updates and to blurred traits that routing needs.
The team finds that shared knowledge sits in strong components, while uniform rules force those parts apart. Working at block level, rather than on small pieces, cuts gradient clashes a lot with fewer settings. Their fix, mtLoRA, guards strong parts, adapts by blocks, and routes with fine per-dimension weights. Across vision and language suites with many tasks, it beat top rivals while using fewer settings and less training time.
Catalog record
| CREATOR | Zichen Tian, Antoine Ledent, Qianru Sun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62167 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.01526