Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation
Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu · 2026
Read Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation by Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26768.txt.
Description
Su, Zhang, Ai, and Liu tackle a flaw in stored retrieval generation, where document modules trained on task goals encode both document facts and reusable habits. When modules merge at inference, overlapping task habits pile up with document updates, making the merged module less stable and less focused on wanted knowledge. Their separate subspace method trains a Task LoRA for reusable habits, then trains document modules in a separate subspace. Tests across knowledge tasks and model scales show the approach improves composition strength, especially when many document modules merge.
Catalog record
| CREATOR | Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40244 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26768