Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation

Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu · 2026

Read Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation by Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26768.txt.

Description

Su, Zhang, Ai, and Liu tackle a flaw in stored retrieval generation, where document modules trained on task goals encode both document facts and reusable habits. When modules merge at inference, overlapping task habits pile up with document updates, making the merged module less stable and less focused on wanted knowledge. Their separate subspace method trains a Task LoRA for reusable habits, then trains document modules in a separate subspace. Tests across knowledge tasks and model scales show the approach improves composition strength, especially when many document modules merge.

Catalog record

CREATORWeihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40244
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.26768

Browse the full library · About the Tower of Babel