Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy
Wooseong JeongKAIST [email protected], Wonyoung LeeKAIST [email protected] Kuk-Jin YoonKAIST [email protected] · 2026
Read Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy by Wooseong JeongKAIST [email protected], Wonyoung LeeKAIST [email protected] Kuk-Jin YoonKAIST [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26299.txt.
Description
Jeong, Lee, and Yoon propose TARA-Merging, a way to merge several LoRA adapters without losing what each one learned. Naive merging can mute key update directions and stress minor ones. Their method views the problem through subspace coverage and directional imbalance. It sets merge weights with a preference-weighted loss while keeping task subspaces intact. Across eight vision tests and six language tests, TARA beats plain and LoRA-aware baselines and holds up well on new tasks.
Catalog record
| CREATOR | Wooseong JeongKAIST [email protected], Wonyoung LeeKAIST [email protected] Kuk-Jin YoonKAIST [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84880 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26299