Bridging Domains through Subspace-Aware Model Merging

Levy Chaves, Chao Zhou, Rebekka Burkholz, Eduardo Valle, Sandra Avila · 2026

Read Bridging Domains through Subspace-Aware Model Merging by Levy Chaves, Chao Zhou, Rebekka Burkholz, Eduardo Valle, Sandra Avila (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05768.txt.

Description

Chaves and team look into merging fine-tuned models from distinct domains. Merging joins task models. Recent research improved in-spread or multi-task merging, but spread stays little studied. Looking at setting competition via singular value decomposition, they show shifts induce stronger subspace conflicts. These conflicts exceed multi-task settings. To ease this, they propose SCORE, easing subspace conflicts. SCORE finds a shared right-angle basis from main parts. It projects task tables into the basis, pruning off-line entries. SCORE beats current methods in domain spread across designs.

Catalog record

CREATORLevy Chaves, Chao Zhou, Rebekka Burkholz, Eduardo Valle, Sandra Avila
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39915
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05768

Browse the full library · About the Tower of Babel