Resolving Interference ( RI ): Disentangling Models for Improved Model Merging

Pratik Ramesh, George Stoica, Arun Iyer, Leshem Choshen, Judy Hoffman · 2026

Read Resolving Interference ( RI ): Disentangling Models for Improved Model Merging by Pratik Ramesh, George Stoica, Arun Iyer, Leshem Choshen, Judy Hoffman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13467.txt.

Description

Pratik Ramesh and co-authors tackle interference when merging expert models into one multitask model. Models trained apart on distinct tasks often clash, which degrades the merged model's performance. They define Cross-Task Interference as drift in the merged model's representation versus its parts. Their Resolving Interference method adapts experts to be functionally orthogonal to other tasks' spaces. It needs only unlabeled auxiliary data, so it suits data-scarce cases. It improved top merging methods and generalization to unseen domains in their tests.

Catalog record

CREATORPratik Ramesh, George Stoica, Arun Iyer, Leshem Choshen, Judy Hoffman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39902
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13467

Browse the full library · About the Tower of Babel