Fine-Grained Model Merging via Modular Expert Recombination PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE

Haiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan · 2026

Read Fine-Grained Model Merging via Modular Expert Recombination PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Haiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06552.txt.

Description

Combining several fine-tuned models into one versatile model avoids retraining, but current merging methods treat each model as an indivisible block. Haiyun Qiu and 3 co-authors present MERGE, a method that merges at the component level instead. Attention layers and feed-forward layers, they note, merge differently and tolerate merging differently.

MERGE formulates component-wise merging as a two-goal optimization balancing cross-task performance against storage cost. A surrogate-assisted evolutionary search finds the Pareto-optimal configurations, the trade-offs where neither goal can improve without hurting the other. Good configurations can be reused, and modules can be recombined on demand for a given input. The design restores efficient batch inference that instance-specific merging had sacrificed.

Catalog record

CREATORHaiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06552

Browse the full library · About the Tower of Babel