Fine-Grained Model Merging via Modular Expert Recombination PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE
Haiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan · 2026
Read Fine-Grained Model Merging via Modular Expert Recombination PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Haiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06552.txt.
Description
Combining several fine-tuned models into one versatile model avoids retraining, but current merging methods treat each model as an indivisible block. Haiyun Qiu and 3 co-authors present MERGE, a method that merges at the component level instead. Attention layers and feed-forward layers, they note, merge differently and tolerate merging differently.
MERGE formulates component-wise merging as a two-goal optimization balancing cross-task performance against storage cost. A surrogate-assisted evolutionary search finds the Pareto-optimal configurations, the trade-offs where neither goal can improve without hurting the other. Good configurations can be reused, and modules can be recombined on demand for a given input. The design restores efficient batch inference that instance-specific merging had sacrificed.
Catalog record
| CREATOR | Haiyun Qiu, Xingyu Wu*, Liang Feng, Kay Chen Tan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06552