CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging
Zongzhen Yang, Binhang Qi, Hailong Sun, Wenrui Long, Ruobing Zhao, Xiang Gao · 2025
Read CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging by Zongzhen Yang, Binhang Qi, Hailong Sun, Wenrui Long, Ruobing Zhao, Xiang Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01874.txt.
Description
Zongzhen Yang and co-authors improve in 2025 a cheap way to fuse specialist models into one multitask model. Merging works on task vectors, the weight shifts each fine-tune produced, but the vectors clash where they overlap and pile up unevenly. Their framework, CABS, prunes the vectors with two ideas in sequence.
Conflict-aware pruning masks each vector in turn so the survivors claim distinct parameters. Balanced pruning then keeps key weights while spreading what remains evenly across layers. Across tasks and model sizes, the merged models beat prior merging methods in the team's tests, without any retraining.
Catalog record
| CREATOR | Zongzhen Yang, Binhang Qi, Hailong Sun, Wenrui Long, Ruobing Zhao, Xiang Gao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01874