MERGE đťź‘ \mathbf{\text{MERGE}^{3}} : Efficient Evolutionary Merging on Consumer-grade GPUs
Tommaso Mencattini, Adrian Robert Minut, Donato Crisostomi, Andrea Santilli, Emanuele Rodolà · 2025
Read MERGE đťź‘ \mathbf{\text{MERGE}^{3}} : Efficient Evolutionary Merging on Consumer-grade GPUs by Tommaso Mencattini, Adrian Robert Minut, Donato Crisostomi, Andrea Santilli, Emanuele RodolĂ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10436.txt.
Description
Breeding better language models by merging can now run on a home gaming computer, thanks to a 2025 framework by Tommaso Mencattini and co-authors. Evolutionary merging mixes trained models into strong multi-task hybrids, but scoring each candidate the usual way is far too costly for consumer hardware.
Their system, MERGE-cubed, cuts that scoring cost fiftyfold in three moves. It evaluates on a shrunken data set, estimates each model's abilities with item response theory, borrowed from exam science, and evolves merges using those estimates. Performance holds up, and the method shines at blending models across languages. Theory backs the shortcuts, and the code is open, widening who gets to build merged models.
Catalog record
| CREATOR | Tommaso Mencattini, Adrian Robert Minut, Donato Crisostomi, Andrea Santilli, Emanuele RodolĂ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10436