Heterogeneous Swarms : Jointly Optimizing Model Roles and Weights for Multi-LLM Systems
Shangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister · 2025
Read Heterogeneous Swarms : Jointly Optimizing Model Roles and Weights for Multi-LLM Systems by Shangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04510.txt.
Description
Shangbin Feng and co-authors design teams of language models automatically in a 2025 paper. Pairing several models can beat any single model on hard tasks. Someone must still decide which model does what and how much each counts. Hand tuning those choices is slow and rarely finds the best mix.
Their swarm method tunes roles and weights together in rounds. Roles form a flow chart of models passing work along in order. A score rates each model's true share in the best chart found. Across a dozen tasks, the tuned swarms beat fifteen rival design methods.
Catalog record
| CREATOR | Shangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04510