Heterogeneous Swarms : Jointly Optimizing Model Roles and Weights for Multi-LLM Systems

Shangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister · 2025

Read Heterogeneous Swarms : Jointly Optimizing Model Roles and Weights for Multi-LLM Systems by Shangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04510.txt.

Description

Shangbin Feng and co-authors design teams of language models automatically in a 2025 paper. Pairing several models can beat any single model on hard tasks. Someone must still decide which model does what and how much each counts. Hand tuning those choices is slow and rarely finds the best mix.

Their swarm method tunes roles and weights together in rounds. Roles form a flow chart of models passing work along in order. A score rates each model's true share in the best chart found. Across a dozen tasks, the tuned swarms beat fifteen rival design methods.

Catalog record

CREATORShangbin Feng Zifeng Wang Palash Goyal Yike Wang Weijia Shi, Huang Xia Hamid Palangi Luke Zettlemoyer Yulia Tsvetkov Chen-Yu Lee Tomas Pfister
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-04510

Browse the full library · About the Tower of Babel