The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu · 2026
Read The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models by Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03425.txt.
Description
Yan Wang and five co-authors question a common belief about Mixture of Experts models, which route each input to a small set of expert networks. People often assume those experts specialize by topic. The team built COMMITTEEAUDIT, a tool that studies routing at the level of expert groups rather than single experts.
Across three models and the MMLU test set, they found a small Standing Committee of experts that takes most routing traffic in every domain. Those core experts seem to anchor reasoning and syntax, while outer experts hold topic knowledge. The result suggests these models centralize far more than expected. Training rules that force even expert use may fight the model's natural path and waste effort.
Catalog record
| CREATOR | Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03425