Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search
Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov · 2026
Read Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search by Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-23739.txt.
Description
Lukhi, Moradiya, Timofte, and Ignatov automate the search for vision models built from four different expert networks, in a 2026 paper on neural architecture search. Mixture of experts models route inputs through specialized networks whose outputs a gating network blends. Mixing four architecture families could combine their strengths, but the possible quartets from twenty nine families run to tens of thousands.
Over a twenty eight day campaign on one graphics card, the pipeline generated more than four thousand candidate models and evaluated about a thousand successfully. A critical flaw surfaced: alphabetical listing anchored the entire explored space, under five percent of the total, to one family, AirNet. The authors pin down the bias, trace it to the generator, and propose stratified random sampling as the fix. Within the covered scope, ShuffleNet and MobileNetV3 pairings score highest.
Catalog record
| CREATOR | Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.23739