Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search

Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov · 2026

Read Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search by Yashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-23739.txt.

Description

Lukhi, Moradiya, Timofte, and Ignatov automate the search for vision models built from four different expert networks, in a 2026 paper on neural architecture search. Mixture of experts models route inputs through specialized networks whose outputs a gating network blends. Mixing four architecture families could combine their strengths, but the possible quartets from twenty nine families run to tens of thousands.

Over a twenty eight day campaign on one graphics card, the pipeline generated more than four thousand candidate models and evaluated about a thousand successfully. A critical flaw surfaced: alphabetical listing anchored the entire explored space, under five percent of the total, to one family, AirNet. The authors pin down the bias, trace it to the generator, and propose stratified random sampling as the fix. Within the covered scope, ShuffleNet and MobileNetV3 pairings score highest.

Catalog record

CREATORYashkumar R Lukhi, Harsh Rameshbhai Moradiya, Radu Timofte, Dmitry Ignatov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.23739

Browse the full library · About the Tower of Babel