MoE-GPS: G uidlines for P rediction S trategy for Dynamic Expert Duplication in MoE Load Balancing

Haiyue Ma, Zhixu Du, Yiran Chen · 2025

Read MoE-GPS: G uidlines for P rediction S trategy for Dynamic Expert Duplication in MoE Load Balancing by Haiyue Ma, Zhixu Du, Yiran Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07366.txt.

Description

Haiyue Ma, Zhixu Du, and Yiran Chen guide how to predict expert loads in mixture-of-experts models spread over many chips. Hot experts swamp their home chips while others idle. Duplicating busy experts helps only if the rush is foreseen. Predicting each token's expert is costly; predicting the overall spread is cheap.

Their MoE-GPS framework prices predictor designs against whole-system speed. Distribution-only forecasts slash the overhead of the classic route. On a Mixtral model graded with MMLU, the cheap route won clearly. End-to-end speed rose by more than twenty-three percent. The guide adapts its advice to each deployment's shape.

Catalog record

CREATORHaiyue Ma, Zhixu Du, Yiran Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07366

Browse the full library · About the Tower of Babel