MoETuner : Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing

Seokjin Go, Divya Mahajan · 2025

Read MoETuner : Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing by Seokjin Go, Divya Mahajan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06643.txt.

Description

Seokjin Go and Divya Mahajan tackled a speed problem in large models that split work among expert parts, in a 2025 paper. Only a few experts wake for each token, which saves cost. Spread across many graphics chips, though, uneven routing leaves some chips swamped while others wait.

Their system, MoETuner, places experts on chips by solving a planning problem that weighs load, talk between chips, and compute cost. It also uses a pattern in routing: tokens sent to one expert tend to go to a small set next. In tests, end-to-end speed rose by about nine percent on one node and by more across many nodes. The gain came from cutting the long tail of waits.

Catalog record

CREATORSeokjin Go, Divya Mahajan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-06643

Browse the full library · About the Tower of Babel