RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang · 2025

Read RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress by Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23995.txt.

Introduction

Ruixuan Huang and six co-authors expose a denial-of-service weakness hiding in mixture-of-experts language models. These models route each token to a few specialist subnetworks, and systems spread the experts across devices for efficiency. Nothing at inference time forces balance, so a crafted input can herd all tokens onto the same few experts, overloading some devices while others idle and breaking latency promises.

Their attack, RepetitionCurse, is a cheap black-box method built from simple repetitive token patterns that exploit a universal quirk of router behavior, no insider access needed. On the widely deployed Mixtral model, the attack more than tripled end-to-end inference latency. The authors frame the flaw as a conversion of an efficiency mechanism into an availability risk for anyone serving mixture-of-experts models, and the paper was accepted at ICML 2026.

Catalog record

CREATORRuixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-23995

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang Abstract Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top- k k experts, which creates computational bottlenecks on certain devices while forcing others to idle. This converts an efficiency mechanism into a denial-of-service attack vector, leading to violations of service-level agreements for time to first token. We propose RepetitionCurse, a low-cost black-box strategy to exploit this vulnerability. By identifying a universal flaw in MoE router behavior, RepetitionCurse constructs adversarial prompts using simple repetitive token patterns in a model-agnostic manner. On widely deployed MoE models like Mixtral-8x7B, our method increases end-to-end inference latency by 3.063 × 3.063\times , degrading service availability significantly. 1 Introduction The capabilities of large language models (LLMs) continue to advance rapidly ( 4 ; 15 ) , but deploying dense models at scale incurs prohibitive computational and memory costs. To address this challenge, the Mixture-of-Experts (MoE) architecture ( 19 ) has emerged as a practical solution by sparsely activating only a subset of experts during inference, enabling substantially larger models to be deployed under fixed resource budgets. The MoE architecture introduces a fundamentally different model structure compared to dense models. Instead of a single monolithic feed-forward network (FFN) per layer, MoE layers contain multiple independent expert networks, with a router dynamically selecting a small subset of experts for each token. This structural independence makes conventional tensor parallelism (TP), which partitions individual operators across devices ( 20 ) inefficient for MoE inference. Instead, MoE models are naturally deployed using expert parallelism (EP), where distinct experts are entirely hosted on different physical devices ( 8 ; 9 ) , which minimizes inter-GPU communication and maximizes memory efficiency. Figure 1 : RepetitionCurse attack overview. Under attack inputs, all tokens are routed to the same top- k experts, causing a single GPU to become a straggler while other GPUs have to idle and wait. However, this design also introduces a fragile dependency on the token-to-expert routing distribution ( 17 ) , creating a significant attack surface for Denial-of-Service (DoS) vulnerabilities, as illustrated in Figure 1 . During training, MoE models utilize expert and device balance losses or bias ( 5 ; 6 ) to enforce load balancing, expecting all experts to be trained evenly. However, these explicit constraints are typically absent during inference. Under natural language inputs, the router’s generalization ability results in a relatively uniform distribution of tokens across experts. In this paper, we demonstrate that this assumption is easily broken. We reveal that attackers can craft adversarial inputs, specifically utilizing repetitive token patterns that override the router’s training distribution. These inputs induce extreme routing imbalance, where the workload is targeted at a few bottleneck experts and their hosting GPUs. With EP, this forces all other GPUs to idle while waiting for the straggler device to complete its computation. This behavior constitutes an algorithmic DoS attack: by exploiting the architectural logic rather than flooding network bandwidth, an attacker can consume system resources multiplicatively and degrade service latency for all users within its batch . We introduce RepetitionCurse, a simple yet highly effective method to generate these DoS prompts that can cause router imbalance. Unlike adversarial attacks that require gradient access or searching ( 27 ; 12 ) , RepetitionCurse operates effectively in black-box settings. Our investigation of 139 MoE configurations with the most downloads on Huggingface reveals that nearly all current models are susceptible to this vulnerability . Our experiments show that on Mixtral-8x7B API, RepetitionCurse amplifies prefilling latency by 3.063\times (Section 4.2 ), turning the system’s parallelism against itself. This creates a dilemma for hardware providers during deployment: they are discouraged from using large-scale EP, as the system becomes increasingly vulnerable to these attacks as the degree of parallelism grows . As commercial inference services increasingly adopt highly sparse MoE models (e.g., DeepSeek-V3), this attack vector warrants urgent attention. Beyond service degradation, we uncover that this vulnerability poses a privacy risk. Since the latency spike is intrinsic to the physical deployment of experts, we demonstrate that an attacker can observe response times to infer hidden backend information, such as whether the target is an MoE model. Our contributions are summarized as follows: • We identify a widespread DoS vulnerability in EP-based MoE serving systems caused by routing imbalance. • We propose RepetitionCurse, a black-box attack method that uses repetitive tokens to induce router imbalance. • We demonstrate that routing latency serves as a side-channel for inferring backend model architecture. Mitigation: Since RepetitionCurse is a newly proposed attack vector, we propose defense strategies in Section 5.3 to ensure ethical compliance. 2 Backgrounds 2.1 Inference of Transformer-based LLMs The Transformer architecture serves as the fundamental backbone for current LLMs ( 22 ; 21 ) . The body of an LLM is constructed by stacking multiple Transformer blocks. Each block typically comprises a self-attention layer and a feed-forward network (FFN) layer. An FFN layer generally consists of two MLP layers. MoE models are replacing the dense FFN layers with MoE layers. The inference of Transformer-based autoregressive LLMs proceeds in two phases: • Prefill: The model processes the entire input sequence in parallel, computing the attention keys and values for all input tokens and populating the initial KV cache. The prefill phase is compute-bound , with its heavy computational load to process the full input sequence. • Decoding: The model generates output tokens sequentially in an autoregressive manner. The decoding phase is memory-bound due to the high memory bandwidth overhead incurred by frequent access to the KV cache. Modern inference engines, such as vLLM ( 14 ) and SGLang ( 26 ) , employ prefill-decoding separation strategies to allocate these phases to distinct hardware resources for improved performance. As a result, the latency of the prefill phase directly impacts the time to first token (TTFT), a key metric for interactive LLM serving ( 1 ) , potentially violating latency-sensitive service-level agreements ( 11 ) . 2.2 Mixture-of-Expert Models with Expert Parallelism Distinct from dense models, an MoE layer comprises multiple weight-independent experts. For each input token, only a small subset of these experts is activated. Formally, considering a model containing E experts per layer and input hidden state h , one MoE layer first determines the participation of experts via the logits provided by the router: G(h)=\text{Softmax}(h\cdot W_{\text{router}})\in\mathbb{R}^{E} (1) where W_{\text{router}} is the weight of the router. Then, based on Top- k routing strategy, the top- k experts (denoted as \mathcal{E} ) with the highest router logits are selected to process the token: y=\sum_{i\in\mathcal{E}}G(h)_{i}\cdot E_{i}(h) (2) Under expert parallelism (EP), the complete weights of one or more experts are hosted on a single GPU. Modern inference engines employ fused kernels, such as grouped GEMM ( 23 )

Read the complete text