ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma · 2026
Read ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping by Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10088.txt.
Introduction
Zijian Zhu and coauthors speed up diffusion language models by skipping unimportant tokens in early layers. These models process the full context at every iteration, making inference costly despite parallel generation. The team finds that keys, values and hidden states barely change between successive steps. ES-dLLM estimates token importance from those small changes plus past confidence, then skips low value tokens with no retraining, gaining large speedups.
Catalog record
| CREATOR | Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50196 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10088
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma Abstract Diffusion large language models (dLLMs) are emerging as a promising alternative to autoregressive models (ARMs) due to their ability to capture bidirectional context and the potential for parallel generation. Despite the advantages, dLLM inference remains computationally expensive as the full input context is processed at every iteration. In this work, we analyze the generation dynamics of dLLMs and find that intermediate representations, including key, value, and hidden states, change only subtly across successive iterations. Leveraging this insight, we propose ES-dLLM , a training-free inference acceleration framework for dLLM that reduces computation by skipping tokens in early layers based on the estimated importance. Token importance is computed with intermediate tensor variation and confidence scores of previous iterations. Experiments on LLaDA-8B and Dream-7B demonstrate that ES-dLLM achieves throughput of up to 226.57 and 308.51 tokens per second (TPS), respectively, on an NVIDIA H200 GPU, delivering 5.6 × \times to 16.8 × \times speedup over the vanilla implementation and up to 1.85 × \times over the state-of-the-art caching method, while preserving generation quality. The source code is available at https://github.com/zhuzj19/ES-dLLM . 1 Introduction Autoregressive models (ARMs) have been the dominant paradigm in large language models (LLMs), achieving remarkable success in a wide range of applications ( 4 ) . ARMs generate text in a left-to-right pattern, producing tokens sequentially. Recently, diffusion-based LLMs (dLLMs) have emerged as a promising alternative. Unlike ARMs, dLLMs generate text through an iterative denoising process over a sequence of mask tokens, enabling bidirectional attention and offering the potential for parallel decoding ( 23 ) . Industrial dLLMs such as Mercury ( 12 ) and Gemini Diffusion ( 9 ) have drawn significant attention for their ultra-fast generation speed. Despite this progress, open-source dLLMs, such as LLaDA ( 15 ) and Dream ( 22 ) , remain far less efficient, even slower than ARMs of comparable size. A key factor contributing to the inefficiency of dLLMs is that each iteration processes the entire sequence as input, introducing substantial computational overhead. To mitigate this cost, recent studies ( 14 ; 21 ; 13 ) have proposed techniques such as caching and parallel decoding mechanisms to improve generation efficiency. Nevertheless, more opportunities remain for further acceleration. In each iteration during dLLM generation, only one or a few tokens with the highest confidence are unmasked, while the majority of mask tokens are processed without producing useful results. Moreover, since the inputs of adjacent iterations differ only in the positions of newly unmasked tokens, the intermediate states of most tokens remain almost unchanged. Despite this redundancy, conventional inference procedures still compute logits for all token positions, leading to excessive and unnecessary computation. To better understand these inefficiencies, we conduct a series of experiments to analyze the characteristics during the dLLM generation process. The results reveal that both intermediate tensors and confidence scores exhibit only subtle variation across successive iterations. Therefore, we identify opportunities to predict the importance of token positions and to eliminate redundant computation in early layers for tokens that contribute little to the outcome. Motivated by these observations, we propose ES-dLLM 1 1 1 The name is inspired by early-exit (EE) ( 7 ) , which accelerates inference by exiting “easy” tokens early via trained EE layers. In contrast, ES-dLLM early-skip (ES) irrelevant positions without additional training. , a training-free inference acceleration framework designed for diffusion LLMs. Specifically, it focuses on optimizing redundant token computation in each iteration, estimating token importance using intermediate variations and prior confidence scores, and skipping low-importance positions in early layers of the inference process. ES-dLLM consists of two key components: • Importance Score Estimation : ES-dLLM skips irrelevant token positions by estimating their importance scores in early layers of the model, based on the variation of intermediate tensors and confidence from previous iterations. Using importance scores, only the top- k positions are selected for further inference, while the remaining positions are skipped in the current iteration. • Partial Cache Update and Early Skip : ES-dLLM maintains intermediate tensors as caches to be reused in subsequent iterations, including key and value tensors for attention layers and hidden states as the indicator for variation estimation in importance score. Since only a subset of tokens passes through the inference, caches corresponding to the selected positions are updated with an in-place scatter operation, while the others are reused without recomputation. The experimental results show that ES-dLLM substantially accelerates inference, achieving throughput of up to 226.57 and 308.51 tokens per second (TPS) on the LLaDA-8B and Dream-7B models, respectively, using an NVIDIA H200 GPU, without compromising generation quality. In summary, this paper makes the following contributions: 1. We analyze the characteristics during the dLLM generation process and observe that intermediate tensors and confidence scores of most positions exhibit only minor variation across iterations , thereby revealing opportunities to eliminate redundant computation. 2. We propose ES-dLLM, a training-free inference acceleration framework that reduces per-iteration computation by early-skipping low-importance token positions. 3. We conduct extensive experiments and ablation studies, demonstrating that ES-dLLM achieves significant speedups of 5.6 \times -16.8 \times over the original implementation and up to 1.85 \times compared to state-of-the-art caching methods, all without sacrificing generation quality . 2 Preliminary on diffusion large language models Diffusion large language models introduce a new paradigm in natural language processing that adopts the diffusion mechanism. Unlike traditional autoregressive models, which generate text sequentially in a token-by-token manner, dLLMs go through a series of denoising iterations, progressively unmasking tokens appended to the input. Let \mathcal{V} denote the vocabulary that includes a special mask token [\mathtt{mask}]\in\mathcal{V} . Given an input sequence {\bm{x}}=(x_{1},x_{2},\ldots,x_{n}) where x_{i}\in\mathcal{V} , text generation begins by appending l mask tokens to form {\bm{x}}^{(0)}=(x_{1},x_{2},\ldots,x_{n},[\mathtt{mask}],\ldots,[\mathtt{mask}]) , where l specifies the desired generation length. In the t -th iteration, a Transformer-based token predictor f is used to estimate the probability distribution over the vocabulary for all token positions (rather than just the next token as in ARMs) in the sequence. p_{\theta}(\hat{{\bm{x}}}^{(t)}|{\bm{x}}^{(t-1)})=f({\bm{x}}^{(t-1)};\theta) From this distribution, a subset of mask tokens is replaced with sampled tokens, producing the updated sequence {\bm{x}}^{(t)} . Several replacement strategies exist, and a widely used approach is to unmask positions with the highest confidence ( 23 ) , where confidence for each position is defined as the maximum probability over the vocabulary. This iterative process continues until all mask tokens are replaced. Compared to ARMs, which condition predictions solely on preceding tokens, dLLMs leverage the entire sequence context in each iteration. This enables them to capture bidirectional dependencies and generate more coherent text. However, it also introduces computational challenges due to bidirectional attention and flexible, non-sequential generation order. At the same time, computing entire logits creates opportunities for parallel decoding 21 . 3 Related works for dLLM inference acceleration Semi-autoregressive Generation. LLaDA ( 15 ) adopts a semi-autoregressive generation strategy that partitions the output sequence into multiple blocks, each generated in a diffusion manner while preserving sequential order across blocks. This approach constrains the generation order, but provides better performance. Furthermore, 2 trained the BD3-LM model, which applies unidirectional attention between output blocks. This design enables lossless Key-Value (KV) caching of previous blocks, but sacrifices