Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
Zekai Li, Ji Liu, Yiqing Huang, Ziqiong Liu, Dong Li, Emad Barsoum · 2026
Read Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation by Zekai Li, Ji Liu, Yiqing Huang, Ziqiong Liu, Dong Li, Emad Barsoum (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30753.txt.
Introduction
Li and colleagues treat diffusion language-model decoding as a control problem to cut delay from repeat refinement. Old methods use step-local confidence rules or fixed plans, sensitive to prompt and task change and missing position effects. Their trace-aware design has two parts. Temporal-Spatial Parallel Decoding uses a light controller to fix settled tokens. Confidence Extrapolation, a training-free state-space module, predicts future logit trends with uncertainty. Together, the parts cut needless denoising rounds while keeping output quality.
Catalog record
| CREATOR | Zekai Li, Ji Liu, Yiqing Huang, Ziqiong Liu, Dong Li, Emad Barsoum |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50924 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30753
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation Zekai Li, Ji Liu, Yiqing Huang, Ziqiong Liu, Dong Li, Emad Barsoum Abstract Diffusion-based large language models (dLLMs) support parallel text generation via iterative denoising, yet inference remains latency-heavy because many steps are spent on redundant refinement and repeated remasking of tokens whose final values are already determined. Prior acceleration methods mainly depend on step-local confidence heuristics or fixed schedules, which are sensitive to prompt and task variation and ignore strong positional effects within a sequence. We cast diffusion decoding as a dynamic control problem and show that token-wise denoising trajectories provide the key signal for reliable control. We propose a trace-aware decoding framework with two components. First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporal-spatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed. Second, we introduce Confidence Extrapolation (CE) , a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions, including safe look-ahead and targeted stabilization when trajectories are oscillatory or underconfident. Together, TSPD and CE reduce unnecessary denoising iterations while preserving output quality, and they compose cleanly with system optimizations such as KV caching. 1 Introduction Diffusion-based large language models (dLLMs) ( 32 ; 49 ; 22 ; 50 ) offer an alternative to autoregressive decoding ( 37 ; 34 ) by updating many token positions in parallel through iterative denoising. This framework supports flexible generation orders, infilling, and a direct quality speed trade off through the number of denoising steps ( 15 ; 40 ; 30 ) . Recent masked diffusion Language models ( 36 ; 13 ; 31 ) and LLaDA style models ( 32 ) show that diffusion can be competitive at scale. Despite this parallelism, inference latency remains high in practice. Each denoising step typically requires a full sequence forward pass under bidirectional attention ( 38 ; 46 ; 11 ) , and many steps are spent refining tokens whose final values are already determined. Repeated remasking further amplifies this cost when uncertainty persists at a small subset of positions. Prior work reduces this overhead by reusing computation across steps, adjusting unmasking schedules, or committing more tokens per step ( 48 ; 47 ; 23 ; 44 ; 20 ) . These methods improve throughput, but most commit and stopping decisions still depend on step local heuristics, fixed schedules, or global thresholds ( 24 ; 8 ; 6 ) . Such signals are brittle because diffusion confidence is not uniformly calibrated across prompts, domains, and token positions. Empirically, diffusion decoding exhibits structured temporal and spatial behaviors, including delayed stabilization, underconfident but correct tokens, and oscillatory confidence ( 19 ; 43 ) . A single step snapshot or heuristic rule cannot separate these regimes. As a result, local rules often pay for redundant denoising after convergence, or they commit prematurely and lose quality. In this work, we treat diffusion decoding as a sequential control problem rather than a sequence of independent threshold tests. A key observation is that denoising produces token wise trajectories across steps, such as confidence and entropy traces, whose temporal patterns reveal both correctness and stability. This shifts the question from “is the token confident at step t ” to “does the trajectory indicate convergence, and how will it change if we continue denoising.” Answering this requires a temporal-spatial-aware controller that can interpret patterns such as steady growth, oscillation, and delayed stabilization, as well as a mechanism that can exploit predictable trends to avoid unnecessary steps. We propose Temporal-Spatial Parallel Decoding (TSPD) , a temporal-spatial-aware framework that improves both efficiency and accuracy. TSPD introduces a lightweight temporal-spatial controller that predicts whether a position is ready to be fixed using sequential trace features (confidence, entropy, and momentum over recent steps) together with relative spatial position within a block. A shared sequential perception model captures dynamics that step local classifiers and hand tuned thresholds often miss, improving robustness across prompts and positional effects. To reduce waiting time when confidence is rising but not yet decisive, we further introduce a training-free and plug-in Confidence Extrapolation mechanism (CE) . CE models confidence evolution as a simple state space process and forecasts near future confidence together with uncertainty. A risk-aware horizon rule limits look ahead based on left context progress and forecast uncertainty, so extrapolation is applied only when supported by reliable history. We evaluate our framework (TSPD and CE) on LLaDA-8B-Instruct ( 32 ) across four benchmarks spanning arithmetic reasoning and code generation, including GSM8K ( 10 ) , MATH ( 14 ) , HumanEval ( 7 ) , and MBPP ( 3 ) . Across multiple dLLM backbones and generation settings, our method delivers a consistently stronger speed quality tradeoff than state of the art baselines. On GSM8K with 256 token generation, it achieves 5.0\times speedup without cache, and compounds to 11.2\times when combined with KV caching, showing that our controller level gains are complementary to system optimizations. The advantage increases with longer generation: at 1024 tokens, our approach attains 64.1 Tokens Per Second (TPS) and up to 58.3\times speedup, while keeping accuracy essentially unchanged and in some cases slightly improved. In summary, our contributions are threefold: • We propose TSPD , a temporal spatial aware decoding controller that uses temporal confidence dynamics and positional context to make robust token fixing decisions, mitigating the brittleness of step local heuristics. • We introduce a training free and plug and play confidence extrapolation module that forecasts near future confidence under a state space model with quantified uncertainty, and applies risk aware look ahead to complement diverse decoding controllers. • We provide extensive evaluations and ablations on a wide range of dLLMs, task domains, and generation settings, demonstrating substantial speedups with near lossless accuracy and full compatibility with KV cache. 2 Related Work 2.1 Diffusion-Based Large Language Models Diffusion models ( 39 ; 41 ) have been extended from continuous data to discrete sequences, enabling non-autoregressive generation via iterative denoising. Early discrete diffusion work established Markov formulations in categorical spaces, including multinomial diffusion ( 17 ) and D3PM ( 2 ) . Continuous-time views ( 5 ) further unify discrete diffusion and support flexible sampling schedules. SEDD ( 27 ) improves learning by modeling ratios of marginal probabilities with a score-entropy objective, narrowing the gap to autoregressive language models (LMs). Building on these foundations, masked dLLMs have become a practical paradigm for text generation. MDLM ( 36 ) shows that a simplified masked-diffusion recipe can substantially improve perplexity and sampling quality. At scale, diffusion LLMs are developed either by adapting strong autoregressive backbones (e.g., DiffuLLaMA) ( 12 ) or by training from scratch (e.g., LLaDA) ( 32 ) , achieving competitive instruction following and reasoning. Dream 7B ( 49 ) further advances open diffusion LLMs and highlights diffusion-specific capabilities such as arbitrary-order generation and infilling with tunable quality–speed trade-offs. Figure 1 : Same local confidence with different trace can lead to different decoding results. Figure 2 : Tokens at more rightward spatial positions tend to stabilize later. 2.2 Decoding Acceleration for Diffusion LLMs Despite parallel updates, diffusion decoding can be latency-heavy because each denoising