Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding

Sudhanshu Agrawal 1, Risheek Garrepalli, Raghavv Goel, Christopher Lott, Fatih Porikli · 2025

Read Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding by Sudhanshu Agrawal 1, Risheek Garrepalli, Raghavv Goel, Christopher Lott, Fatih Porikli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18085.txt.

Introduction

Sudhanshu Agrawal and 5 co-authors speed up diffusion language models with a graph of draft tokens. Their 2025 method, Spiffy, deals with a new class of models that write in blocks, both ways at once. Such models could write far faster than step-by-step ones, if draft checks keep up.

Spiffy drafts from the model itself, so no second draft model is needed. Draft states form a directed graph, tuned off-line for high take rates and pruned live for speed. Paired with cache and threshold tricks, it cut model runs by large factors on three models. The output spread stays provably the same.

Catalog record

CREATORSudhanshu Agrawal 1, Risheek Garrepalli, Raghavv Goel, Christopher Lott, Fatih Porikli
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-18085

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding Sudhanshu Agrawal 1, Risheek Garrepalli, Raghavv Goel, Christopher Lott, Fatih Porikli Abstract Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token generation rates. However, currently available open-source dLLMs often generate at much lower rates, typically decoding only a single token at every denoising timestep in order to maximize output quality. We present Spiffy, a speculative decoding algorithm that accelerates dLLM inference by 2.8 − 3.1 × \mathbf{2.8{-}3.1\times} while provably preserving the model’s output distribution. This work addresses the unique challenges involved in applying ideas from speculative decoding of AR-LLMs to the dLLM setting. Spiffy proposes draft states by leveraging the dLLM’s distribution itself in an auto-speculative manner. This approach is efficient and effective, and eliminates the overheads of training and running an independent draft model. To structure the candidate draft states, we propose a novel directed draft graph which is uniquely designed to take advantage of the bidirectional, block-wise nature of dLLM generation and can be verified in parallel by the dLLM. To further optimize the structure of these draft graphs, we introduce an efficient, offline calibration algorithm that procedurally determines high-quality graph configurations. These optimized draft graphs, enabling increased acceptance rates, lead to a significant boost in the overall speedup achieved by the system. Crucially, Spiffy is also complementary to other recent innovations in improving dLLM generation speeds such as KV-caching and multi-token unmasking. We demonstrate that when combined with such parallel decoding algorithms, Spiffy is able to effectively multiply the benefits of these methods leading to total speedups of up to 7.9 × \mathbf{7.9\times} . 1 Introduction A majority of large language models (LLMs) today are autoregressive LLMs (AR-LLMs) ( Brown et al. 2020 ) . These models operate in a left-to-right fashion, generating new tokens that simply depend on the previous tokens in the sequence. Such LLMs have demonstrated impressive capabilities in a variety of domains. Yet, due to their autoregressive nature, they are constrained to generating only a single token per model inference. As overall latency is typically dominated by the total model inference time, this constraint slows down the LLM application as a whole. More recently, the emergence of diffusion LLMs (dLLMs) ( Nie et al. 2025 ; Inception AI Labs et al. 2025 ) represents a paradigm shift in language modeling. Unlike AR-LLMs, dLLMs are inherently parallel in nature and are not bound by a strict causal factorization. Instead, they model the joint distribution of over entire token blocks, leveraging bidirectional attention ( Devlin et al. 2019 ) , and allow tokens to be decoded in arbitrary orders. dLLMs frame token decoding as an iterative denoising process, taking a sequence from an initial high-noise state to its final low-noise state, representing the generated token sequence. While commercial dLLMs such as Mercury ( Inception AI Labs et al. 2025 ) and Gemini Diffusion ( Google 2025 ) may achieve token rates in excess of 1000 tokens/second, open-sourced dLLMs such as LLaDA ( Nie et al. 2025 ) and Dream ( Ye et al. 2025 ) do not yet achieve these decoding speeds. In particular, to maintain output quality, these dLLMs generate only a single token per model inference, restricting their efficiency. On the other hand, recent work on AR-LLMs attempts to circumvent their autoregressive constraints via speculative decoding ( Leviathan et al. 2023 ; Chen et al. 2023 ) . This has emerged as a popular and powerful method to accelerate AR-LLMs due to its accuracy guarantee and potential for efficient deployment. However, transferring the ideas developed for AR-LLM speculation to dLLMs encounters fundamental challenges. Since dLLM token generations have bidirectional dependence and model the joint distribution over sequences, drafting and verification must take this into account to remain lossless. Typical speculative strategies, such as tree-based drafting, also require substantial re-thinking. Importantly, the choice of a compatible and efficient drafter is essential to enable speculation in dLLMs. In this work, we address these challenges and introduce speculative decoding acceleration into dLLM systems. Spiffy ( Sp eculation for D iff usion LLM Efficienc y ), is a novel algorithm that accelerates dLLM inference with a draft then verify strategy which provably maintains output quality. During generation, Spiffy samples draft states from the dLLM distribution itself, avoiding the need for, and costs of an auxiliary draft model. Spiffy proposes these drafts in the form of a directed draft graph to maximize their acceptance. While these draft graphs are similar in motivation to draft trees ( Li et al. 2024 ; Miao et al. 2024 ; Jeon et al. 2024 ) , Spiffy’s draft graphs respect and also take advantage of the bidirectional nature of dLLM generation. In contrast to trees, nodes in these draft graphs may have multiple parent nodes and consequently multiple pathways to being accepted. This drafting structure more effectively captures the state dependencies inherent to dLLM generation. To further enhance these draft graphs and maximize their efficacy, we additionally introduce a novel calibration procedure to determine optimized draft graph structures. This calibration procedure is performed once, offline, and the resulting graph configurations remain fixed throughout the denoising process, significantly increasing acceptance rates during generation. Spiffy is versatile and extensible as a framework, remaining compatible with alternate drafting mechanisms and arbitrary denoising rates. This allows for it to be combined with other parallel decoding techniques for dLLMs and optimizations such as KV-caching to further boost inference speeds. In particular, we are able to effectively combine Spiffy with the parallel decoding scheme proposed by fast-dllm ( Wu et al. 2025 ) to multiply its speedup benefits. Thus, we propose Spiffy as a powerful, plug-in acceleration solution for a variety of dLLM systems. We summarize our key contributions below. 1. Drafting and verification for dLLMs We introduce Spiffy, a novel, versatile framework for speculative decoding of block-wise dLLMs. We propose seamless multi-token drafting and define verification as a lossless procedure, proven formally and experimentally. 2. Auto-speculation We demonstrate the efficacy of this framework with an auto-speculative approach, leveraging the current dLLM distribution to speculate draft states, thus avoiding the need to train and run a draft model. 3. Directed draft graphs We define directed draft graphs, a generalization of speculative draft trees, designed to take advantage of the bidirectional nature of dLLMs and increase the likelihood of successive draft acceptances. 4. Offline graph structure calibration To further optimize these draft graph structures, we develop a novel calibration algorithm that procedurally determines high-quality graph configurations, boosting acceptance rates during speculation. Our contributions are validated with rigorous experimentation and ablations with multiple open-source dLLMs (LLaDA-Base-8B, LLaDA-Instruct-8B, LLaDA-1.5-8B) with standard benchmarks (GSM8K, HumanEval, MATH, MBPP), enabling up to 7.9\times total speedup. 2 Related Work Diffusion LLMs Diffusion models ( Sohl-Dickstein et al. 2015 ; Ho et al. 2020 ) have become a dominant approach in generative modeling, particularly excelling in continuous domains such as image synthesis ( Song et al. 2020 ; Saharia et al. 2022 ; Esser et al. 2024 ; Black Forest Labs et al. 2025 ) . Their capacity for parallel generation has motivated their adaptation to language modeling, a discrete domain with inherently

Read the complete text