Planned Diffusion
Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, Michael Carbin · 2025
Read Planned Diffusion by Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, Michael Carbin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18087.txt.
Introduction
Daniel Israel, with six co-authors, lets a language model plan its own writing order before filling in the text. Word-by-word models are slow in one way, while diffusion models that write many tokens at once need a smart order for revealing them. Fixed rules for that order trade quality against speed.
Under planned diffusion, the model first sketches a plan that splits the answer into independent chunks. It then fills all chunks in parallel. On a standard chat benchmark the approach ran notably faster than plain word-by-word writing. Model builders get a way to let the text decide how it wants to be written.
Catalog record
| CREATOR | Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, Michael Carbin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18087
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Planned Diffusion Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, Michael Carbin Abstract A central challenge in large language model inference is the trade-off between generation speed and output quality. Autoregressive models produce high-quality text but generate tokens sequentially. Diffusion models can generate tokens in parallel but often need many iterations to match the same quality. We propose planned diffusion , a hybrid method that combines the strengths of both paradigms. Planned diffusion works in two stages: first, the model creates a short autoregressive plan that breaks the output into smaller, independent spans. Second, the model generates these spans simultaneously using diffusion. This approach expands the speed–quality Pareto frontier and provides a practical path to faster, high-quality text generation. On AlpacaEval, a suite of 805 instruction-following prompts, planned diffusion achieves Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x speedup over autoregressive generation with only 0.87% to 5.4% drop in win rate, respectively. Our sensitivity analysis shows that the planning mechanism of planned diffusion is minimal and reliable, and simple runtime knobs exist to provide flexible control of the quality-latency trade-off. 1 Introduction Language model text generation is subject to a fundamental tradeoff between modeling textual dependencies and leveraging the parallel computation of modern hardware. Autoregressive (AR) models have defined the state-of-the-art in quality by excelling at the former ( Radford et al. 2019 ) ; generating tokens sequentially allows them to capture dependencies with high fidelity, but this process inherently creates a latency bottleneck. Conversely, diffusion language models ( Sahoo et al. 2024 ) are designed for parallelism but can struggle with coherence when generating with a low number of iterative steps needed for low latency ( Wu et al. 2025 ) . This establishes a performance tradeoff for LLMs, in which there exists a difficult compromise between inference speed and output quality. We propose planned diffusion , a framework that challenges this compromise by treating text generation as a dynamic parallel scheduling problem. The motivation behind this technique is that the dependency structure among tokens is context-dependent; typical language model responses include semantically independent spans of tokens that can generate concurrently. For example, an answer that contains a bulleted list can generate each bullet point in parallel ( Ning et al. 2024 ) . Figure 1: A real example of planned diffusion. (1) Autoregressive Plan : The model first generates a sequential plan using control tags to define the structure and length of independent text spans. (2) Programmatic Scaffold : This plan is then translated into a scaffold where each span is initialized with a corresponding number of mask tokens. (3) Diffusion Denoise : Finally, the model denoises all spans in parallel with diffusion, generating the text for each section simultaneously to produce the complete response. Planned diffusion realizes this concept through a novel hybrid architecture where a single, unified model transitions between autoregressive and diffusion-based generation. First, in a sequential planning stage, the model operates autoregressively to generate a high-level execution plan composed of structural control tags. This plan partitions the task into a set of conditionally independent sub-tasks. Second, in a parallel diffusion stage, the model executes this plan, simultaneously generating the text for all planned segments. By integrating these two modes, our method optimizes for parallel generation while preserving coherence and quality. This single-model, hybrid approach offers a distinct architectural advantage over other acceleration techniques, such as speculative decoding ( Leviathan et al. 2023 ) , which require multiple, separate models. To the best of our knowledge, this is the first text-only model that is trained with both discrete diffusion and autoregressive objectives. Figure 1 presents a sample generation produced by our planned diffusion model. We make the following contributions: 1. We introduce planned diffusion , a new parallel generation technique that decomposes text generation into a sequential planning stage and a parallel diffusion stage. 2. We designed the control tag language, model training methodology, and inference algorithm that enable a single model to perform this hybrid generation process. 3. We demonstrate that planned diffusion achieves a state-of-the-art trade-off between speed and quality. On AlpacaEval, a suite of 805 instruction-following prompts, planned diffusion achieves Pareto-optimal trade-off between quality and latency; it achieves a 1.27x to 1.81x speedup over autoregressive generation while incurring only a 0.87% to 5.4% drop in win rate, respectively. 4. We present sensitivity analysis which validates that the planning mechanism design of our model is minimal and reliable; we show that simple runtime knobs exist to offer tunable control over the trade-off between generation speed and quality. 2 Related Work Our work builds upon recent developments in non-autoregressive and parallel decoding strategies. We position our contributions in the context of two primary research areas: diffusion-based language models and methods for achieving semantic parallelism. Diffusion Language Models. Diffusion models have recently emerged as a new paradigm for generative language tasks ( Austin et al. 2021 ; Sahoo et al. 2024 ) . A significant body of research is focused on accelerating the inference process, which traditionally involves many iterative denoising steps ( Liu et al. 2025a ) . These acceleration techniques include KV caching for diffusion models ( Ma et al. 2025 ; Liu et al. 2025c ) , the use of autoregressive verification ( Hu et al. 2025 ; Israel et al. 2025a ) , and the development of fast sampling strategies that reduce the number of required steps ( Wu et al. 2025 ; Li et al. 2025 ; Hong et al. 2025 ) . Our proposed planned diffusion framework is orthogonal and complementary to these methods; any diffusion sampling strategy can be integrated into the diffusion component of our algorithm to further increase performance. Other related work include block diffusion ( Arriola et al. 2025 ) which enforces an autoregressive structure over blocks and planned denoising ( Liu et al. 2025b ) which learns an adaptive denoising schedule. While relevant, neither targets semantic parallelism, which is the purpose of planned diffusion. Semantic Parallelism. While diffusion models achieve token-wise parallelism, they are not trained to achieve parallelism across larger portions of text at the semantic level. We define semantic parallelism as a broad class of techniques that produce models capable of parallelizing over semantically independent chunks of tokens. Many recent works explore semantic parallelism ( Ning et al. 2024 ; Liu et al. 2024 ; Jin et al. 2025 ; Rodionov et al. 2025 ; Pan et al. 2025 ; Wen et al. 2025 ; Yang et al. 2025 ) . While existing works operate within a purely autoregressive framework, to the best of our knowledge, our work is the first to propose a hybrid autoregressive-diffusion model for text. By combining an autoregressive planning phase with a parallel diffusion phase, our method utilizes diffusion parallelism in a structured manner, presenting a novel approach for efficient text generation. Other Parallel Generation Techniques. There is a rich line of work on parallel generation. Insertion-based models parallelize by inserting tokens into partial positions, reducing decoding depth via simultaneous span updates ( Stern et al. 2019 ) . Speculative decoding accelerates autoregressive models by drafting multiple tokens and verifying them in parallel ( Leviathan et al. 2023 ;