Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
Marianne Arriola, Yair SchiffHao PhungAaron GokaslanVolodymyr Kuleshov Department of Computer Science, Cornell University {marriola,yairschiff,hao}@cs.cornell.edu, {akg87,kuleshov}@cornell.edu · 2025
Read Encoder-Decoder Diffusion Language Models for Efficient Training and Inference by Marianne Arriola, Yair SchiffHao PhungAaron GokaslanVolodymyr Kuleshov Department of Computer Science, Cornell University {marriola,yairschiff,hao}@cs.cornell.edu, {akg87,kuleshov}@cornell.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22852.txt.
Introduction
Diffusion language models can sample tokens in parallel. That promises faster inference than step by step models. Marianne Arriola leads a team with four co-authors. In 2025 they propose an encoder decoder design. Prior diffusion models used decoder only designs. Those designs rerun the full network each step.
Diffusion work splits into two kinds of compute. One represents clean tokens. The other denoises corrupted tokens. An encoder can handle clean tokens once. A light decoder then refines the noisy sequence. The design also speeds block diffusion training. Tests cover summary, translation, and math reasoning. Trade offs between quality and throughput improve.
Catalog record
| CREATOR | Marianne Arriola, Yair SchiffHao PhungAaron GokaslanVolodymyr Kuleshov Department of Computer Science, Cornell University {marriola,yairschiff,hao}@cs.cornell.edu, {akg87,kuleshov}@cornell.edu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-22852
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Encoder-Decoder Diffusion Language Models for Efficient Training and Inference Marianne Arriola, Yair SchiffHao PhungAaron GokaslanVolodymyr Kuleshov Department of Computer Science, Cornell University {marriola,yairschiff,hao}@cs.cornell.edu, {akg87,kuleshov}@cornell.edu Abstract Discrete diffusion models enable parallel token sampling for faster inference than autoregressive approaches. However, prior diffusion models use a decoder-only architecture, which requires sampling algorithms that invoke the full network at every denoising step and incur high computational cost. Our key insight is that discrete diffusion models perform two types of computation: 1) representing clean tokens and 2) denoising corrupted tokens, which enables us to use separate modules for each task. We propose an encoder-decoder architecture to accelerate discrete diffusion inference, which relies on an encoder to represent clean tokens and a lightweight decoder to iteratively refine a noised sequence. We also show that this architecture enables faster training of block diffusion models, which partition sequences into blocks for better quality and are commonly used in diffusion language model inference. We introduce a framework for E fficient E ncoder- D ecoder D iffusion (E2D2), consisting of an architecture with specialized training and sampling algorithms, and we show that E2D2 achieves superior trade-offs between generation quality and inference throughput on summarization, translation, and mathematical reasoning tasks. We provide the code 1 1 1 Code: https://github.com/kuleshov-group/e2d2 , model weights, and blog post on the project page: https://m-arriola.com/e2d2 . \starttitlesymbolfootnotes \stoptitlesymbolfootnotes 1 Introduction Discrete diffusion models ( austin2021structured ) , originally proposed as an extension of Gaussian diffusion ( ho2020denoising ; sohl2015deep ) , have steadily improved in quality on tasks, such as language modeling ( gulrajani2024plaid ; lou2023discrete ; sahoo2024simple ) , music generation ( sun2022score ) , and biological sequence design ( schiff2025discrete_guidance ; wang2025remasking ) . Recent successes include diffusion models at the 7-8B parameter scale, which outperform or match comparably sized autoregressive (AR) models nie2025large ; dream2025 . To generate samples, diffusion models iteratively refine a sequence consisting of both clean and corrupted tokens. We observe that this process consists of: 1) producing useful representations of clean tokens and 2) using these representations as context for denoising corrupted tokens. However, prior diffusion language models jointly perform both tasks within the same decoder-only architecture. As a result, these models must invoke the full network at every denoising step, incurring high computational cost. In this work, we propose an encoder-decoder transformer architecture to separate the computation for representing clean tokens and denoising corrupted tokens. In particular, we use an encoder to represent clean tokens and a lightweight decoder to iteratively refine a noised sequence conditioned on the encoder’s representation. This enables faster inference, as we call the lightweight decoder multiple times to iteratively denoise tokens and invoke the encoder only periodically to update its representations. Our novel framework for E fficient E ncoder- D ecoder D iffusion (E2D2) consists of an encoder-decoder transformer architecture complemented with efficient training and sampling algorithms that enable both faster inference and KV caching support. In addition to faster inference, our proposed encoder-decoder architecture enables faster training of block diffusion models arriola2025block , which partition sequences into blocks to improve generation quality and support KV caching. Block diffusion is widely used for generating with large diffusion language models, even those trained with standard full-sequence diffusion, such as LLaDA nie2025large , Seed Diffusion song2025seed , MMaDA yang2025mmada . However, decoder-only block diffusion incurs higher training costs, with forward passes that are 2\times more expensive than standard diffusion, as both the full clean and noised sequences must be processed in every transformer layer arriola2025block . We derive an efficient training algorithm for encoder-decoder block diffusion that uses the encoder to process the clean sequence and the decoder to process the noised sequence, which halves training costs compared to a decoder-only model of equal size. To demonstrate the efficacy of our encoder-decoder diffusion framework, we train domain-specific diffusion models for summarization, translation, and mathematical reasoning, and show that E2D2 outperforms decoder-only diffusion models in generation speed and quality. Contributions In summary, we make the following contributions: 1) We propose an encoder-decoder architecture for discrete diffusion modeling: the encoder represents clean tokens and the decoder denoises corrupted tokens. 2) We derive efficient diffusion sampling algorithms that accelerate inference by denoising tokens using a lightweight decoder. We also derive an efficient training algorithm for block diffusion modeling using our architecture, which halves training costs compared to a decoder-only architecture. 3) We demonstrate E2D2’s effectiveness by training task-specific models on summarization, translation, and mathematical reasoning. We also map the Pareto frontier of the quality-throughput trade-off by varying the decoder size and demonstrate that E2D2 achieves state-of-the-art performance compared to decoder-only discrete diffusion models. Figure 1: Efficient Encoder-Decoder Diffusion (E2D2) enables faster generation than decoder-only architectures. We accelerate inference by using a lightweight decoder to iteratively denoise for a fixed number of sampling steps, without invoking the encoder. The encoder processes the newly generated tokens periodically to update its representations. 2 Background 2.1 Discrete Diffusion Models Diffusion generative models ( ho2020denoising ; sohl2015deep ; song2019generative ) train a denoising network p_{\theta} to remove noise from latent variables {\mathbf{z}}_{t} , for t\in[0,1] , obtained from a pre-defined corruption process q . This forward process transforms data {\mathbf{x}} by adding noise increasing in t , eventually reaching a limiting distribution {\mathbf{z}}_{1} . Discrete denoising diffusion probabilistic models (D3PM austin2021structured ) take inspiration from diffusion and define corruption processes over discrete one-hot vectors {\mathbf{z}}_{t} . A recent successful class of discrete diffusion models focuses on masking (absorbing state) diffusion ( ou2024your ; sahoo2024simple ; shi2024simplified ) , where the limiting distribution corresponds to {\mathbf{z}}_{1}={\mathbf{m}} , a one-hot representation of a special [MASK] token. In this framework, the corruption process is defined via interpolation between clean data {\mathbf{x}} and the prior {\mathbf{m}} : q({\mathbf{z}}_{t}|{\mathbf{x}})=\text{Cat}({\mathbf{z}}_{t};\alpha_{t}{\mathbf{x}}+(1-\alpha_{t})\bm{{\mathbf{m}}}), where \text{Cat}(\cdot;\cdot) represents a categorical distribution and \alpha_{t}=\alpha(t)\in[0,1] is a noise schedule decreasing in t. For a given corruption process of this form, the true posterior q({\mathbf{z}}_{s}\mid{\mathbf{z}}_{t},{\mathbf{x}}) is known, where {\mathbf{z}}_{s} represents less noisy latents for s<t. The optimal diffusion model p_{\theta} will exactly recover this posterior. However, during inference, we do not have access to {\mathbf{x}} . A common parameterization defines p_{\theta} as p_{\theta}({\mathbf{z}}_{s}|{\mathbf{z}}_{t})=q({\mathbf{z}}_{s}|{\mathbf{z}}_{t},{\mathbf{x}}={\mathbf{x}}_{\theta}({\mathbf{z}}_{t})) where the denoising model {\mathbf{x}}_{\theta}({\mathbf{z}}_{t}) predicts clean tokens {\mathbf{x}} given noisy latents {\mathbf{z}}_{t} . To model sequences of L tokens {\mathbf{x}}^{1:L} , the corruption process q is applied independently to each token in a sequence and the denoising network is assumed to factorize independently as well. This framework can easily be extended to accommodate conditional generation by conditioning on some prompt {\mathbf{x}}^{1:P} in both the forward q({\mathbf{z}}_{t}^{1:L}\mid{\mathbf{x}}^{1:L},{\mathbf{x}}^{1:P}) and reverse p_{\theta}({\mathbf{z}}_{s}^{1:L}\mid{\mathbf{z}}_{t}^{1:L},{\mathbf{x}}^{1:P}) processes. 2.2 Block Diffusion Language Modeling Block diffusion language models (BD3LM arriola2025block ) improve sample quality of standard discrete diffusion and enable efficient KV caching by introducing a parameterization that interpolates between AR and fully-parallel diffusion. In block diffusion, blocks of tokens are modeled autoregressively with diffusion applied within each block. Formally, tokens in {\mathbf{x}}^{1:L} are chunked into B blocks each of length S : B=L/S . Adopting the shorthand from arriola2025block , for each block b\in[1,B], the tokens {\mathbf{x}}^{1+(b-1)\cdot S:b\cdot S} are denoted as {\mathbf{x}}^{b} and all preceding tokens {\mathbf{x}}^{1:(b-1)\cdot S} are denoted as {\mathbf{x}}^{<b} . The likelihood of this model is defined as \log p_{\theta}({\mathbf{x}})=\sum_{b=1}^{B}\log p_{\theta}({\mathbf{x}}^{b}\mid{\mathbf{x}}^{<b}), where each p_{\theta}({\mathbf{x}}^{b}\mid{\mathbf{x}}^{<b}) is modeled