Diffusion Language Models Are Natively Length-Aware

Vittorio Rossi, Giacomo Cirò, Davide Beltrame, Luca Gandolfi, Paul Röttger, Dirk Hovy · 2026

Read Diffusion Language Models Are Natively Length-Aware by Vittorio Rossi, Giacomo Cirò, Davide Beltrame, Luca Gandolfi, Paul Röttger, Dirk Hovy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06123.txt.

Introduction

Rossi and his team note that diffusion language models use a fixed set of denoising steps on a fixed window no matter how long the reply must be. Short replies then waste compute. They say the prompt's latent representation already hints at the length that is needed. They test a zero-shot method that crops the window before generation starts. Four benchmarks cover reasoning, code, instruction following, and question answering, and show sharp compute savings with little cost to results and gains on two tasks.

Catalog record

CREATORVittorio Rossi, Giacomo Cirò, Davide Beltrame, Luca Gandolfi, Paul Röttger, Dirk Hovy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43804
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06123

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Diffusion Language Models Are Natively Length-Aware Vittorio Rossi, Giacomo Cirò, Davide Beltrame, Luca Gandolfi, Paul Röttger, Dirk Hovy Abstract Unlike autoregressive language models, which terminate variable-length generation upon predicting an End-of-Sequence ( EoS ) token, Diffusion Language Models (DLMs) operate over a fixed maximum-length context window for a predetermined number of denoising steps. However, this process is independent of the required response length, resulting in computational waste for the majority of short responses common in reasoning and chat tasks. To address this problem, we conjecture that the latent prompt representation contains sufficient information to estimate the required output length. We provide empirical evidence for this phenomenon and propose a zero-shot mechanism to dynamically crop the context window before generation begins, leading to fewer diffusion steps and substantial computational savings. We evaluate our approach on four benchmarks with diverse tasks— GSM8K (reasoning), HumanEval (code generation), IfEval (instruction following), and LongFormQA (question answering)—revealing massive efficiency gains at minimal performance impact. We report significant reductions in FLOPs across all tasks, with no statistically significant performance degradation, and significant performance improvements in 2 out of 4 tasks. † † affiliationnotice: Equal contribution 1 Introduction Language generation with Large Language Models (LLMs) has been dominated by autoregressive models, which generate text sequentially, predicting one token at a time ( Zou et al. 2023 ) . While proven successful, this sequential mechanism inherently limits inference speed and brings various disadvantages, such as the inability to perform global refinements during the generation process and error propagation due to “wrong” early tokens. Recently, Diffusion Language Models ( Austin et al. 2021 ; Lou et al. 2024 , DLMs,) have emerged as a promising alternative, offering the potential for accelerated, non-autoregressive generation through iterative denoising Figure 1 : Predicted Length Distributions. Our SmartCrop ( \tau=0.9 ) method successfully predicts task-specific output lengths across four benchmark datasets. The abrupt truncations observed in certain distributions correspond to context length constraints (refer to Section 4 for details). DLMs operate by progressively unmasking tokens on a fixed-length canvas. This process is initialized with a prompt, while the remainder of the maximum context window is filled with placeholder mask tokens. In each denoising step, the model predicts logits for the entire masked sequence and unmasks a subset of tokens, typically those with the highest confidence. Unmasked tokens are kept and the process repeated. While this allows for flexible, parallel sequence generation, the requirement of a fixed canvas length remains a significant bottleneck, as the dimensions must be defined a priori based on heuristics or domain knowledge. To support variable-length outputs, current approaches often rely on padding with special End-of-Sequence ( EoS ) tokens to prevent unmasking beyond a certain point ( Nie et al. 2025 ) . However, this introduces substantial computational waste: the model must still process the entire context window during every forward pass, regardless of the actual output length. In this work, we conjecture that DLMs implicitly encode the required output length within their latent representation of the prompt. In other words, DLMs encode an expectation about how many answer tokens are needed, depending on the task or question they are prompted with. While these models are explicitly trained to predict EoS tokens at appropriate positions, we show that this length signal can be extracted and exploited before generation begins, rather than discovered iteratively during denoising. Building on this, we introduce SmartCrop , a model-native method to optimize DLM inference. We transform EoS logits into a cumulative “inverse survival” probability across positions, modeling the likelihood of the response terminating at any given point on the canvas. By identifying the first position where this probability exceeds a specified threshold (e.g., \tau=0.9 ), we dynamically crop the canvas before generation begins. We then run the standard denoising schedule on the new, shorter canvas. We evaluate our approach using LLaDA ( Nie et al. 2025 ) , a state-of-the-art 8 billion parameters DLM trained to handle variable-length outputs via EoS padding, and test it on four benchmarks with a diverse range of tasks: GSM8K ( Cobbe et al. 2021 , reasoning,) , HumanEval ( Chen et al. 2021 , code generation,) , IfEval ( Zhou et al. 2023 , instruction following,) , and LongFormQA ( Köksal et al. 2024 , question answering) . Our results demonstrate that SmartCrop drastically reduces computational costs, measured in FLOPs, without statistically significant performance degradation across any benchmark but significant performance improvements in 2 of the 4 evaluation suites. These findings suggest that DLMs trained with the EoS paradigm are inherently “length-aware”, and that SmartCrop effectively leverages this previously unobserved behavior to bridge the efficiency gap between fixed canvas diffusion and variable-length generation. We make our code and results publicly available on GitHub. 1 1 1 https://github.com/VittorioRossi/smartcrop 2 Related Work DLMS typically decode by denoising a fixed-length canvas of length L_{c} for T steps, yielding an inference cost that scales roughly with L_{c}\times T even when the desired output is short. We therefore situate SmartCrop along two orthogonal axes explored by prior work: improving the trajectory (e.g., reducing steps T or improving sampling quality) versus improving the canvas allocation (i.e., adapting L_{c} to the prompt). We review (i) foundational diffusion methods for text, (ii) scaling diffusion to the LLM regime where the padding tax becomes practically significant, (iii) sampling-efficiency methods that reduce per-sample cost but keep L_{c} fixed, (iv) diffusion-specific variable-length decoding that adapts length during or via retraining, and (v) length prediction ideas in other non-autoregressive LMs that motivate extracting termination signals from internal states. Diffusion Models for Text. Diffusion models originate from non-equilibrium thermodynamics and were first developed for continuous data ( Sohl-Dickstein et al. 2015 ; Ho et al. 2020 ) . Several works adapt diffusion to discrete text. Structured Denoising Diffusion Models in Discrete State-Spaces (D3PM) define masked corruption processes for tokens and show that discrete diffusion can serve as a general generative framework ( Austin et al. 2021 ) . Diffusion-LM applies diffusion in the embedding space and improves controllable text generation ( Li et al. 2022 ) . DiffuSeq extends this idea to sequence-to-sequence tasks ( Gong et al. 2023 ) , and DiffusionBERT integrates diffusion training with pre-trained masked language models ( He et al. 2023 ) . Lou et al. 2024 model discrete diffusion by estimating ratios of the data distribution and obtain strong likelihood estimates. Zou et al. 2023 survey this growing line of work under the umbrella term diffusion language models. These works primarily address modeling and generation quality ; they do not focus on inference-time waste from denoising large masked regions when the correct output is short, which is the bottleneck we target. Scaling. Recent work scales DLMs to the LLM regime (DLLMs). Gong et al. 2025 adapt large autoregressive transformers into diffusion models and study how performance scales with model size. Liang et al. 2025 derive scaling laws for diffusion transformers and analyze compute–performance trade-offs. Nie et al. 2025 train a DLLM from scratch and shows that it can match autoregressive LLMs on instruction-following and reasoning benchmarks. These works demonstrate that DLLMs are competitive with autoregressive models, but they still rely on fixed-length diffusion canvases at inference time. Our method is designed to be orthogonal to scaling,

Read the complete text