EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients
Sebastian U. Stich, Author Name, He-Yen Hsieh, Hong Wang, H.T. Kung · 2025
Read EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients by Sebastian U. Stich, Author Name, He-Yen Hsieh, Hong Wang, H.T. Kung (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00670.txt.
Introduction
Diffusion language models refine their answers through many denoising steps, but the answer often stabilizes early. Sebastian Stich and co-authors propose EDIT in 2025, an early termination method for diffusion model inference. The name stands for early diffusion inference termination.
EDIT stops denoising adaptively once the reasoning is stable enough relative to training-time behavior, judged from the dynamics of training gradients. Stopping early saves computation on every query without retraining the model. As diffusion models enter production, such inference-time savings compound into major cost and energy reductions. Diffusion language models are a newer rival to the autoregressive models most people know. Stability relative to training behavior gives the stopping rule its reference point. For firms that serve such models, the gain is plain cash and watts. Each cut step is time and power saved, on each call, all day, at full scale.
Catalog record
| CREATOR | Sebastian U. Stich, Author Name, He-Yen Hsieh, Hong Wang, H.T. Kung |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00670
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients Sebastian U. Stich, Author Name, He-Yen Hsieh, Hong Wang, H.T. Kung Abstract Diffusion-based large language models (dLLMs) refine token generations through iterative denoising, but answers often stabilize before all steps complete. We propose EDIT (Early Diffusion Inference Termination), an inference-time criterion that adaptively stops denoising once sufficient reasoning stability relative to training-time reasoning is detected. EDIT monitors the alignment between token activations and a reasoning map derived from AdamW-aggregated LoRA updates captured during supervised fine-tuning (SFT). During training, optimization dynamics generate rich metadata about parameter importance that in prior methods is typically discarded upon model release. We preserve this information as a compact representation of learned reasoning pathways. During inference, alignment scores are converted to a distribution over the tokens already unmasked at the current denoising step, and convergence is detected when KL divergence between consecutive steps falls below a threshold on the matched unmasked (visible) tokens. Across reasoning benchmarks, EDIT reduces diffusion steps by 11.8% to 68.3% while preserving or improving accuracy in most settings, with approximately 0.02% storage overhead (about 1.5-2 MB for all QKV modules across 32 blocks in an 8 GB model). By utilizing training-gradient dynamics, our work opens a new research direction for reducing dLLM inference time and cost. Code Repository \optauthor CISPA, Germany \optauthor Harvard University Intel Corporation Harvard University 1 Introduction Modern language model deployment follows a wasteful paradigm: during training, optimization dynamics on gradients generate rich information about which parameters are critical for specific capabilities, yet this metadata is routinely discarded once training completes. We challenge this practice by demonstrating that training-time optimization trajectories contain valuable signals that can guide intelligent inference-time decisions, specifically enabling adaptive early inference termination for diffusion language models. Diffusion-based language models (dLLMs) 8 ; 11 ; 2 represent a promising alternative to autoregressive generation of tokens, employing iterative denoising processes that progressively refine outputs. However, their inference remains computationally expensive due to the fixed number of denoising steps, even when high-quality outputs emerge early in the process. Current approaches to this challenge operate without knowledge of which model parameters drove learning during training, essentially making uninformed decisions during inference termination optimization. We introduce EDIT (Early Diffusion Inference Termination), a method that utilizes training optimization metadata to identify opportunities for early inference termination. Our key insight is that the AdamW optimizer’s 6 moment estimates during fine-tuning training encode which parameters consistently receive strong, directionally-aligned updates when learning reasoning tasks. The effectiveness of EDIT is validated from the perspective of gradient convergence during inference, as shown in Figure 1 . These patterns, which we term AdamW evolution , represent a map of the model’s learned reasoning pathways. Rather than discarding this information when training is complete, we store it as compact metadata (requiring minimal additional storage) and use it to guide inference termination decisions. Our Approach: Utilizing Training Metadata for Guiding Inference Termination. During supervised fine-tuning (SFT) on reasoning tasks, certain LoRA parameters receive consistent gradient signals across training steps, indicating their importance for encoding reasoning patterns. The AdamW optimizer naturally tracks this through its moment estimates, with the first moment capturing gradient direction and the second moment reflecting gradient stability. Parameters with large, stable updates become critical components of the learned reasoning pathways. Traditional inference deployment discards this valuable information. EDIT preserves it by saving aggregated AdamW updates across training steps, creating a fingerprint of which parameters matter most for reasoning. At inference time, we compare current token activations against these preserved patterns using cosine similarity, assessing whether the model’s current state aligns with its learned reasoning pathways. When this alignment stabilizes—indicating the model has reached its learned reasoning configuration—we can confidently terminate the diffusion inference process early. Instead of relying on inference-time heuristics such as confidence or output stability, EDIT leverages training-time knowledge to terminate once key reasoning components are engaged. Contributions. (1) We establish a new paradigm for early inference termination that leverages training metadata which is usually discarded in prior methods. (This approach opens future research directions in not only early inference termination, but also informing dynamic compute allocation, quality prediction, and other inference-time optimizations.) (2) We provide a practical instantiation through EDIT, demonstrating that AdamW evolution patterns can reliably indicate when diffusion models have completed their core reasoning. Our method requires no architectural changes, adds minimal storage overhead, and integrates seamlessly with existing diffusion language models. (3) We validate our approach across multiple reasoning benchmarks, showing inference speedups of 11.8% to 68.3% while maintaining or improving accuracy in most settings. These gains come purely from utilizing training information that already existed but was previously thrown away, highlighting the inefficiency of current practices. 2 Preserving and Utilizing Training Metadata We detail how EDIT captures optimization dynamics during training and leverages them for intelligent early termination during inference. Our approach consists of two phases: metadata extraction during fine-tuning (Section 2.1 ) and metadata-guided termination during inference (Section 2.2 ). Utilizing Training Metadata for Guiding Inference Termination. During supervised fine-tuning (SFT), some parameters receive strong, stable updates that encode core reasoning patterns, while others show weak or oscillating updates and contribute less. We track this distinction through the AdamW update history—what we call the AdamW evolution —which forms a map of reasoning-relevant parameters. At inference, activations are compared against this map; when alignment with the learned reasoning pathways is reached, early termination is enabled with confidence. 2.1 Capturing AdamW Evolution During Training We consider a pre-trained base model with LoRA (Low-Rank Adaptation) 4 modules inserted into the Query ( Q ), Key ( K ), and Value ( V ) projections of each Transformer block. During SFT on reasoning tasks, only these LoRA parameters are updated. Each LoRA module consists of matrices (A,B) where A\in\mathbb{R}^{r\times d_{\text{in}}} and B\in\mathbb{R}^{d_{\text{out}}\times r} implement a low-rank update to the corresponding projection. Notation note: We use \mathcal{L}_{k} for the loss at training step k , and L for block length in the diffusion process. For the LoRA-B matrix B\in\mathbb{R}^{d_{out}\times r} (where d_{out} is the output dimension and r is the rank), the AdamW optimizer maintains first and second moment estimates at each training step k : M_{k,B}=\beta_{1}M_{k-1,B}+(1-\beta_{1})G_{k,B},\quad V_{k,B}=\beta_{2}V_{k-1,B}+(1-\beta_{2})G_{k,B}^{\odot 2}, (1) where G_{k,B}=\nabla_{B}\mathcal{L}_{k} is the gradient tensor, \beta_{1},\beta_{2}\in[0,1) are decay rates, and \odot denotes element-wise operations. The element-wise update magnitude at step k is: U_{k,B}=\frac{M_{k,B}}{\sqrt{V_{k,B}}+\epsilon}, (2) where \epsilon ensures numerical stability and division is element-wise. To create a stable representation of the parameter importance patterns, we define the AdamW evolution tensor as the average over all \mathcal{K} fine-tuning steps: \bar{U}_{B}=\frac{1}{\mathcal{K}}\sum_{k=1}^{\mathcal{K}}U_{k,B}\in\mathbb{R}^{d_{out}\times r}. (3) This tensor captures which elements of the LoRA-B matrix consistently received strong, directional updates during training. To enable comparison with token activations \mathbf{f}_{s}\in\mathbb{R}^{d_{out}} , we reduce \bar{U}_{B} to a feature-aligned vector \mathbf{u}\in\mathbb{R}^{d_{out}} using row-wise energy: \mathbf{u}[p]=\left\|\bar{U}_{B}[p,:]\right\|_{2}=\sqrt{\sum_{j=1}^{r}\bar{U}_{B}[p,j]^{2}},\quad p=1,\ldots,d_{out}. (4) This reduction preserves the update magnitude each output dimension receives across low-rank components, forming a parameter-importance signature aligned with the feature space. 2.2 Metadata-Guided Early Termination During Inference At inference time, we leverage the preserved AdamW evolution to determine when the diffusion process can safely terminate. Our approach operates on block-level diffusion 8 ; 11 ; 2 , where the sequence is divided into blocks of length L , and