Learnability-Informed Fine-Tuning of Diffusion Language Models

Shubham Parashar, Atharv Chagi, Jacob Helwig, Lakshmi Jotsna, Sushil Vemuri, James Caverlee, Dileep Kalathil, Shuiwang Ji · 2026

Read Learnability-Informed Fine-Tuning of Diffusion Language Models by Shubham Parashar, Atharv Chagi, Jacob Helwig, Lakshmi Jotsna, Sushil Vemuri, James Caverlee, Dileep Kalathil, Shuiwang Ji (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22939.txt.

Introduction

Parashar and seven coauthors propose LIFT, a fine-tuning method for diffusion language models that respects what tokens are learnable, and when. Standard supervised fine-tuning ignores learnability: rare tokens resist learning when most input is masked, while common tokens teach little when most input is visible. LIFT instead learns easy tokens under heavy masking and hard tokens with more context. It beat existing fine-tuning baselines on six reasoning benchmarks, with up to threefold relative gains on AIME tests.

Catalog record

CREATORShubham Parashar, Atharv Chagi, Jacob Helwig, Lakshmi Jotsna, Sushil Vemuri, James Caverlee, Dileep Kalathil, Shuiwang Ji
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48343
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22939

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learnability-Informed Fine-Tuning of Diffusion Language Models Shubham Parashar, Atharv Chagi, Jacob Helwig, Lakshmi Jotsna, Sushil Vemuri, James Caverlee, Dileep Kalathil, Shuiwang Ji Abstract We aim to improve the reasoning capabilities of diffusion language models (DLMs). While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces challenges and can even hurt performance, though the underlying causes remain understudied. Our analysis reveals that vanilla SFT overlooks learnability , namely what and when tokens are learned. Specifically, rare tokens are difficult to learn when most of the input is masked, whereas it is straightforward and thus of little value to learn common tokens when most of the input is unmasked. Motivated by our analysis, we propose LIFT , an efficient SFT-based post-training algorithm for DLMs. LIFT learns easy tokens when most of the input is masked and hard tokens when more context is available, thus aligning the training with the information available at different diffusion time steps. Our results show that LIFT outperforms existing SFT baselines across six reasoning benchmarks, achieving up to a 3 × \times relative gain on AIME’24 and AIME’25. Our code is publicly available at https://github.com/divelab/LIFT . #1# 1 Introduction Diffusion models have shown impressive performance in image ( 30 ; 24 ) video ( 16 ) generation applications. Recently, diffusion models have been successfully applied to textual data, leading to the recent surge of interest in Diffusion Language Models (DLMs) ( 2 ; 29 ) . A central promise of DLMs over autoregressive language models (ARLMs) is their ability to generate multiple tokens in parallel per model call, yielding substantial gains in inference throughput ( 18 ; 35 ) . Several open-weight DLMs, such as LLaDA ( 25 ) and Dream ( 38 ) , are now available, and they largely match the performance of similarly-sized ARLM counterparts. Figure 1 : Performance on AIME benchmarks . Pass@16 accuracy comparison on AIME’24 and AIME’25 for LLaDA-8B-Instruct, vanilla SFT, and LIFT. LIFT achieves substantial relative improvements over vanilla SFT on both challenging mathematical reasoning datasets, demonstrating the effectiveness of learnability-informed training. (a) Frequency vs. confidence. (b) Token-level confidence across timesteps. Figure 2 : Token Analysis with LLaDA. Using data collated from 4 post-training corpora ( 23 ; 5 ; 26 ; 31 ) , we analyze 0.5B masked tokens and aggregate token-level confidence and frequencies. (a) We bin tokens by log-scaled frequency and plot the mean model confidence against the average frequency. The marginalized plot ( top ) reveals that rare tokens have lower confidence on average, demonstrating that certain tokens are more difficult to predict ( what dimension). We perform a more nuanced analysis by breaking down the marginalized plot by diffusion timestep t ( bottom ), revealing an interaction between the what and when dimensions. Specifically, we observe a t -induced bias, when at large t many of the model inputs are masked, low frequency tokens become disproportionately difficult to predict, suggesting that the information content of heavily masked inputs arising later in the forward diffusion process as diffusion time t\to 1^{+} is insufficient to learn certain tokens reliably. Conversely, as t\to 0^{-} , less frequent tokens become more learnable, whereas predicting frequent tokens become trivial. (b) We sample representative high and low -frequency tokens, visualizing their (average) confidence across diffusion time. Rare tokens increasingly suffer as t\to 1^{+} , and experience more extreme drops in confidence than high frequency tokens. Following the success of post-training of ARLMs to improve reasoning, recent works have explored post-training of DLMs using supervised or instruction finetuning (SFT) ( 38 ; 25 ) and reinforcement learning (RL) ( 41 ) . However, in contrast to ARLMs, RL in DLMs is substantially more challenging both technically and algorithmically due to intractable sequence-level likelihoods, and most works on RL for DLMs propose approximations to overcome this challenge ( 41 ; 20 ; 33 ) . SFT has been studied less thoroughly, and to date no work has systematically examined the challenges involved in applying SFT to DLMs. Recent results suggest that SFT can in fact degrade model performance relative to pretraining ( 38 ) . This motivates the central question of our work, which we decompose into two sub-questions: (i) what are the major factors that influence SFT post-training of DLMs , and (ii) how can we design an SFT algorithm that accounts for them to effectively post-train DLMs? As our first contribution, we address (i) by analyzing SFT in DLMs and characterizing its failure cases. Specifically, we conduct an extensive analysis in Fig. 2(a) spanning 0.5B tokens collated from four popular post-training reasoning datasets ( 23 ; 5 ; 31 ; 26 ) . Across several pre-trained DLMs ( 38 ; 25 ) , our findings reveal two crucial considerations whose interplay govern SFT dynamics; those are, what tokens are learned, and when tokens are learned in the diffusion process. Our findings show that rare tokens in the corpus are more difficult to predict than frequent tokens ( what ). Additionally, rare tokens become more learnable when more context is available, corresponding to early forward diffusion times. However, at later forward diffusion times, the reduced information in the input disproportionately lowers the model’s confidence on rare tokens, in some cases making them effectively unlearnable ( when ). These findings suggest that as forward diffusion time t\to 1^{+} , rare tokens often become unlearnable, making it more effective to focus compute on frequent tokens. In contrast, as forward diffusion time t\to 0^{-} , frequent tokens are easy to predict, while rare tokens become more learnable. While prior works have proposed heuristics partially adhering to these guidelines by considering either the what or when dimensions in isolation ( 38 ; 36 ) , our study is the first to systematically analyze their combined effect during supervised fine-tuning. We show that modeling the interaction between token difficulty and diffusion time is critical for improving training. As our second contribution, motivated by these insights, we propose and develop LIFT , the first post-training approach to target the interaction between what and when during DLM training. LIFT trains the model on masked tokens that are most appropriate to learn at each diffusion time given the available context. We obtain state-of-the-art results among various SFT training frameworks across two DLM base models on four reasoning benchmarks. We also evaluate LIFT on the challenging AIME-24 ( 1 ) and AIME-25 ( 22 ) , where it achieves up to a 3\times improvement over SFT baselines. Remarkably, LIFT attains performance close to the RLVR baseline d1 ( 41 ) while using roughly 500\times fewer GPU hours, establishing a new Pareto frontier for DLM post-training. Figure 3 : Learnability-Informed Fine-Tuning (LIFT). LIFT increases learnability by using model confidence and diffusion time to construct a learnability-informed mask so as to train on the highest utility tokens at each point in the diffusion process. Utility is estimated as a function of model confidence and diffusion time. In the first stage, a mask is sampled with rate t+\rho and used to estimate model confidences p_{\theta}(x_{0}\mid x_{t+\rho}) over all masked positions. LIFT then selects a subset of masked tokens from x_{t+\rho} to supervise based on model confidences and diffusion time. Depending

Read the complete text