Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models
Hanhan ZhouAWS AI LabsSanta Clara, CA 95054 [email protected], Shamik RoyAWS AI LabsSanta Clara, CA 95054 [email protected] Rashmi GangadharaiahAWS AI LabsSanta Clara, CA 95054 [email protected] · 2026
Read Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models by Hanhan ZhouAWS AI LabsSanta Clara, CA 95054 [email protected], Shamik RoyAWS AI LabsSanta Clara, CA 95054 [email protected] Rashmi GangadharaiahAWS AI LabsSanta Clara, CA 95054 [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10971.txt.
Description
Hanhan Zhou, Shamik Roy, and Rashmi Gangadharaiah show how to steer discrete diffusion language models without breaking them. Diffusion models clean up all positions in parallel, but control methods borrowed from autoregressive models act evenly at every step, degrading quality, with damage growing across jointly steered traits.
Sparse autoencoders trained on four diffusion models reveal that traits lock in on distinct timetables: topic settles within the first two percent of denoising, while sentiment emerges gradually over twenty percent. Even steering therefore wastes effort on steps where a trait has solidified or not yet emerged. An adaptive scheduler focuses action on steps where the trait actively forms, with a closed-form cost-control trade-off governed by timetable spread. Controlled-generation builders get steering that respects each trait's timing.
Catalog record
| CREATOR | Hanhan ZhouAWS AI LabsSanta Clara, CA 95054 [email protected], Shamik RoyAWS AI LabsSanta Clara, CA 95054 [email protected] Rashmi GangadharaiahAWS AI LabsSanta Clara, CA 95054 [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 170609 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10971