Locally Coherent Parallel Decoding in Diffusion Language Models
Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi · 2026
Read Locally Coherent Parallel Decoding in Diffusion Language Models by Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20216.txt.
Description
Hersche and co-authors make diffusion language models decode in parallel without losing local coherence. These models promise fast, bidirectional generation, which suits code work well. Standard versions sample tokens independently, so joint structure breaks and syntax errors creep in. Their CoDiLA method hands local decoding to a small autoregressive helper model. The helper runs on the diffusion latents and enforces order inside each block. Bidirectional modeling across blocks stays intact. A compact helper model removed most coherence artifacts. Code benchmarks show a new speed and accuracy frontier.
Catalog record
| CREATOR | Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62627 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20216