Locally Coherent Parallel Decoding in Diffusion Language Models

Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi · 2026

Read Locally Coherent Parallel Decoding in Diffusion Language Models by Michael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20216.txt.

Description

Hersche and co-authors make diffusion language models decode in parallel without losing local coherence. These models promise fast, bidirectional generation, which suits code work well. Standard versions sample tokens independently, so joint structure breaks and syntax errors creep in. Their CoDiLA method hands local decoding to a small autoregressive helper model. The helper runs on the diffusion latents and enforces order inside each block. Bidirectional modeling across blocks stays intact. A compact helper model removed most coherence artifacts. Code benchmarks show a new speed and accuracy frontier.

Catalog record

CREATORMichael Hersche, Nicolas Menet, Ronan Tanios, Abbas Rahimi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62627
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20216

Browse the full library · About the Tower of Babel