Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
Benjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil · 2026
Read Learned Relay Representations for Forward-Thinking Discrete Diffusion Models by Benjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22967.txt.
Description
Rozonoyer and six coauthors propose Learned Relay Representations, a way for masked diffusion models to carry latent information forward instead of resetting between denoising rounds. A differentiable per-token channel passes information across forward passes, trained by truncated backpropagation through time. Tested first on a Sudoku planning task, then scaled to Fast-dLLM v2, it beat standard supervised fine-tuning on coding tasks while cutting inference latency by up to 32 percent.
Catalog record
| CREATOR | Benjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51898 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22967