Learned Relay Representations for Forward-Thinking Discrete Diffusion Models

Benjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil · 2026

Read Learned Relay Representations for Forward-Thinking Discrete Diffusion Models by Benjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22967.txt.

Description

Rozonoyer and six coauthors propose Learned Relay Representations, a way for masked diffusion models to carry latent information forward instead of resetting between denoising rounds. A differentiable per-token channel passes information across forward passes, trained by truncated backpropagation through time. Tested first on a Sudoku planning task, then scaled to Fast-dLLM v2, it beat standard supervised fine-tuning on coding tasks while cutting inference latency by up to 32 percent.

Catalog record

CREATORBenjamin Rozonoyer 1 Jacopo Minniti 2 Dhruvesh Patel 1 Neil Band 3 Avishek Joey Bose 4,5 Tim G. J. Rudner 2,6 Andrew McCallum 1 1 University of Massachusetts Amherst 2 University of Toronto 3 Stanford University 4 Imperial College London 5 Mila 6 Vijil
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51898
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22967

Browse the full library · About the Tower of Babel