GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Models

Lipeng Wan, Junjie Ma, Jianhui Gu, Zeyang Liu, Xuyang Lu, Xuguang Lan · 2026

Read GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Models by Lipeng Wan, Junjie Ma, Jianhui Gu, Zeyang Liu, Xuyang Lu, Xuguang Lan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26675.txt.

Description

Wan and co-authors present GeoBlock, a decoding method that sets block size in diffusion language models from attention geometry. Block diffusion refines many tokens at once, but safe block size depends on how tokens depend on each other. Fixed rules and local confidence hints miss that structure. GeoBlock reads cross-token dependency patterns to find stable regions for joint refinement. Tightly ordered zones get sequential updates, while cohesive zones refine in parallel. The method needs no extra training and plugs into existing block diffusion models.

Catalog record

CREATORLipeng Wan, Junjie Ma, Jianhui Gu, Zeyang Liu, Xuyang Lu, Xuguang Lan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54274
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26675

Browse the full library · About the Tower of Babel