Continuous Diffusion Models Can Obey Formal Syntax

Jinwoo Kim, Taylor Berg-Kirkpatrick, Loris D’Antoni · 2026

Read Continuous Diffusion Models Can Obey Formal Syntax by Jinwoo Kim, Taylor Berg-Kirkpatrick, Loris D’Antoni (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12468.txt.

Description

Diffusion language models generate text globally rather than word by word, but their continuous latent dynamics make discrete rules hard to enforce. Jinwoo Kim led this work with two co-authors, asking how to guarantee outputs such as a JSON file matching a given schema. Their training-free guidance method steers continuous diffusion models to satisfy formal syntactic constraints expressed as regular expressions.

The method builds an analytic score estimating the chance that a latent state decodes to a valid string accepted by a given regular expression. It then uses its gradient to guide sampling, with no auxiliary classifiers to train. Denoising targets the base model conditioned on syntactic validity. Implemented as Diffinity on the PLAID model and tested on 180 regular-expression constraints, it reaches 68 to 96 percent satisfaction. It incurs only a small perplexity cost, beating autoregressive constrained decoding on both satisfaction and quality.

Catalog record

CREATORJinwoo Kim, Taylor Berg-Kirkpatrick, Loris D’Antoni
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-12468

Browse the full library · About the Tower of Babel