Constrained Code Generation with Discrete Diffusion

Lize Shao, Michael Cardei, Zichen Xie, Ferdinando Fioretto, Wenxi Wang · 2026

Read Constrained Code Generation with Discrete Diffusion by Lize Shao, Michael Cardei, Zichen Xie, Ferdinando Fioretto, Wenxi Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16829.txt.

Introduction

Shao's team presents Constrained Diffusion for Code, a training-free system that sets program constraints during discrete-diffusion code building. These models refine broken token strings side-by-side and expose a global program state at each step, a natural point to step in. CDC adds tools mixing tuning with code analysis to find key regions and adjust it locally. Across benchmarks, CDC lifts quality, safety, and syntax over diffusion and autoregressive rivals, with less fix work and more local edits.

Catalog record

CREATORLize Shao, Michael Cardei, Zichen Xie, Ferdinando Fioretto, Wenxi Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79299
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16829

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Constrained Code Generation with Discrete Diffusion Lize Shao, Michael Cardei, Zichen Xie, Ferdinando Fioretto, Wenxi Wang Abstract Discrete diffusion models are a powerful, emerging paradigm for code generation. They construct programs through iterative refinement of partially corrupted token sequences and enable parallel token refinement. Importantly, this paradigm exposes a global program state at each denoising step, which provides a natural intervention point for enforcing program-level functionality and security constraints, guiding the generation before the final code is committed. Building on this observation, the paper introduces Constrained Diffusion for Code (CDC), a training-free neurosymbolic inference framework that integrates constraint satisfaction directly into the reverse denoising process. CDC augments the base discrete diffusion sampler with constraint-aware denoising operators that combine mathematical optimization with program analysis to identify constraint-relevant regions of the intermediate program state and locally adjust the denoising trajectory, steering generation toward feasible programs while remaining close to the base model. Across code generation benchmarks, CDC consistently improves constraint satisfaction in functional correctness, security, and even syntax, outperforming discrete diffusion and autoregressive baselines with less corrective computation and more localized edits. Figure 1: CDC vs. other diffusion code baselines on HumanEval-X (HE-X), MBPP, CWEval, and LLMSecEval+. 1 Introduction Code generation has become an increasingly important capability in modern generative modeling systems, with notable applications in program synthesis, developer assistance, and agentic software workflows [ 16 , 33 , 31 ] . In this setting, discrete diffusion language models are emerging as a highly promising paradigm, constructing programs through iterative refinement of partially corrupted token sequences [ 10 , 29 ] . However, the objective used to train generative models does not by itself capture the requirements that make generated code correct or secure. These models are trained to match a target distribution and produce high-fidelity samples, but a program may be highly plausible under the base model while still producing incorrect behavior, or containing an exploitable vulnerability. In practice, generated code must satisfy explicit program-level constraints, including syntactic validity, and more importantly, functional correctness, and security properties. Since programs are executable objects, violations of these constraints can lead directly to runtime errors, incorrect outputs, or security failures [ 21 , 3 ] . Incorporating such constraints is therefore critical for both code quality and safety. These requirements expose a central challenge for code generation: Program-level constraints often depend on global structure or semantic properties that are not visible from any single local token decision . More broadly, common guidance mechanisms such as prompting, reranking, rejection sampling, or post-hoc repair typically intervene only after candidate programs have already been generated [ 27 , 8 ] . In code generation, this is particularly limiting because the underlying search space is highly combinatorial, while the subset of programs satisfying semantic or security constraints may constitute only a small feasible region under the model distribution. Thus, post-hoc approaches can be both computationally inefficient and unreliable, especially when constraint satisfaction requires steering generation away from the model’s dominant unconstrained modes and toward rarer but feasible solutions. In contrast, as this paper shows, discrete diffusion models offer a different opportunity: their approach to refine the entire sequence through intermediate denoising states exposes a global program representation throughout sampling. This structure creates natural intervention points for integrating constraints directly into the generated code during the denoising process. Building on this opportunity, this paper introduces Constrained Diffusion for Code (CDC), a training-free neurosymbolic framework that integrates program-level constraints (i.e., functionality and security) directly into the reverse denoising process of discrete diffusion models. CDC treats diffusion sampling as an editable program trajectory: at each denoising step, the model proposes a full clean-state distribution over the program; CDC decodes this proposal, evaluates the resulting candidate against program-level constraints, localizes feedback to relevant regions, and applies a constraint-aware correction before sampling continues. Each correction follows a localized constrained-sampling principle: reduce constraint violation while remaining close to the base denoiser, revising constraint-relevant regions without unnecessarily perturbing the rest of the program. The framework supports both soft and hard feedback. For functional correctness, CDC instantiates this principle as GradGuide, which uses soft surrogate execution signals to localize likely errors and guide targeted denoising updates. For security, CDC instantiates it as MDFI, which uses static program analysis to identify vulnerable regions and correct them through localized remasking, insertion, and feedback injection. In this way, CDC exploits the global, editable states exposed by diffusion sampling to perform focused constraint correction during generation, rather than relying on post-hoc filtering or whole-program regeneration after failure. This work makes three main contributions: 1. Constraint-aware localization. It introduces a general localization mechanism for constrained diffusion code generation, mapping program-level feedback from surrogates or static analyzers to the token regions that should be revised during denoising. 2. Constraint-aware correction. It proposes training-free correction operators that revise these localized regions during reverse diffusion while staying close to the base denoiser. The operators support differentiable surrogate guidance through gradient and KL-anchored updates, and non-differentiable symbolic analyzer feedback through targeted remasking, mask insertion, and feedback injection. 3. Empirical evaluation. It provides empirical evidence that denoising-time constraint integration improves constrained code generation across functional-correctness and security-oriented benchmarks. As summarized in Figure 1 , CDC improves functionality from 34.1\% to 65.2\% on HumanEval-X C++ and from 27.7\% to 59.2\% on MBPP-C++ over Dream-Coder 7B, while MDFI improves CWEval joint functionality-security success from 12.04\% to 34.26\% over state of the art diffusion with far fewer edited tokens. Although CDC targets functionality and security, it also improves syntactic correctness from 67.1\% to 79.2\% on HumanEval-X C++ and from 41.1\% to 72.0\% on MBPP-C++. 2 Related Work Discrete diffusion language models generate sequences through iterative denoising rather than left-to-right decoding, exposing partially generated global states throughout sampling. Recent code-oriented diffusion models including DiffuCoder [ 10 ] and Dream-Coder 7B [ 29 ] show that this paradigm can achieve competitive code-generation performance while supporting flexible, non-autoregressive refinement. CDC builds on this property: rather than using diffusion only as an alternative decoder, it treats intermediate denoising states as editable program representations where constraints can be evaluated, localized, and incorporated before generation is complete. Controllable code generation has been studied primarily in autoregressive models through constrained decoding, reranking, rejection sampling, verification, and iterative repair [ 14 , 19 , 8 , 18 , 4 ] . These methods can improve syntactic validity, type consistency, functional correctness, or security, but they typically operate either at the next-token level or after a complete candidate has been generated. This is limiting for program-level constraints: semantic and security violations often depend on global dataflow or execution behavior, and repairing them post hoc may require regenerating large suffixes or entire programs. CDC instead injects feedback during denoising, when the full program state is visible and localized regions can still be revised. Recent work has explored controllable generation for discrete diffusion models through classifier guidance, classifier-free guidance, hidden-state optimization, and projection-based constrained sampling [ 26 , 11 , 6 , 9 ] . Closest to our setting, Mündler et al. [ 17 ] enforce context-free grammar constraints during diffusion-LLM decoding, improving syntactic correctness for code. CDC targets a broader class of program-level constraints, including functional and security requirements that cannot generally be expressed as grammar membership or local

Read the complete text