Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints
Omer Rochman-Sharabi, Gilles Louppe · 2026
Read Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints by Omer Rochman-Sharabi, Gilles Louppe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21033.txt.
Introduction
Omer Rochman-Sharabi and Gilles Louppe teach diffusion models to obey hard rules. Their 2026 paper presents Predict-Project-Renoise, a sampling framework. Diffusion-based neural emulators show promise for scientific work. Vanilla models, though, cannot guarantee physical accuracy. Nor can they promise that constraints will hold. A generated weather field or flow must respect laws and observations. The framework enforces hard constraints at generation time.
The name sketches the loop. The model predicts a sample, projects it onto the constraint set, then renoises and repeats. Projection forces each draw to satisfy the rules exactly. Physical laws and observational consistency are the target constraints. That guarantee matters when outputs feed real decisions. The work serves scientists who need generative speed without lawless samples.
Catalog record
| CREATOR | Omer Rochman-Sharabi, Gilles Louppe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21033
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints Omer Rochman-Sharabi, Gilles Louppe Abstract Neural emulators based on diffusion models show promise for scientific applications, but vanilla models cannot guarantee physical accuracy or constraint satisfaction. We address this by introducing a constrained sampling framework that enforces hard constraints, such as physical laws or observational consistency, at generation time. Our approach defines a constrained forward process that diffuses only over the feasible set of constraint-satisfying samples, inducing constrained marginal distributions. To reverse this, we propose Predict-Project-Renoise (PPR), an iterative algorithm that samples from the constrained marginals by alternating between denoising predictions, projecting onto the feasible set, and renoising. Experiments on 2D distributions, PDEs, and global weather forecasting demonstrate that PPR reduces constraint violations by over an order of magnitude while improving sample consistency and better matching the true constrained distribution compared to baselines. 1 Introduction Diffusion models ( 37 ; 15 ; 41 ) have emerged as the state-of-the-art framework for deep generative modeling, showing strong performance in approximating high-dimensional data across modalities. These models learn to reverse a gradual corruption process, transporting a simple prior distribution to a complex data distribution p . However, in many scientific and engineering domains, such as medical imaging, fluid dynamics, or climate modeling, sampling p is insufficient. Instead, the samples must satisfy hard constraints, such as exact consistency with observations or governing equations, zero divergence in incompressible fluids, conservation laws, boundary and initial conditions, or geometric constraints. Although training-based approaches can incorporate constraints ( 47 ; 20 ; 28 ) , they often lack flexibility, requiring retraining for different constraints, and enforce the constraints only softly. For this reason, some recent efforts focus on training-free constrained sampling , where a pre-trained unconditional model is guided to satisfy constraints during inference. Current methods typically rely on treating constraints as observations ( 9 ) or implement projection-based corrections ( 6 ; 31 ) . When treating the constraint as an observation, Bayesian decomposition can be used to express the posterior score as a prior score plus a likelihood score. While effective for noisy inverse problems ( 7 ) , especially in the context of images, these methods are ill-suited for hard constraints: the implicit likelihood approaches a Dirac delta, causing gradients to become unstable and informative only in the immediate vicinity of the feasible set. On the other hand, projection methods (Section 4 ) can enforce feasibility explicitly but they come with both theoretical and practical pitfalls. The core problem is that the true constrained distribution induced by the constraints on the prior is not correctly sampled. This can happen for three reasons: (i) the constraints are not respected, (ii) the probability mass distribution is incorrect, and (iii) the sample lies outside of the support of the original distribution p . These problems are due to a number of factors. For nonlinear constraints, the projection typically has no closed-form solution, so it must be approximated with an iterative solver at every step. Moreover, it is often unclear what exactly should be projected. Directly projecting the noisy state onto the constraint set can be ill-posed or numerically unstable, since many constraints are only meaningful for clean data. Projecting an estimate of the clean sample is not a clear fix either. Sample consistency with p is often broken, for example, by introducing discontinuities into otherwise continuous samples. Finally, repeated projection steps can systematically bias the sampler. In the extreme, this behaves like an attractor toward “nearest” feasible points, reducing diversity and causing mode collapse rather than sampling from the full constrained distribution. Contributions. In this work, we address these limitations by formalizing the constrained forward process and arguing that correct constrained sampling requires approximating the constrained marginal distributions p_{t}^{\mathcal{C}} induced by a forward diffusion SDE starting from the constrained set \mathcal{C} . We define the constrained forward process, derive the associated constrained marginals and backward process, and design the Predict-Project-Renoise (PPR) sampling algorithm that approximates the constrained marginals at every step. This approach achieves orders of magnitude lower constraint violations and better mass distribution while remaining consistent with \mathcal{C} . 2 Score-based generative modeling Forward process. The goal of generative modeling is to approximate the data distribution p(x) of x\in\mathbb{R}^{D} . In score-based diffusion ( 41 ) , this is done by learning to reverse a forward corruption process. Following 23 , we define the forward corruption process as the Stochastic Differential Equation (SDE) \mathrm{d}x_{t}=f(x_{t},t)\mathrm{d}t+g(t)\mathrm{d}w_{t}, (1) where f:\mathbb{R}^{D+1}\to\mathbb{R}^{D} is a drift term, g:\mathbb{R}\to\mathbb{R} is a scalar diffusion coefficient, and w_{t} is a standard Wiener process. Starting from p_{0}=p , this SDE defines a series of marginal densities \{p_{t}(x_{t})\}_{t\in[0,T]} , with x_{t}\sim p_{t}(x_{t}) for all t , which interpolate between the data distribution and p_{T} . Choosing f and g defines the perturbation or forward kernels p(x_{t}|x_{0}) , inducing the marginals p_{t}(x_{t})=\int p(x_{t}\mid x_{0})\,p(x_{0})\,\mathrm{d}x_{0}. (2) If f and g are chosen appropriately, then p(x_{t}|x_{0}) and p_{T} become Gaussian ( 33 ) . In particular, p_{T} can become \mathcal{N}(0,\sigma_{T}I) . We also note that the forward kernels p(x_{t}|x_{0}) depend only on f and g , and are independent of the data distribution p . Backward process. 2 shows that given a forward process (Eq. 1 ), there exists a reverse-time or backward process from T to 0 transporting p_{T} back to p_{0} given by \mathrm{d}x_{t}=\left[f(x_{t},t)-g^{2}(t)\nabla_{x_{t}}\log p_{t}(x_{t})\right]\mathrm{d}t+g(t)\mathrm{d}\tilde{w}_{t}, (3) with x_{T}\sim p_{T}(x_{T}) and where \tilde{w}_{t} is the reverse-time Wiener process. Remarkably, in this reverse process, the marginal distributions of x_{t} are identical to those of the forward process. This allows for sampling from p by generating a sample from p_{T} and then simulating the reverse process. However, the score s(x_{t})=\nabla_{x_{t}}\log p_{t}(x_{t}) is unknown because we do not know p(x) . Training and sampling. As the score is unknown, it has to be learned from data. In this work, we follow 19 and parameterize the score using Tweedie’s formula s_{\theta}(x_{t})=(d_{\theta}(x_{t})-x_{t})/\sigma_{t}^{2} ( 42 ; 10 ) . The denoiser d_{\theta}(x_{t})=\mathbb{E}[x_{0}|x_{t}] predicts the posterior mean of the clean sample given the noisy input. The resulting denoising objective minimizes \mathbb{E}_{x_{0}\sim p(x_{0})}\mathbb{E}_{x_{t}\sim p(x_{t}|x_{0})}\|d_{\theta}(x_{t})-x_{0}\|^{2}. (4) Once the denoiser is available, a way to sample from the reverse process is to use the predictor-corrector (PC) sampler ( 41 , Algorithm 1 ). This algorithm alternates between a Predict step (e.g. DDPM ( 15 ) , DDIM ( 39 ) ), which moves the samples from p_{t} to p_{t-1} and a number of Correct steps (e.g. Langevin steps based on the score), which corrects the marginals p_{t-1} . The Predict step provides an update along the reverse dynamics, but discretization error and imperfect scores can leave the resulting samples slightly off-distribution. The Correct step reduces this mismatch by pulling samples toward higher-density regions of p_{t-1} while re-injecting controlled noise to preserve diversity. Algorithm 1 Predictor-corrector reverse sampling. 1: Input: denoiser d_{\theta} , predictor Predict , corrector Correct , and N . 2: Initialize: sample x_{T}\sim p_{T}(x_{T}) 3: for decreasing t\in\{T,...,1\} do 4: \hat{x}_{0}\leftarrow d_{\theta}(x_{t}) 5: x_{t-1}\leftarrow\textsc{Predict}\!\left(x_{t},\hat{x}_{0},t\right) 6: for i=1,\dots,N do 7: \hat{x}_{0}\leftarrow d_{\theta}(x_{t-1}) 8: x_{t-1}\leftarrow\textsc{Correct}(x_{t-1},\hat{x}_{0},t-1) 9: end for 10: end for 11: Output: x_{0} 3 Method The reverse process (Eq. 3 ) allows us to sample (approximately) from the data distribution p . However, in many domains, we are interested in imposing