Is Monotonic Sampling Necessary in Diffusion Models?
Muhammad Haris Khan · 2026
Read Is Monotonic Sampling Necessary in Diffusion Models? by Muhammad Haris Khan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11773.txt.
Introduction
In 2026, Muhammad Haris Khan questions a habit every diffusion sampler shares: noise levels that only ever fall. He tests four families of nonmonotonic schedules on DDPM, EDM, and flow matching models across 90 setups on CIFAR-10. None beats the monotonic baseline, though the penalty ranges from severe in DDPM to negligible in EDM. He formalizes this spread as a Schedule Sensitivity Coefficient, a cheap check of whether a denoiser has truly converged.
Catalog record
| CREATOR | Muhammad Haris Khan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89507 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11773
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Is Monotonic Sampling Necessary in Diffusion Models? Muhammad Haris Khan Abstract Diffusion models generate samples by iteratively denoising a Gaussian prior, traversing a sequence of noise levels that, in every published sampler, decreases monotonically. Six years of intensive work has refined nearly every aspect of this recipe — the corruption operator, the training objective, the schedule shape, the architecture, and the ODE solver — yet the assumption of monotonicity itself has never been systematically tested. Here we ask whether monotonic sampling is load-bearing or merely conventional. We design four families of structured non-monotonic schedules and apply them to three architecturally distinct generative models — DDPM, EDM and Flow Matching — across NFE budgets ranging from 10 10 to 200 200 function evaluations, plus a 42 42 -cell hyperparameter ablation, on CIFAR-10. Across all 90 90 tested configurations, no tested non-monotonic schedule improves on the monotonic baseline. The magnitude of the penalty, however, spans nearly three orders of magnitude: persistent and substantial in DDPM, intermediate in Flow Matching, and indistinguishable from zero in EDM. We show that this variation is not noise but a structural property of each trained denoiser, and we formalise it as the Schedule Sensitivity Coefficient — a cheap, architecture-agnostic diagnostic that provides evidence of non-convergence to the Bayes-optimal denoiser at the critical noise level. Our findings justify the field’s tacit reliance on monotonic schedules and supply a new probe of diffusion-model quality complementary to sample-quality metrics such as Fréchet Inception Distance. 1 Introduction Every diffusion sampler in widespread use today—DDIM, DDPM, EDM with Heun integration, DPM-Solver, UniPC, the Euler and stochastic flow-matching samplers—shares a single, unstated structural assumption: the noise level decreases monotonically along the sampling trajectory . A schedule in which \hat{\sigma}_{\tau_{i}} is strictly decreasing in i is treated as a definitional property of the reverse process, not as a design choice. This monotonicity assumption has remained a silent default for the roughly six years since the publication of DDPM, despite the field’s otherwise exhaustive ablation of nearly every other design dimension: the corruption operator ( Bansal et al. 2023 ; Daras et al. 2023b ; Hoogeboom & Salimans 2023 ) , the parametrisation ( Karras et al. 2022 ) , the training objective and weighting ( Kingma et al. 2021 ; Hang et al. 2023 ) , the variance schedule shape ( Nichol & Dhariwal 2021 ; Chen 2023 ) , the model architecture ( Peebles & Xie 2023 ; Karras et al. 2024b ) , and the sampling solver ( Lu et al. 2022a ; Zhang & Chen 2023 ; Zhao et al. 2023 ) . What has not been studied is whether the trajectory must move monotonically in the first place. Three independent threads of evidence suggest the question is non-trivial. First, the DDIM update equation ( Song et al. 2021a ) is mathematically well-defined for any choice of next-timestep \tau_{i+1} , with no restriction to \tau_{i+1}<\tau_{i} ( Proposition 1 ). Second, stochastic DDPM sampling already performs an implicit form of reheating: at every step it adds fresh Gaussian noise that partially counteracts the denoising progress made in the same step, and at high NFE this stochastic “churn” yields measurably better FID than deterministic DDIM ( Ho et al. 2020 ; Karras et al. 2022 ) . Third, Restart Sampling ( Xu et al. 2023 ) explicitly injects a large block of noise mid-trajectory and reports state-of-the-art quality when paired with deterministic correction. Taken together, these observations leave open a basic question that the literature has not answered: if diffusion samplers are mathematically permitted to step backward along the noise schedule, and if both stochastic DDPM and Restart Sampling appear to benefit from doing so, then is monotonicity merely a historical convention—something we could relax for free, or even profit from? By analogy to annealing-style escape—a heuristic familiar from non-convex optimisation, where occasional uphill steps allow an iterative procedure to leave poor local optima—one might hypothesise that structured reheating , i.e. deliberate non-monotonic increases in \hat{\sigma} during sampling, should help the sampler escape suboptimal trajectories and reach better samples. This hypothesis is consistent with every piece of evidence cited above. Yet it has never been tested. We test it. We design four families of structured non-monotonic schedules ( Figure 4 ): Single Reheat , a single backward step of magnitude \delta at position t_{\text{reheat}} ; Sawtooth , periodic reheats of equal magnitude; Damped Oscillation , geometrically decaying reheats concentrated at the critical noise level \log\mathrm{SNR}\approx 0 identified by Hang et al. 2024 as governing perceptual quality; and Adaptive Reheat , an online schedule that triggers reheating when consecutive score predictions diverge. For the three fixed-budget non-adaptive families (Single Reheat, Sawtooth, Damped Oscillation), the total NFE budget remains fixed and matches the monotonic baseline—reheats are inserted within the budget, not added on top of it. Adaptive Reheat is the one exception: it adds reheat steps online up to a cap of 15 , and we report both fixed-NFE and time-matched comparisons for it ( Section 4.2 ). We evaluate all four families against monotonic DDIM on three architecturally distinct model families: DDPM ( Ho et al. 2020 ) , EDM ( Karras et al. 2022 ) , and Flow Matching ( Lipman et al. 2023 ) , on CIFAR-10, across NFE budgets from 10 to 200 . A 42-configuration grid search over (t_{\text{reheat}},\delta) provides exhaustive ablation. Two findings emerge, one negative and one positive. The negative finding is that reheating never helps. Across 48 schedule–budget combinations and the 42-cell ablation, no tested non-monotonic configuration improves upon the monotonic baseline on any model family at any NFE budget ( Figures 3 and 2 , Table 1 ). The penalty is monotonically increasing in reheat magnitude \delta at every reheat position, with no local minimum below zero. The result is robust: it holds for explicit single-reheat schedules, for periodic and damped-oscillation schedules, for an online adaptive criterion, and for both deterministic and stochastic-corrected variants. The positive finding—and the paper’s main contribution—is that while reheating uniformly hurts, the magnitude of the penalty is sharply, predictably, and architecture-dependent . At NFE = 100, the damped-oscillation schedule produces a +1.31 FID penalty on DDPM that has not converged toward zero even at the highest budget tested; a +0.04 FID penalty on Flow Matching, an order of magnitude smaller; and a -0.003 FID change on EDM—statistically indistinguishable from zero at the noise floor of FID-25K ( Figure 1 , Table 2 ). The penalty thus spans nearly three orders of magnitude across model families that achieve roughly tenfold-different baseline FID. This is not noise: the gap is consistent across NFE budgets, schedule types, and reheat magnitudes. We bound the per-reheat-step squared displacement by K_{1}\,L^{2}_{t}\,\varepsilon^{2}(\hat{\sigma}_{i})+K_{2}\,\varepsilon^{2}(\hat{\sigma}^{\prime}) , where L_{t} is the local Lipschitz constant of the trained denoiser at the reheated noise level and \varepsilon is the per-noise-level denoiser error ( Proposition 2 ); for small reheats this reduces to a linear-in- \delta FID penalty whose slope is L_{t}\cdot\varepsilon(\hat{\sigma}_{i}) ( Section 5 ). A reheat step re-corrupts the sampler’s current best estimate \hat{\mathbf{x}}_{0} and demands the denoiser re-process it; for a Bayes-optimal denoiser this second pass extracts no additional information and the step is a wasted NFE. For an imperfect denoiser, errors