Optimal Stepsize for Diffusion Sampling

Jianning Pei, Han Hu, Shuyang Gu · 2025

Read Optimal Stepsize for Diffusion Sampling by Jianning Pei, Han Hu, Shuyang Gu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21774.txt.

Introduction

Diffusion models draw beautiful images slowly, taking many small denoising steps because nobody knows the ideal step sizes. Jianning Pei, Han Hu, and Shuyang Gu compute them in a 2025 paper. Their framework, optimal stepsize distillation, learns a schedule by distilling it from reference trajectories.

Cast as recursive error minimization, the schedule search gains a dynamic programming structure with guaranteed global bounds. The distilled schedules transfer: they hold up across model architectures, solvers, and noise schedules. Text-to-image generation ran ten times faster while keeping 99.4 percent of benchmark performance. The authors release their code, arguing step size deserves the same principled attention as denoising direction.

Catalog record

CREATORJianning Pei, Han Hu, Shuyang Gu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21774

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optimal Stepsize for Diffusion Sampling Jianning Pei, Han Hu, Shuyang Gu Abstract Diffusion models achieve remarkable generation quality but suffer from computational intensive sampling due to suboptimal step discretization. While existing works focus on optimizing denoising directions, we address the principled design of stepsize schedules. This paper proposes Optimal Stepsize Distillation, a dynamic programming framework that extracts theoretically optimal schedules by distilling knowledge from reference trajectories. By reformulating stepsize optimization as recursive error minimization, our method guarantees global discretization bounds through optimal substructure exploitation. Crucially, the distilled schedules demonstrate strong robustness across architectures, ODE solvers, and noise schedules. Experiments show 10 × \times accelerated text-to-image generation while preserving 99.4% performance on GenEval. Our code is available at https://github.com/bebebe666/OptimalSteps . 1 Introduction Score-based generative models [ 29 , 28 ] have emerged as a groundbreaking paradigm in generative modeling, achieving state-of-the-art results across diverse domains. Unlike traditional approaches that directly model complex data distributions, these methods decompose the target distribution into a sequence of conditional distributions structured as a Markov chain. While this decomposition enables tractable training through iterative noise prediction, it inherently necessitates computational expensive sampling procedures. From a theoretical perspective, diffusion models approximate the data generation process as a continuous transformation from noise to structured data, parameterized by an infinite series of conditional distributions mapping. But in practice, computational constraints require discretizing this continuous trajectory into a finite sequence of steps, introducing a critical trade-off between sampling efficiency and approximation fidelity. The discretization includes two key factors: the update direction (determined by the score function) and the stepsize (controlling the progression towards the target distribution), as shown in Figure 2 . While significant attention has been devoted to optimizing update directions, including higher-order solvers [ 18 , 19 ] and corrected network predictions [ 38 ] , the principled design of stepsize remains underexplored. Existing methods predominantly rely on heuristic schedules, such as the uniform timesteps in DDIM [ 27 ] or the empirically designed schedule in EDM [ 10 ] , lacking theoretical guarantees of optimality under constrained step budgets. Figure 2 : Two key factors in diffusion sampling: direction strategy(left) and stepsize strategy(right). In this work, we propose a principled framework for searching the O ptimal S tepsize in diffusion S ampling ( OSS ), compatible with arbitrary direction strategies. Specifically, we regard the stepsize searching problem as a knowledge distillation problem, where a ”student” sampling process with limited steps aims to approximate the results of a ”teacher” sampling process with abundant steps. Crucially, we identify that this distillation objective exhibits recursive substructure—the optimal step schedule for N steps inherently contain optimal sub-schedules for fewer steps. Exploiting this insight, we develop a dynamic programming algorithm that systematically derives theoretically optimal step sequences, ensuring maximal proximity to the teacher model’s output under any pre-defined student sampling steps. It guarantees that the distilled schedule achieves the closest approximation to the teacher’s continuous trajectory under computational constraints. Importantly, our experimental analysis demonstrates the universal applicability and robustness of the proposed stepsize distillation framework. It exhibits consistent performance across various model architectures (e.g., U-Net [ 10 ] vs. Transformer-based [ 21 ] ), ODE solver orders (1st to 3th-order [ 19 ] ), noise schedules (linear [ 27 ] /Flow Matching [ 17 , 15 ] /EDM [ 10 ] ), and diverse denoising direction strategies [ 38 , 18 , 19 ] . Furthermore, when applying the optimal stepsize to text2image and text2video diffusion pipelines, our method achieves 10 \times speedups while preserving 99.4% of the teacher model’s performance metrics on GenEval [ 5 ] benchmark. Such universal acceleration capability suggests broad practical potential for deploying latency-efficient diffusion models without sacrificing output quality. Above all, our key contributions are as follows: 1. Theoretically Optimal Stepsize Distillation: We formulate stepsize optimization as a dynamic programming problem, proving that our solution can achieve global error minimization. 2. Architecture-Agnostic Robustness: We demonstrate that the distilled schedules generalize across datasets, noise schedules, and ODE solvers, outperforming heuristic baselines in diverse settings. 3. Efficient Adaptation: Our method enables lightweight schedule calibration across tasks, achieving 5-10 \times speedup with negligible performance degradation. 2 Related Work 2.1 Diffusion sampling direction strategy Recent advances focused on correcting high-order direction for diffusion sampling acceleration. The earliest work [ 27 ] adopted first-order Euler for discretization diffusion ODEs. EDM [ 10 ] pioneers adopting Heun’s [ 1 ] method for second-order gradient estimation between steps, reducing the Euler approximation error. PNDM [ 16 ] figured out the way of solving the denoising differential equation on manifold. AMED [ 40 ] utilized an additional network to predict the middle point of each step in the second order Heun solver. DPM-Solver [ 18 ] leveraged the semi-linear structure of diffusion ODEs, decomposing backward processes into linear/nonlinear components through Taylor expansion. Building on this, DPM-Solver++ [ 19 ] introduced a multistep gradient reuse and shifts schedule, further saving the computational efforts. UniPC [ 38 ] generalized these via a predictor-corrector framework using Lagrange polynomial interpolation, enabling arbitrary-order accuracy with minimal computational overhead. These innovations collectively established theoretical foundations for balancing discretization precision and computational overhead in diffusion sampling. 2.2 Diffusion sampling stepsize strategy Traditional diffusion models typically employ predefined fixed-step strategies. For instance, DiT [ 21 ] and SDXL [ 22 ] adopt uniform timestep allocations, while EDM [ 10 ] and CM [ 30 ] rely on manually crafted schedules. These strategies often yield suboptimal results in few-step sampling scenarios, prompting recent studies to explore stepsize optimization. Existing methods fall into two categories: training-free optimization and learning-based strategies. The former approaches, including DM [ 36 ] , which minimizes inference error bounds via constrained trust-region methods, have a poor generalization ability since it shares the stepsize strategy across different diffusion networks and different solvers. Similarly, AYS [ 25 ] optimizes stepsize through KL divergence upper bounds (KLUB) but remains confined to SDE frameworks. [ 8 ] gets a better stepsize according to the similarity with the results of the high-order solver, which mixes the direction and stepsize of denoising. [ 35 ] selects the sampling steps based on dynamic programming and minimizes the sum of ELBO but incurs significant performance degradation. GITS [ 3 ] also employs dynamic programming to optimize stepsize but exhibits deviations from minimizing global error due to fixed cost matrices. The latter category, exemplified by LD3 [ 32 ] , leverages a monotonic network for stepsize prediction, which is supervised by global errors. However, it requires additional high computational training. GGDM [ 34 ] adds trainable parameters to the schedule and achieves high-quality samples through well-designed training. We summarize some recent methods in Table 1 from different perspectives. Criterion AYS [ 25 ] GITS [ 3 ] DM [ 36 ] LD3 [ 32 ] OSS Training-free ✓ ✓ ✓ ✕ ✓ Stability ✕ ✓ ✓ ✓ ✓ Solver-aware ✓ ✕ ✕ ✓ ✓ Global error ✕ ✕ ✕ ✓ ✓ Table 1 : Related work comparison. Training-free denotes whether eliminating backpropagation during optimization. Stability denotes whether requires specialized stabilization techniques. Solver-aware means the algorithm integrates solver dynamics and network states. Global error means optimizing the final calibration error. 3 Method 3.1 Diffusion sampling preliminaries The continuous diffusion

Read the complete text