DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling
Tong Zhao 1,2,∗ , Mingkun Lei 2,∗ , Liangyu Yuan 3,∗ , Yanming Yang 2 , Chenxi Song 2 , Yang Wang 2 , Beier Zhu 4 , Chi Zhang 2,† 1 Zhejiang University, 2 AGI Lab, Westlake University, 3 TongJi University, 4 University of Science and Technology of China https://github.com/Westlake-AGI-Lab/DyWeight · 2026
Read DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling by Tong Zhao 1,2,∗ , Mingkun Lei 2,∗ , Liangyu Yuan 3,∗ , Yanming Yang 2 , Chenxi Song 2 , Yang Wang 2 , Beier Zhu 4 , Chi Zhang 2,† 1 Zhejiang University, 2 AGI Lab, Westlake University, 3 TongJi University, 4 University of Science and Technology of China https://github.com/Westlake-AGI-Lab/DyWeight (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11607.txt.
Introduction
Zhao and co-authors present DyWeight, a learned multi-step solver that speeds up diffusion sampling. Diffusion models sample slowly, needing hundreds of evaluations per output. Newer solvers reuse past gradients, yet their handcrafted coefficients cannot track shifting sampling dynamics. DyWeight learns time-varying weights that gather gradients and scale the step size on their own. This calibration lines the solver's path up with the model's denoising dynamics, even at large steps. Across image benchmarks and text-to-image models, it wins on fidelity and stability with fewer evaluations.
Catalog record
| CREATOR | Tong Zhao 1,2,∗ , Mingkun Lei 2,∗ , Liangyu Yuan 3,∗ , Yanming Yang 2 , Chenxi Song 2 , Yang Wang 2 , Beier Zhu 4 , Chi Zhang 2,† 1 Zhejiang University, 2 AGI Lab, Westlake University, 3 TongJi University, 4 University of Science and Technology of China https://github.com/Westlake-AGI-Lab/DyWeight |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80468 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11607
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling Tong Zhao 1,2,∗ , Mingkun Lei 2,∗ , Liangyu Yuan 3,∗ , Yanming Yang 2 , Chenxi Song 2 , Yang Wang 2 , Beier Zhu 4 , Chi Zhang 2,† 1 Zhejiang University, 2 AGI Lab, Westlake University, 3 TongJi University, 4 University of Science and Technology of China https://github.com/Westlake-AGI-Lab/DyWeight Abstract Diffusion Models (DMs) have achieved state-of-the-art generative performance across multiple modalities, yet their sampling process remains prohibitively slow due to the need for hundreds of function evaluations. Recent progress in multi-step ODE solvers has greatly improved efficiency by reusing historical gradients, but existing methods rely on handcrafted coefficients that fail to adapt to the non-stationary dynamics of diffusion sampling. To address this limitation, we propose Dy namic Gradient Weight ing ( DyWeight ), a lightweight, learning-based multi-step solver that introduces a streamlined implicit coupling paradigm. By relaxing classical numerical constraints, DyWeight learns unconstrained time-varying parameters that adaptively aggregate historical gradients while intrinsically scaling the effective step size. This implicit time calibration accurately aligns the solver’s numerical trajectory with the model’s internal denoising dynamics under large integration steps, avoiding complex decoupled parameterizations and optimizations. Extensive experiments on CIFAR-10, FFHQ, AFHQv2, ImageNet64, LSUN-Bedroom, Stable Diffusion and FLUX.1-dev demonstrate that DyWeight achieves superior visual fidelity and stability with significantly fewer function evaluations, establishing a new state-of-the-art among efficient diffusion solvers. Code is available at https://github.com/Westlake-AGI-Lab/DyWeight . Figure 1 : Qualitative comparison on FLUX.1-dev 23 ; 22 at 7 NFEs. From top to bottom, the rows show results from DPM-Solver++(2M) 30 , iPNDM(2M) 28 ; 60 , and our DyWeight . Our method delivers superior visual fidelity, prompt alignment, and structural coherence. † † footnotetext: ∗ Equal contribution. † Corresponding author. Emails: {zhaotong68, chizhang}@westlake.edu.cn 1 Introduction Diffusion Models (DMs) 45 ; 12 ; 47 ; 27 have established a new state-of-the-art in generative modeling, capable of synthesizing data with unprecedented fidelity and diversity across modalities such as images 23 ; 7 ; 54 , audio 10 ; 14 , and video 51 ; 29 ; 20 . At their core, DMs transform a simple noise distribution into complex data through the reversal of a predefined noising process, which can be mathematically formulated as solving a corresponding Probability Flow Ordinary Differential Equation (PF-ODE) 46 ; 48 . Despite their remarkable performance, diffusion models suffer from inherently slow sampling speed 12 ; 46 . Generating a single high-quality sample typically requires evaluating a large neural network ( e.g . , U-Net 38 ) hundreds or even thousands of times, leading to substantial computational latency. This inefficiency poses a significant barrier for real-time and interactive applications, motivating extensive research on efficient numerical solvers to accelerate diffusion sampling 46 ; 15 ; 28 ; 60 ; 31 ; 30 ; 62 ; 61 ; 3 ; 2 . Among various acceleration approaches, explicit multi-step ODE solvers ( e.g . , DPM-Solver++ 30 and iPNDM 60 ) have emerged as particularly promising due to their ability to exploit historical gradient information. By reusing the model’s past denoising predictions as a “free lunch”, these solvers can improve accuracy at negligible extra cost, offering a compelling trade-off between quality and speed. Although multi-step solvers improve efficiency, their handcrafted coefficients are derived under the assumption that denoising gradients at neighboring timesteps share similar statistics 28 ; 60 . This stationarity assumption, however, does not hold in diffusion sampling, where the dynamics are highly non-stationary across time. As a result, a fixed weighting scheme fails to adapt to these changing dynamics, early noisy gradients may overpower or conflict with later, more reliable ones, causing accumulated bias and degraded image quality, particularly under large step sizes. Furthermore, the coefficients in these solvers are designed based on classical ODE assumptions of smooth, low-order trajectories within small integration intervals 9 ; 1 ; 6 . When extended to few-step sampling with large step gaps, these assumptions break down, introducing severe truncation errors. Recently, learning-based solvers have attempted to address these limitations by optimizing both solver coefficients and time schedules 50 ; 52 ; 25 ; 8 ; 63 ; 64 . However, these approaches typically adopt an explicitly decoupled optimization paradigm, maintaining an independent time sequence alongside the solver coefficients. To implement this, both components must be parameterized independently: the gradient weights are typically constrained to satisfy classical numerical rules, while the time sequence is parameterized with principled mathematical functions, such as cumulative softmax, to ensure valid and strictly decreasing steps. While functionally valid, this decoupled formulation yields a notoriously complex optimization landscape when solver coefficients and time discretization variables are jointly optimized, often necessitating cumbersome alternating strategies ( e.g . , multi-loop bi-level optimization) 8 . To overcome these bottlenecks in few-step sampling, we propose Dy namic Gradient Weight ing ( DyWeight ), a learning-based multi-step solver that introduces a streamlined implicit coupling paradigm. Rather than maintaining a separate, heavily constrained time sequence, DyWeight elegantly relaxes the classical numerical constraint that gradient weights must sum to one ( i.e . , not requiring \sum w=1 ). This simple yet profound relaxation yields a dual benefit: the normalized part of the weights adaptively learns to aggregate historical gradients, while the unnormalized sum inherently acts as an implicit time shifting mechanism that dynamically adjusts the macro-level integration boundary. This eliminates the need for complex timeline parameterizations, transforming a difficult joint-optimization problem into a highly efficient, single-round learning process. Furthermore, to correct the misalignment between the numerical timeline and the network’s internal noise expectations 24 ; 32 caused by large step gaps, we supplement our method with a micro-level time scaling to adjust the exact query time fed into the neural network. Together, implicit time shifting and scaling constitute our time calibration mechanism. To optimize these parameters efficiently, we employ a training-data-free distillation framework using on-the-fly teacher generation 63 ; 64 . By relying solely on endpoint supervision 50 rather than rigid intermediate path supervision, the solver is granted the flexibility to deviate from the teacher mid-way while achieving more accurate alignment with the data manifold at the final step. Our contributions are summarized as follows: • We propose DyWeight , a learning-based solver that replaces complex, decoupled parameterization with a relaxed weight formulation, implicitly coupling gradient aggregation with timestep adjustments. • We introduce a time calibration strategy combining implicit time shifting and time scaling, enabling the solver to maintain temporal alignment with the diffusion model’s internal dynamics in the few-step sampling regime. • Extensive experiments demonstrate that ours method achieves state-of-the-art visual fidelity with significantly fewer NFEs across diverse benchmarks, including complex flow-matching models like FLUX.1-dev 23 ; 22 , while requiring minimal optimization overhead. 2 Related work Figure 2 : Overview of DyWeight . Our method dynamically aggregates buffered historical gradients ( w_{t_{n},j}\cdot\mathbf{v}_{\text{buffer}} ) using unconstrained learned weights ( i.e . , not requiring \sum w=1 ). The unnormalized sum part adjusts the effective step size via time calibration ( t_{n}\rightarrow{\color[rgb]{0.9727,0.4805,0.4648}\tilde{t}_{n}} ). As illustrated on the right, DyWeight exhibits a distinct advantage in few-step generation scenarios. Multi-step diffusion solvers. A primary approach to accelerating DPM sampling is the design of efficient, training-free ODE solvers that reduce truncation error under few steps. Solvers are broadly categorized as