On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
Yi Zhang, Zhenyu Liao, Jingfeng Wu, Difan Zou · 2025
Read On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models by Yi Zhang, Zhenyu Liao, Jingfeng Wu, Difan Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16154.txt.
Introduction
Yi Zhang and co-authors name a failure mode in 2025 in the fast samplers used to run diffusion image models. Set rule samplers have become the default way to draw samples from these models. The team finds that such samplers can collapse the output, piling samples into a small local patch of the data space instead of spreading them as the data do.
They coin a measure for this collapse and show it occurs across many settings. The cause is a see saw in training: fitting the model well at low noise harms its fit at high noise. That poor high noise fit, joined with the set path of these samplers, drives the pile up. Guided by that account, they apply known fixes and test them. The paper warns that speed in sampling can quietly cost diversity.
Catalog record
| CREATOR | Yi Zhang, Zhenyu Liao, Jingfeng Wu, Difan Zou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-16154
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models Yi Zhang, Zhenyu Liao, Jingfeng Wu, Difan Zou Abstract Despite the widespread adoption of deterministic samplers in diffusion models (DMs), their potential limitations remain largely unexplored. In this paper, we identify collapse errors , a previously unrecognized phenomenon in ODE-based diffusion sampling, where the sampled data is overly concentrated in local data space. To quantify this effect, we introduce a novel metric and demonstrate that collapse errors occur across a variety of settings. When investigating its underlying causes, we observe a see-saw effect , where score learning in low noise regimes adversely impacts the one in high noise regimes. This misfitting in high noise regimes, coupled with the dynamics of deterministic samplers, ultimately causes collapse errors. Guided by these insights, we apply existing techniques from sampling, training, and architecture to empirically support our explanation of collapse errors. This work provides intensive empirical evidence of collapse errors in ODE-based diffusion sampling, emphasizing the need for further research into the interplay between score learning and deterministic sampling, an overlooked yet fundamental aspect of diffusion models. 1 Introduction Maximum likelihood-based generative modeling methods have demonstrated impressive capabilities for recovering data distributions, with diffusion methods being the latest advancement. A key advantage of diffusion models is their ability to achieve better diversity, whereas previous GAN-based methods goodfellow2014gan often struggle to fully capture the multi-modality of the data distribution rombach2022ldm ; liu2023flow ; diffusionbeatgan . In diffusion models, the data distribution is learned by estimating the score function (the gradient of the log probability) through training denoisers. To enhance their performance, the score function is learned across various noise regimes and utilized in an annealing manner. The trained score models are then employed to sample from the data distribution, either via score-based Markov Chain Monte Carlo (MCMC) song2019scoremcmc or a reverse diffusion process ho2020ddpm ; song2020score ; anderson1982reversesde . These models have achieved remarkable success in tasks such as super-resolution li2022SR ; yue2024resshiftSR ; saharia2022imageSR , text-to-image generation rombach2022ldm ; saharia2022imageGen ; nichol2021glide ; ramesh2022hierarchical , and video generation ho2022video ; wu2023video ; blattmann2023alignvideo . At first glance, diffusion models appear to be a complete solution: they possess strong theoretical foundations and achieve state-of-the-art practical performance, providing a double-layered validation. However, their practical behavior remains poorly understood. As researchers delve deeper into the success of diffusion models, they uncover intriguing phenomena—such as memorization gu2023memorization ; wen2024detectingmemo ; dar2023investigatingmemo ; carlini2023extractingmemo ; somepalli2023diffusionmemo , generalization kadkhodaie2024generalization ; li2023generalization ; yoon2023diffusiongeneralization , and hallucination aithal2024hallucinations ; kim2025tacklinghallucinations ; evaluatehallucinations —that further complicate our understanding. A key observation is that, the Deep Neural Network (DNN) used for score learning plays a critical role in these phenomena. For instance, generalization in diffusion models has been attributed to certain inductive biases inherent in DNNs kadkhodaie2024generalization . Additionally, hallucination is often linked to underfitting in low noise regimes, where the target score function is complex aithal2024hallucinations . Conversely, memorization occurs when the model perfectly fits the optimal empirical denoiser gu2023memorization , indicating an overfitting of the score function of the true population distribution. Motivated by these insightful findings, we aim to continue this line of research by discovering and understanding the potential issues of the diffusion model paradigm. In particular, in addition to the previous works that mainly focus on the score learning, we also pay attention to the sampler in the diffusion model (i.e., the inference algorithm for DMs), especially the deterministic ones, e.g., DDIM ( song2021ddim ) , which have gained significant popularity due to their improved sampling efficiency, controllable generation, and theoretical coherence. To systematically study this, we conducted extensive experiments by training hundreds of score-based DNNs on both real image and synthetic datasets, using different model sizes, dataset sizes, and sampling algorithms. Through our study, we identify a critical yet previously overlooked issue arising from the interplay between deterministic sampling and score learning. In specific, we observed a novel phenomenon that is orthogonal to the previously identified issues (see Figures 1 and 2 ): Despite the success of deterministic samplers in diffusion models, the samples they generate tend to become overly concentrated in certain regions of the data space , compared to training samples and those generated by stochastic samplers (e.g., DDPM ( ho2020ddpm ) ). We term this phenomenon collapse error . We summarize the main findings of this paper as follows: • What is Collapse Error? We identify collapse error , a failure mode in diffusion models using deterministic samplers, which is orthogonal to previously known issues. To quantify this phenomenon, we introduce TID, a numerical metric that reveals the universality of collapse error across both synthetic and real-world datasets under diverse experimental settings. • Why Do They Occur? We reveal that collapse error stems from the interplay between deterministic sampling dynamics and misfitting of the score function in high noise regimes. Through extensive empirical analysis, we show that this misfitting arises from the simultaneous learning of score functions in low and high noise regimes, a phenomenon we term the see-saw effect . To our knowledge, this is the first study to explicitly examine how score learning and sampler dynamics jointly contribute to failure modes in diffusion models. • How Is Our Explanation Validated? To support our interpretation of collapse error, we evaluate several existing techniques originally proposed for other empirical purposes across three dimensions: sampling strategies, training methodologies, and model architectures. We find that these techniques coincide with our theoretical understanding and effectively reduce collapse error, thereby providing indirect validation of our hypothesis. 2 Related Works Mode Collapse. When discussing “collapse”, it is natural to associate it with mode collapse in GANs goodfellow2014gan , a well-known issue where the generator fails to capture the full multi-modality of the data distribution che2017ganmode ; srivastava2017veegancollapse ; zhang2018ganconvergence . Early work by Goodfellow et al. goodfellow2014gan introduced the GAN framework but acknowledged its instability during training, which often leads to mode collapse. Numerous efforts have sought to mitigate this issue. Techniques such as minibatch discrimination salimans2016improved , unrolled GANs metz2016unrolled , and PacGAN lin2018pacgan attempt to diversify the generator’s output by improving the discriminator’s ability to detect mode collapse. Wasserstein GANs (WGAN) arjovsky2017wasserstein and its improved variant (WGAN-GP) gulrajani2017improvedgan tackle mode collapse by reformulating the loss function to improve stability. Moreover, architectural innovations such as progressive growing of GANs karras2018pg and BigGAN brock2018large have shown improved diversity in generated samples by leveraging better network designs. Despite these advances, mode collapse remains an active area of research, especially in tasks requiring high data complexity and diversity. While diffusion model collapse errors share some similarities with GAN mode collapse, a key distinction is that collapse errors in diffusion models can occur within a single mode. Furthermore, mode collapse in GANs is primarily caused by the discriminator dominating the training process arjovsky2017wasserstein ; gulrajani2017improvedgan ; lin2018pacgan , whereas the dynamics of collapse errors in diffusion models are fundamentally different. Samplers of Diffusion Models. Stochastic samplers for diffusion models include annealing Langevin dynamics song2019scoremcmc and reverse stochastic differential equations song2020score . Empirical studies have shown that these stochastic methods suffer from high computational costs. In Langevin dynamics, a large number of steps is required to ensure adequate mode mixing