Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions

Tejas Jayashankar ∗, J. Jon Ryu ∗, Gregory Wornell · 2025

Read Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions by Tejas Jayashankar ∗, J. Jon Ryu ∗, Gregory Wornell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09609.txt.

Introduction

Tejas Jayashankar, Jon Ryu, and Gregory Wornell simplify one-step generative model training in a 2025 paper. Diffusion models make fine images but need many steps; one-step models promise speed yet train awkwardly. Their framework, Score-of-Mixture Training, reframes the job around mixtures of real and generated samples.

Training minimizes a skewed divergence by estimating the mixture's score across noise levels. The same machinery can distill a pretrained diffusion model into a one-step student. Implementation is short, tuning is light, and training stays stable. On two standard image benchmarks the method matched or beat existing approaches. The authors pitch it as the simple route to fast generators.

Catalog record

CREATORTejas Jayashankar ∗, J. Jon Ryu ∗, Gregory Wornell
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE90 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09609

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions Tejas Jayashankar ∗, J. Jon Ryu ∗, Gregory Wornell Abstract We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the α \alpha -skew Jensen–Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64×64 show that SMT/SMD are competitive with and can even outperform existing methods. † † affiliationnotice: Equal contribution 1 Introduction Fast and efficient sampling is a key characteristic sought after in modern generative samplers. For many years, generative adversarial networks (GANs) ( Goodfellow et al., 2014 ) set the benchmark for high-quality one-step generative sampling. However, due to the inherent training instabilities associated with discriminator training, attention has recently shifted toward diffusion-based generative models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ; Karras et al., 2022b ) . These models trade-off sampling efficiency for more stable training and significantly improved downstream sample quality through iterative sampling. More recently, the diffusion distillation approach has been studied as an appealing option to significantly reduce the number of sampling steps. Early work ( Luhman & Luhman, 2021 ; Salimans & Ho, 2022 ; Meng et al., 2023 ; Berthelot et al., 2023 ) focused on training a student model with a lower sampling budget by condensing multiple teacher denoising steps into one. The most recent works on distillation improve performance further by leveraging a pretrained model for distribution matching via minimization of the reverse KL divergence ( Luo et al., 2024a ; Yin et al., 2024b ; Yin et al., 2024a ; Salimans et al., 2024 ; Xie et al., 2024 ) . While attractive, distillation approaches necessitate a pretrained diffusion model which adds a significant overhead on the required compute. As yet another alternative, consistency models ( Song et al., 2023 ; Song & Dhariwal, 2024b ) and their variants ( Kim et al., 2024 ) have been proposed for training few-step generative models from scratch by simulating the trajectories of the induced probability flow ODE ( Song et al., 2020 ) of a diffusion process. While consistency models have demonstrated promising results in both distillation and training from scratch, training is sensitive to the choice of noise schedule and distance measure ( Geng et al., 2025 ) . In this paper, we tackle the problem of training high-quality one-step generative models more directly, i.e., without simulating an iterative reverse diffusion process for sampling or leveraging a pretrained diffusion model during training. Starting from first principles of statistical divergence minimization, we show that a high-quality one-step generative model can be trained from scratch in a stable manner, via the multi-noise-level denoising score matching (DSM) technique ( Vincent, 2011 ) used in diffusion models. We emphasize that we do not require a simulation of the reverse diffusion process in our framework. The proposed framework achieves the best of several worlds: (1) a new, simple statistical divergence minimization framework without probability paths of ODE (like GAN), (2) stable training using denoising score matching (like diffusion models), (3) training from scratch without a pretrained diffusion model (like consistency models), and (4) near state-of-the-art one-step image generative performance (like GAN and consistency models). We also demonstrate that the proposed method can be extended to distill from a pretrained diffusion model, and can achieve performance similar to state-of-the-art methods for the same. See Table 1 for the overview of comparison. Table 1 : Comparison of different generative modeling techniques capable of high-quality sample generation. Generative models Training idea Generation Training stability Require pretrained model? GAN minimizing JSD, with discriminator one-step unstable N Diffusion models training multi-noise-level denoisers via DSM multi-step stable N Diffusion distillation (mostly) minimizing reverse KLD (in DMD) { one ,few}-step stable Y Consistency distillation simulating trajectories of probability flow ODE { one ,few}-step stable Y Consistency training unstable N Score-of-Mixture Training (ours) minimizing \{\alpha{\text{-JSD}}\}_{\alpha\in[0,1]} with multi-noise-level training & scores of mixtures via DSM one-step stable N Score-of-Mixture Distillation (ours) Y The rest of the paper is organized as follows: In Sec. 2 we introduce the necessary background and related works central to our proposed method. In Sec. 3 we introduce our novel one-step generative modeling approach and in Sec. 4 we detail how our framework can be modified to perform diffusion distillation. We describe practical implementation details in both the latter sections and present experimental results in Sec. 5 . We conclude with remarks in Sec. 6 . Proofs and training details are deferred to Appendix. 2 Preliminaries and Related Work In one-step generative modeling, we wish to align the generated sample distribution q_{\theta}({\bf x})\mathrel{\mathop{\mathchar 58\relax}}=\int\delta({\bf x}-{\bf g}_{\theta}({\bf z}))q({\bf z})\mathop{}\!\mathrm{d}{\bf z} with the true data distribution p({\bf x}) . Here, {\bf g}_{\theta}\colon\mathcal{Z}\rightarrow\mathcal{X} is a parametric neural sampler which is also often called an implicit generative model that transforms samples from a base measure q({\bf z}) . In this section, we review some popular methods for training generative models, which will serve as preliminaries for our framework. More detailed discussion on the literature is deferred to Appendix B . Generative Adversarial Networks. The most prominent approach in training implicit generative models is the generative adversarial network (GAN) ( Goodfellow et al., 2014 ) . In its most standard and widely used form, it alternates between the gradient steps of discriminator and generator training, which are \displaystyle\min_{\psi} \displaystyle~\E_{p({\bf x})}[\mathsf{sp}(-\ell_{\psi}({\bf x}))]+\E_{q_{\theta}({\bf x})}[\mathsf{sp}(\ell_{\psi}({\bf x}))], (1) \displaystyle\min_{\theta} \displaystyle~\E_{q_{\theta}({\bf x})}[\mathsf{sp}(-\ell_{\psi}({\bf x}))], (2) respectively, where \mathsf{sp}(y)\mathrel{\mathop{\mathchar 58\relax}}=\log(1+e^{y}) denotes the softplus function. 1 1 1 The generator loss \E_{q_{\theta}({\bf x})}[\mathsf{sp}(-\ell_{\psi}({\bf x}))] in the second line is the so-called non-saturating version, while the original GAN generator loss \E_{q_{\theta}({\bf x})}[-\mathsf{sp}(\ell_{\psi}({\bf x}))] is referred to as saturating . Here, we will call \ell_{\psi}({\bf x}) the discriminator , which is supposed to capture the log density ratio \log\frac{p({\bf x})}{q_{\theta}({\bf x})} . 2 2 2 Note the one-to-one correspondence between the standard definition of discriminator D_{\psi}({\bf x})\mathrel{\mathop{\mathchar 58\relax}}=\frac{\exp({\ell_{\psi}({\bf x})})}{1+\exp({\ell_{\psi}({\bf x})})}\in[0,1] . This so-called adversarial training can be understood as minimizing the Jensen–Shannon divergence (JSD) with the help of discriminator, via the variational characterization of JSD. Despite the popularity of GANs, training them is notoriously difficult. Although various techniques have been proposed to regularize the GAN objective—through alternatives to JSD ( Nowozin et al., 2016 ; Arjovsky et al., 2017 ; Mao et al., 2017 ) , novel regularizers ( Miyato et al., 2018 ) , and specialized network architectures ( Karras et al., 2021 ; Brock et al., 2019 ; Sauer et al., 2022 ) —the discriminator training remains unstable. This has sparked increasing interest in developing new objectives for training generative models which we briefly discuss below. Diffusion Models. Diffusion models or score-based generative models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ) are state-of-the-art generative models that are based on the principles of thermodynamic diffusion. Given a forward stochastic differential equation (SDE) process \text{d}{\bf x}_{t}=f({\bf x}_{t},t)\text{d}t+g(t)\text{d}{\bf w}_{t},

Read the complete text