DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models

Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain, Engelbert Mephu Nguifo · 2026

Read DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models by Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain, Engelbert Mephu Nguifo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00798.txt.

Introduction

Shafi and co-authors fix a hidden flaw in shrinking diffusion models in this 2026 paper. These models make images from noise, guided by a class label. Guidance blends two score branches: one sees the label, and one does not. When a big model is distilled into a small one, training often watches only the blended output.

That leaves the two branches unpinned. Both can collapse to the same output while the loss stays low. Guidance then loses its force when the model runs. Their method, DASH, trains each branch on its own. An anchor term pulls the labeled branch toward true noise. The teacher's learned per-step plan passes to the student as a fixed prior. On CIFAR-10, CIFAR-100, and ImageNet-64, a student shrunk over five-fold stays within four FID points of its teacher. It also keeps at least 89 percent of the teacher's guidance gap.

Catalog record

CREATORAbdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain, Engelbert Mephu Nguifo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00798

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain, Engelbert Mephu Nguifo Abstract Parameter compression of class-conditional diffusion models reveals an underexplored limitation in output-level distillation: the unconditional score branch remains unsupervised, leaving the classifier-free guidance gap underdetermined in the student. This gap, amplified at every denoising step, admits degenerate solutions where both branches collapse toward identical predictions, rendering guidance ineffective despite low output-level training loss. This paper introduces DASH , a dual-branch distillation framework that independently supervises both score branches, uniquely specifying target branch outputs for each training sample through independent branch constraints, with an anchor term regularising conditional predictions toward ground-truth noise. The framework further introduces TIRT Transfer, which copies the teacher’s converged per-timestep importance curriculum into the student as a frozen prior, eliminating the need to relearn it within limited distillation budgets. Experiments on CIFAR-10 and CIFAR-100 demonstrate that 5.9 × 5.9{\times} compression maintains quality within 4 FID points of the teacher at 50-step DDIM sampling, considerably outperforming training from scratch with guidance fidelity well preserved. Ablation studies confirm that unconditional supervision is the dominant contribution, accounting for over 60% of total distillation gain. Curriculum transfer and anchor regularisation provide complementary benefit, together validating dual-branch constraints as empirically essential for guidance-preserving compression. 1 Introduction Denoising diffusion probabilistic models 7 have established themselves as the dominant framework for high-quality image synthesis 3 ; 9 , with classifier-free guidance (CFG) 8 enabling precise conditional control without auxiliary classifiers. CFG trains a single network to produce both conditional ( \bm{\varepsilon}^{y} ) and unconditional ( \bm{\varepsilon}^{\varnothing} ) predictions through stochastic label dropout, combining them at inference as \tilde{\bm{\varepsilon}}=\bm{\varepsilon}^{\varnothing}+w(\bm{\varepsilon}^{y}-\bm{\varepsilon}^{\varnothing}) , where the guidance gap \Delta=\bm{\varepsilon}^{y}-\bm{\varepsilon}^{\varnothing} is amplified at every denoising step to steer the generation trajectory. The quality of this amplification depends critically on both the magnitude and direction of \Delta : an undercalibrated gap suppresses class-specific sharpening, while a collapsed gap renders the guidance weight w entirely ineffective. Despite these benefits, CFG doubles per-step computational cost and requires substantial model capacity. Deploying it in resource-constrained settings therefore depends on parameter compression that preserves the calibrated guidance mechanism. Compressing CFG-guided diffusion models exposes a structural failure mode in existing distillation objectives. Supervising only the conditional branch or the composite guided prediction \tilde{\bm{\varepsilon}}_{S}\approx\tilde{\bm{\varepsilon}}_{T} imposes a single constraint on two independent score branches, leaving multiple branch decompositions consistent with the objective, including the degenerate solution where the guidance gap collapses ( \Delta_{S}{\to}0 ) and conditional and unconditional predictions converge, rendering CFG ineffective at inference even when the composite training objective is well-optimized. Any branch pair satisfying w\Delta_{S}=w\Delta_{T}+(\bm{\varepsilon}^{\varnothing}_{T}-\bm{\varepsilon}^{\varnothing}_{S}) achieves zero loss, and without direct supervision on the unconditional branch, implicit mechanisms such as weight decay cannot reliably prevent this collapse. Step-reduction approaches 27 ; 30 ; 11 ; 31 avoid this issue by retaining full teacher capacity, but leave memory footprint and per-step cost unchanged, motivating parameter compression as a complementary efficiency axis. This paper introduces DASH , which resolves underdetermination through independent branch supervision. The unconditional loss \mathcal{L}_{\mathrm{un}} directly constrains \bm{\varepsilon}^{\varnothing}_{S}\approx\bm{\varepsilon}^{\varnothing}_{T} and the imitation loss \mathcal{L}_{\mathrm{im}} matches \bm{\varepsilon}^{y}_{S}\approx\bm{\varepsilon}^{y}_{T} , together forming a fully determined joint objective whose unique minimiser matches both teacher branches simultaneously. A lightweight anchor term \mathcal{L}_{\mathrm{an}} regularises the conditional branch toward ground-truth noise, preventing drift during early training when the teacher target provides a weak signal. Ablation confirms that \mathcal{L}_{\mathrm{un}} is the dominant component: removing it alone collapses the guidance gap and accounts for 67% and 62% of total distillation gain on CIFAR-10 and CIFAR-100, establishing that unconditional supervision cannot be substituted by composite or conditional objectives. Beyond resolving underdetermination, DASH addresses a second challenge: the teacher’s converged per-timestep training curriculum is a learned asset that the distillation objective cannot recover. TIRT (Timestep-Importance Rebalanced Training) extends Min-SNR 5 with learnable per-timestep weights that concentrate training capacity at content-difficult mid-trajectory regions, improving teacher FID from 9.21 to 7.16 on CIFAR-10; the full teacher pipeline reaches FID 5.47 (Table 2 ). TIRT defines the teacher’s curriculum; TIRT Transfer hands that curriculum to the student as a frozen prior, since distillation gradients reward prediction-matching ease rather than content difficulty and would otherwise corrupt the schedule. Curriculum transfer contributes a further 22% / 17% of total distillation gain on CIFAR-10 and CIFAR-100 respectively (Table 4 ). Experiments on CIFAR-10 and CIFAR-100 validate the framework across 10 and 100 classes respectively. The 6.1M student achieves FID 8.87 and 10.47 at 5.9{\times} parameter compression, within 4 FID points of the 35.8M teacher, with \rho{=}0.91 guidance calibration. The cross-dataset ratio of 1.18{\times} for DASH against 1.51{\times} and 1.62{\times} for single-constraint baselines confirms that dual-branch supervision generalises robustly across label complexity. The calibration pair (\rho,\cos(\Delta)) provides a practical monitoring signal for guided compression beyond this framework. 2 Related Work Diffusion models and classifier-free guidance. Denoising diffusion models 7 ; 21 ; 3 ; 9 establish noise prediction as the dominant generative paradigm. Recent work refines guidance at inference: CFG++ 2 constrains score updates to the data manifold, and interval guidance 13 applies CFG selectively to mid-trajectory timesteps, motivating timestep-dependent guidance. These inference-time adjustments are orthogonal to model compression and do not address calibration under capacity reduction. Timestep weighting and curriculum transfer. Non-uniform timestep weighting substantially affects diffusion convergence. Min-SNR 5 applies fixed reweighting based solely on the noise schedule; perception-prioritised training 1 uses data-agnostic perceptual rankings. The gap DASH exploits is not in the weighting itself but in how it transfers: prior distillation methods do not initialise the student’s curriculum from the teacher’s converged weights, and distillation gradients corrupt rather than preserve it. Transferring a converged per-timestep weighting from teacher to student has not, to our knowledge, been explored in prior diffusion distillation work. Model compression and step-reduction distillation. Parameter-efficient diffusion requires either step reduction or model compression. Luhman et al. 17 introduce student–teacher distillation for diffusion; progressive distillation 27 , consistency models 30 ; 19 , and sampling-acceleration methods 15 ; 16 ; 28 ; 31 achieve few-step sampling but retain full teacher capacity. Meng et al. 20 address step-reduction at full capacity; adapter-based guidance distillation 23 approximates the composite guided prediction in a single forward pass by training lightweight adapters on a frozen base model, halving inference cost without reducing base model parameters. Both prior methods target inference speed; DASH instead reduces the student to 6.1M parameters while retaining the 50-step sampling budget, a complementary efficiency axis. Distribution matching distillation 31 aligns the output distribution but does not target parameter compression. Beyond step reduction, two prior works target parameter compression directly. BK-SDM 10 removes UNet blocks in Stable Diffusion and SnapFusion 14 targets mobile deployment; both operate at 512px in latent space with text conditioning and are not directly comparable to our class-conditional pixel-space setting. BOOT 4 enables data-free distillation but preserves teacher capacity. Parameter compression and step reduction are orthogonal axes; DASH addresses compression and is composable with step-reduction methods. Guided score distillation and underdetermination. Output-level distillation matches student predictions to teacher outputs. For guided models, supervising only the composite score \tilde{\bm{\varepsilon}}_{S}\approx\tilde{\bm{\varepsilon}}_{T} is structurally underdetermined, admitting degenerate solutions where \Delta_{S}{\to}0 nullifies CFG at inference. Score distillation via reparametrised DDIM 18 analyses such objectives theoretically but does not resolve this structural issue. Independent supervision of the unconditional score

Read the complete text