Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models

Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, and Xianglong Liu · 2026

Read Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models by Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, and Xianglong Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00658.txt.

Introduction

Du and co-authors work on cheap video generation in this 2026 paper. Big video diffusion models make fine clips, but each clip needs many steps and a large model in memory. The team shrinks the Wan2.2 model, which creates video from text with two expert branches.

Their recipe mixes two tools. Few-step distillation trains a small student to mimic the big model in far fewer steps. Low-bit quantization then stores its weights with very few bits. They tune each expert branch on its own and shield the first layers, which are touchy. The small model is tuned after distillation, not before, so its values match real use. The authors report that the shrunk model beats the full one at 8 and 20 steps on average. The 20-step setup gives the best mix of quality and speed.

Catalog record

CREATORJinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, and Xianglong Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00658

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models Thanks: This technical report describes our submission to Sub-Challenge 1: W4A4 Quantization for Inference (HiF4 / MXFP4) of the GCC Low-Bit-width Large Model Quantization Challenge, an official challenge track of IEEE ICME 2026. Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, and Xianglong Liu Abstract Large video diffusion models achieve strong visual quality but remain expensive to deploy because each sample requires many denoising steps and a large resident parameter footprint. This paper studies a deployment-oriented compression pipeline for Wan2.2-T2V-A14B by combining few-step distribution-matching distillation with low-bit quantization. The pipeline follows the model’s dual-expert denoising route, calibrates the high-noise and low-noise branches separately, protects sensitive entrance layers, and uses HiF4-style low-bit representation to improve dynamic-range coverage. Quantization is calibrated on the distilled few-step student rather than on the original long-step trajectory, reducing activation-distribution mismatch during inference. The proposed co-design keeps the quantized model close to the same-step full-precision model and surpasses the original full-precision baseline at 8 and 20 steps on average. The 20-step setting gives the best quality-efficiency trade-off in the tested configurations. I Overall Approach I-A Background and Objectives Diffusion models have become a dominant framework for image and video generation, but their iterative denoising process is expensive at deployment time [ 5 , 6 ] . Wan2.2 is an open video generation family whose A14B variant uses a dual-expert denoising architecture to improve capacity without activating all parameters at every step [ 16 ] . This design improves generation quality but leaves two bottlenecks: long diffusion trajectories and large model memory. The goal of this work is to compress and accelerate an existing high-quality model through few-step distillation, low-bit quantization, and closed-loop video-quality evaluation. The deployment target is a model that preserves subject identity, motion continuity, frame fidelity, and prompt alignment under limited GPU memory and latency budgets. Reducing only the number of sampling steps leaves the model expensive per step, whereas quantizing the original long-step model without matching the deployment trajectory may introduce activation-distribution mismatch and increase temporal error propagation. The core objective is therefore to first learn a shorter trajectory and then quantize under the activation distribution induced by that trajectory. I-B End-to-End Technical Pipeline The pipeline is organized as model adaptation \rightarrow few-step distillation \rightarrow low-bit quantization \rightarrow calibration construction \rightarrow generation \rightarrow evaluation . Implementation is built around Diffusers and LLMC [ 15 , 4 ] . First, the original model is adapted to a short denoising trajectory. Then, quantization calibration is performed under the same few-step deployment setting, so the quantizer observes the activation statistics it will meet at inference time. Finally, the full-precision and compressed models generate videos with matched prompts and sampling settings. The automatic evaluation stream computes VBench scores under a unified protocol, while the OpenS2V-based generation stream is used to produce submission videos for the human-scored competition metric. Two principles guide the implementation. First, all calibration and replacement operations must follow the true high-noise and low-noise expert routes of Wan2.2 instead of treating the model as a dense trunk. Second, evaluation must compare models under matched generation conditions so that score changes can be attributed to the compression strategy rather than to prompt selection, sampling settings, or output organization. The pipeline is therefore both a compression method and a verification loop. II Model, Data, and Evaluation Framework II-A Overview of Wan2.2 Wan2.2-T2V-A14B decomposes denoising into a high-noise expert and a low-noise expert. The former handles global layout and motion planning, while the latter refines texture, lighting, and temporal detail. Although the full model contains about 27B parameters, only one roughly 14B expert is active at a given denoising step [ 16 ] . Fig. 1: Illustration of the Wan2.2 MoE architecture. The branch switch makes compression different from single-trunk diffusion models. Calibration data, sensitive-layer protection, and layer replacement must follow the actual high-noise and low-noise execution paths; otherwise, the quantizer mixes incompatible activation distributions and amplifies low-bit error. II-B Dataset Preparation and Preprocessing OpenS2V-5M is used as the prompt source for the OpenS2V-based submission stream and as a source for constructing calibration prompts, because it provides large-scale text-video data [ 20 ] . The raw corpus is training-oriented, so it is converted into a compact inference stream. For each sample, the pipeline extracts the most subject-centered caption, preferring face_cap_qwen and falling back to cap[0] when necessary. Samples with low aesthetic quality or unstable motion are filtered out, and each retained item is mapped to { prompt , negative_prompt }, with the negative prompt left empty when no explicit annotation is available. This preprocessing keeps calibration and evaluation prompts aligned with the subject-consistency demands of video generation. The filtering stage is intentionally conservative. The calibration set should expose the model to ordinary subject and motion variation, but it should not be dominated by corrupted captions, low-quality videos, or extreme motion that makes evaluator scores noisy. Metadata fields that are irrelevant to text-to-video inference are removed, while the prompt text is kept close to the subject description rather than rewritten into a long synthetic instruction. This reduces the gap between calibration inputs and the prompts used by the VBench evaluation loop. II-C VBench Evaluation Metrics Evaluation uses VBench, which decomposes video quality into interpretable dimensions [ 7 ] . We focus on five metrics: Subject Consistency, Aesthetic Quality, Imaging Quality, Overall Consistency, and Motion Smoothness. Subject Consistency uses DINO features to compare adjacent frames and first-frame anchors [ 1 ] . Aesthetic Quality uses CLIP features and a LAION aesthetic predictor [ 11 , 13 ] ; Imaging Quality uses MUSIQ [ 8 ] ; Overall Consistency compares video and text features through ViCLIP/InternVideo representations [ 17 ] ; and Motion Smoothness uses AMT-based frame interpolation [ 9 ] . Together, these metrics cover subject identity, perceptual quality, frame fidelity, prompt alignment, and temporal continuity. TABLE I: VBench dimensions used in the experiments. Metric Evaluator Main sensitivity SC DINO features subject identity over time AQ CLIP + LAION predictor composition and visual appeal IQ MUSIQ blur, artifacts, and detail fidelity OC ViCLIP/InternVideo text-video semantic alignment MS AMT interpolation temporal continuity and motion jitter The selected dimensions are chosen because they correspond to the most visible failure modes after compression. A quantized video model may still produce sharp individual frames while the subject slowly drifts, or it may preserve the main subject while reducing prompt faithfulness. Reporting these dimensions separately avoids hiding such trade-offs behind a single average. The average score is still useful as a compact summary, but per-metric values are needed to determine whether the compression pipeline is deployment-ready. II-D HiFloat4 Number Format HiF4 is a 4-bit block floating-point format designed for low-bit inference [ 10 ] . Each unit stores 64 signed 4-bit values plus 32 bits of shared scaling metadata, giving an average cost of 4.5 bits per value. The metadata contains a three-level scaling hierarchy: \underset{\text{Level-1:\ E6M2}}{\underbrace{8\text{bits}}}+\underset{\text{Level-2:\ E1\_8}}{\underbrace{8\times 1\text{bits}}}+\underset{\text{Level-3:\ E1\_16}}{\underbrace{16\times 1\text{bits}}}=32\text{bits}. Fig. 2: Three-level scaling hierarchy in HiF4. Global and local scaling. The first level stores an E6M2 base scale, X_{\text{E6M2}}=2^{E-48}\times 1.M. With exponent bias 48, the E6M2 base scale provides the coarse exponent range. The second and

Read the complete text