Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution

Bin Chen, Weiqi Li, Shijie Zhao, Xuanyu Zhang, Junlin Li, Li Zhang, Jian Zhang · 2026

Read Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution by Bin Chen, Weiqi Li, Shijie Zhao, Xuanyu Zhang, Junlin Li, Li Zhang, Jian Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00458.txt.

Introduction

Chen and six colleagues slim diffusion video super-resolution, where rich detail comes at the price of slow multi step sampling. One step nets stay heavy, and plain adversarial compression loses the balance between spatial detail and time consistency. Their improved method distils a large diffusion Transformer teacher into a pruned compact backbone, adding light temporal convolutions and a dual head scheme that judges detail and consistency apart. The compressed model cuts parameters by 95 percent and runs eight times faster than its teacher, keeping quality competitive.

Catalog record

CREATORBin Chen, Weiqi Li, Shijie Zhao, Xuanyu Zhang, Junlin Li, Li Zhang, Jian Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62983
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.00458

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution Bin Chen, Weiqi Li, Shijie Zhao, Xuanyu Zhang, Junlin Li, Li Zhang, Jian Zhang Abstract While many diffusion models have achieved impressive results in real-world video super-resolution (Real-VSR) by generating rich and realistic details, their reliance on multi-step sampling leads to slow inference. One-step networks like SeedVR2, DOVE, and DLoRAL alleviate this through condensing generation into one single step, yet they remain heavy, with billions of parameters and multi-second latency. Recent adversarial diffusion compression (ADC) offers a promising path via pruning and distilling these models into a compact AdcSR network, but directly applying it to Real-VSR fails to balance spatial details and temporal consistency due to its lack of temporal awareness and the limitations of standard adversarial learning. To address these challenges, we propose an improved ADC method for Real- VSR . Our approach distills a large diffusion Transformer (DiT) teacher DOVE equipped with 3D spatio-temporal attentions, into a pruned 2D Stable Diffusion (SD)-based AdcSR backbone, augmented with lightweight 1D temporal convolutions, achieving significantly higher efficiency. In addition, we introduce a dual-head adversarial distillation scheme, in which discriminators in both pixel and feature domains explicitly disentangle the discrimination of details and consistency into two heads, enabling both objectives to be effectively optimized without sacrificing one for the other. Experiments demonstrate that the resulting compressed AdcVSR model reduces complexity by 95% in parameters and achieves an 8 × \times acceleration over its DiT teacher DOVE, while maintaining competitive video quality and efficiency. † † footnotetext: ∗ Equal contribution. ⋄ Project lead. † Corresponding author. 1 Introduction Figure 1: Comparison of methods in compressing diffusion networks for Real-VSR. (a) Traditional ADC ( 5 ) distills an SD network with 2D spatial attentions into a pruned student using a single adversarial signal without temporal modeling, suffering from frame flickering. (b) Our improved ADC distills a larger DiT-based teacher with heavier 3D spatio-temporal attention into the same 2D student, augmented by 1D temporal convolutions, using dual-head discriminators {\mathcal{D}} in pixel and feature domains. Through disentangling the discriminations of detail richness and temporal consistency into different heads, it balances the optimization of both. Real-world video super-resolution (Real-VSR) ( 51 ; 41 ) is a fundamental and long-standing problem in computer vision. It targets at recovering high-resolution (HR) videos {\mathbf{x}_{\text{HR}}} from their low-resolution (LR) counterparts {\mathbf{x}_{\text{LR}}} degraded by unknown factors in real-world cases. Traditional non-generative ( 77 ; 2 ; 75 ; 4 ; 3 ; 58 ; 29 ) and generative adversarial network (GAN)-based ( 7 ; 38 ; 72 ) approaches have achieved notable progress, yet most of them struggle under complex, mixed degradations, producing over-smoothed results or artifacts. To enhance the detail richness of super-resolution outputs {\mathbf{x}_{\text{SR}}} , many Real-VSR studies ( 87 ; 74 ; 15 ; 55 ; 70 ; 69 ; 21 ; 57 ; 65 ; 86 ; 62 ; 1 ) have already developed diffusion-based approaches which can generate video frames with richer and more realistic details. However, these methods are hindered by long inference time, as they need multiple sampling steps. Recently, researchers have shifted focus to one-step diffusion ( 63 ; 68 ; 71 ; 34 ) for achieving efficient and high-quality Real-VSR. Building on the pretrained Stable Diffusion (SD) models ( 43 ) , originally developed for image generation, UltraVSR ( 36 ) enhances the temporal consistency in {\mathbf{x}_{\text{SR}}} by propagating and fusing features along the temporal dimension, while DLoRAL ( 50 ) aligns the structure of the previous frame with the current one using the estimated optical flow between them. Building on video diffusion models, SeedVR2 ( 54 ) progressively distills a pretrained 64-step SeedVR ( 55 ) into one sampling step and further enhances it through adversarial post-training. DOVE ( 6 ) adapts pretrained CogVideoX networks ( 76 ) to Real-VSR by fine-tuning them on a curated high-quality video dataset. Despite these advances, such approaches still suffer from high complexity due to large-scale parameters and heavy computation. Meanwhile, two recent methods, AdcSR ( 5 ) and TinySR ( 10 ) , have explored compressing the diffusion networks OSEDiff ( 68 ) and TSD-SR ( 9 ) via structural pruning and adversarial distillation to reduce complexity for real-world image super-resolution (Real-ISR), as shown in Fig. 1 (a). However, it is non-trivial to extend them to Real-VSR. One may use them to compress Real-VSR networks such as SeedVR2, DOVE, and DLoRAL, but two challenges arise: the compressed models are still too large, or video quality is compromised due to the conflicts between optimizing detail richness and temporal consistency ( 50 ) , which are difficult to balance with existing distillation methods. Therefore, it remains underexplored and of great value to design more effective compression approaches for diffusion-based Real-VSR. To improve effectiveness, in this paper, we propose AdcVSR , a novel Real- VSR network that compresses the one-step model DOVE using an improved method of A dversarial D iffusion C ompression (ADC) ( 5 ) . Unlike previous networks that rely on computationally costly 3D spatio-temporal attentions or additional frame alignment modules, we hypothesize that a 2D diffusion backbone ( e.g. , SD2.1) is sufficient to generate rich details, while temporal consistency can be maintained with a few lightweight 1D temporal convolutional layers, and that their combination is also effective in removing degradations. Guided by this insight, as Fig. 1 (b) illustrates, AdcVSR adopts the same pruned SD2.1 backbone as AdcSR, augmented with 1D temporal convolution layers, achieving significantly lower complexity than its heavy 3D teacher DOVE. To improve video quality and address the conflict between optimizing details and consistency ( 50 ) , we introduce a new adversarial distillation scheme that leverages the strong teacher DOVE together with numerous temporally consistent real videos and detail-rich real images. Specifically, two discriminators are employed for adversarial learning: one discriminating in a feature space of variational autoencoder (VAE) decoder and the other in pixel space, each with a “detail” head and a “consistency” head sharing a common backbone, to separately assess the realism of spatial details and temporal consistency. This enables the student to generate super-resolution results which are both detail-rich and temporally consistent. By integrating our “2D + 1D” architectural design with the dual-head, dual-discriminator adversarial distillation scheme, AdcVSR effectively compresses DOVE, yielding substantial efficiency gains while maintaining competitive video quality. Our contributions are summarized as follows: ❑(1) We propose a novel improved ADC approach that combines an effective network design with adversarial distillation to compress a heavy Real-VSR model into an efficient diffusion-GAN hybrid. ❑(2) We demonstrate that a 2D image diffusion backbone augmented with lightweight 1D temporal convolutions can effectively learn Real-VSR mapping from 3D diffusion Transformer (DiT) teacher. ❑(3) We introduce a new adversarial distillation scheme that decouples the discriminations of details and consistency into two heads sharing a common network backbone, applied in both VAE decoder’s feature space and the pixel space. This design enables balanced optimization, avoiding collapse into either over-smoothed outputs (loss of spatial details) or flickering (loss of temporal consistency). ❑(4) Extensive experiments show that our AdcVSR model reduces parameters by 95% and achieves an 8 \times acceleration over its teacher DOVE, while maintaining competitive performance on Real-VSR and striking a balance

Read the complete text