TinySR: Pruning Diffusion for Real-World Image Super-Resolution

Linwei Dong Qingnan Fan Yuhang Yu Qi Zhang, Jinwei Chen Yawei Luo Changqing Zou · 2025

Read TinySR: Pruning Diffusion for Real-World Image Super-Resolution by Linwei Dong Qingnan Fan Yuhang Yu Qi Zhang, Jinwei Chen Yawei Luo Changqing Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17434.txt.

Introduction

Dong and co-authors shrink diffusion models into TinySR for real-world photo upscaling on the fly. Real-world inputs suffer mixed damage: noise, blur, and compression artifacts. Diffusion models restore detail well, but their step-by-step denoising is slow and costly.

One-step distillation speeds things up, yet the models stay large and over-parameterized. TinySR prunes depth with a dynamic activation scheme and an expansion-corrosion strategy. Its image codec is compressed by channel pruning, attention removal, and light convolutions. Time and prompt modules are cut, and pre-caching speeds the rest. The result aims for real-time speed with quality kept intact.

Catalog record

CREATORLinwei Dong Qingnan Fan Yuhang Yu Qi Zhang, Jinwei Chen Yawei Luo Changqing Zou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-17434

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TinySR: Pruning Diffusion for Real-World Image Super-Resolution Linwei Dong Qingnan Fan Yuhang Yu Qi Zhang, Jinwei Chen Yawei Luo Changqing Zou Abstract Real-world image super-resolution (Real-ISR) focuses on recovering high-quality images from low-resolution inputs that suffer from complex degradations like noise, blur, and compression. Recently, diffusion models (DMs) have shown great potential in this area by leveraging strong generative priors to restore fine details. However, their iterative denoising process incurs high computational overhead, posing challenges for real-time applications. Although one-step distillation methods, such as OSEDiff and TSD-SR, offer faster inference, they remain fundamentally constrained by their large, over-parameterized model architectures. In this work, we present TinySR , a compact yet effective diffusion model specifically designed for Real-ISR that achieves real-time performance while maintaining perceptual quality. We introduce a Dynamic Inter-block Activation and an Expansion-Corrosion Strategy to facilitate more effective decision-making in depth pruning. We achieve VAE compression through channel pruning, attention removal and lightweight SepConv . We eliminate time- and prompt-related modules and perform pre-caching techniques to further speed up the model. TinySR significantly reduces computational cost and model size, achieving up to 5.68 × \times speedup and 83% parameter reduction compared to its teacher TSD-SR, while still providing high quality results. 1 Introduction Real-world image super-resolution (Real-ISR) ( Zhang et al. 2021 ; Wang et al. 2021 ) aims to reconstruct high-fidelity images from low-quality observations corrupted by compound degradations including noise contamination, nonlinear blur, and compression artifacts. Recently, diffusion models (DMs) have demonstrated significant promise for Real-ISR by leveraging their powerful priors to effectively address complex degradation patterns while recovering realistic textures and details ( Ho, Jain, and Abbeel 2020 ; Nichol and Dhariwal 2021 ) . Their impressive performance over GAN-based Real-ISR methods ( Liang, Zeng, and Zhang 2022 ) has driven their widespread adoption in practical downstream applications. However, for resource-constrained environments, the iterative sampling nature and high computational demands of DMs fundamentally limit their deployment ( Chen et al. 2025 ) . Significant efforts in reducing diffusion model sampling steps have dramatically improved the inference latency of DM-based Real-ISR approaches. Recent advancements in efficient Real-ISR models, such as OSEDiff ( Wu et al. 2024a ) and TSD-SR ( Dong et al. 2025 ) seek to condense the denoising process into a single step by carefully designed distillation while preserving high-quality restoration performance. However, these methods still depend on large pre-trained models, which require significant computational resources and footprint, posing challenges for real-time applications and edge-device deployment. There is a critical demand for more efficient and compact models that can be readily achieved while maintaining competitive performance. Figure 1: A comprehensive comparison of recent Real-ISR models in terms of visual quality, inference time, computational cost (MACs), and parameter count, highlighting the superior efficiency and performance of our proposed method. Diffusion compression primarily involves several key techniques, including operator optimization ( Dao et al. 2022 ; Teng et al. 2024 ) , precision quantization ( He et al. 2023 ; Li et al. 2023a ) , width pruning ( Wang et al. 2023 ; Castells et al. 2024 ) , and depth pruning ( Daniel Verdú 2024 ; Fang et al. 2024 ) . Depth pruning serves as a simple but effective technique, favored for its linear acceleration and straightforward implementation. The conventional paradigm for depth pruning is dependent upon empirical ( Kim et al. 2024 ) or importance-based metrics ( Men et al. 2024 ; Han et al. 2015 ) to guide layer selection, yet it overlooks the recoverability of the model’s performance on its target task after being pruned. Instead of relying on static importance scores, probability-based mask learning aims to iteratively refine this sampling distribution, such that layers with greater performance recoverability are more likely to be sampled. However, in ultra-deep networks, mask learning confronts a combinatorial explosion in the search space, which results in prohibitive optimization complexity ( Fang et al. 2024 ) . Beyond deep architecture limitation, DMs-based Real-ISR models exhibit two other computational bottlenecks: high overhead from VAE (Variational Auto-Encoder) and inefficiencies from redundant conditions during super-resolution. AdcSR ( Chen et al. 2024a ) eliminates the VAE encoder by employing a PixelUnshuffle operation ( Shi et al. 2016 ) . However, fine-grained channel pruning is required in denoising networks to align channel dimensions, which significantly increases overall complexity and coupling between its components. Regarding prompts and time embeddings, recent studies ( Dong et al. 2025 ; Chen et al. 2024a ) indicate that these conditions contribute minimally to the one-step Real-ISR model, suggesting that a more efficient model can be achieved by eliminating these inputs and related modules. Based on these analyses, we propose TinySR , a compact yet effective DMs-based Real-SR model that eliminates computational redundancies in TSD-SR while maintaining restoration quality. Following mask learning, we propose identifying candidate layers that exhibit high-performance recoverability. We partition the network into non-overlapping blocks and introduce sets of learnable probabilities p(m) for each blcok to constrain search space. We propose Dynamic Inter-block Activation , a method that leverages the learnable probability q(t) for soft boundary exploration, and introduce a novel Expansion-Corrosion Strategy to determine the optimal pruning scheme. These proposed techniques involve a strategic trade-off between optimization complexity and exploratory potential. Furthermore, we propose several strategies to further compress the Real-ISR model. To lighten the VAE, we perform channel-wise pruning, remove its computationally intensive attention modules, and replace its standard convolutions with depthwise separable convolutions ( Howard et al. 2017 ) . We eliminate time- and prompt-related modules to further enhance computational efficiency. Extensive experiments on standard Real-ISR benchmarks demonstrate that our method is up to 5.68 \times faster, with 84% MAC and 83% parameter reductions compared to its teacher, TSD-SR, yet preserves strong perceptual quality ( Figure 1 ) and comparable quantitative results ( Figure 2 ) . Overall, our contribution is summarized as follows: • A Real-ISR model called TinySR that achieves 5.68 \times speedup and 83% parameter reduction compared to its teacher, while maintaining a strong perceptual quality. • A novel depth pruning method that incorporates Dynamic Inter-block Activation and Expansion-Corrosion Strategy to enable more effective pruning decisions. • Component streamlining strategies incorporate lightweight VAE, redundant conditional structures pruning, and modulation parameters pre-caching. Figure 2: Performance and efficiency comparison among DMs-based Real-ISR methods on an NVIDIA V100 GPU. All metrics are evaluated on the RealSR benchmark. TinySR achieves the fastest inference, lightest computation (low MACs) and commendable performance (high CLIPIQA). 2 Related Work Real-World Image Super-Resolution. Real-world image super-resolution (Real-ISR) addresses the challenges of reconstructing high-resolution images from low-quality inputs affected by complex, unknown degradations. Early approaches like BSRGAN ( Zhang et al. 2021 ) and Real-ESRGAN ( Wang et al. 2021 ) pioneered synthetic degradation modeling using random blur, noise, and compression patterns to enhance generalization. While these methods improved the model’s performance, they often introduced undesirable artifacts. The emergence of diffusion models, particularly Stable Diffusion ( Rombach et al. 2022 ; Esser et al. 2024 ) , marked a significant advancement in perceptual quality. Techniques incorporating StableSR ( Wang, Chan, and Loy 2023 ) , DiffBIR ( Lin et al. 2023 ) and SeeSR ( Wu et al. 2024b ) demonstrated remarkable results in SR tasks, but their

Read the complete text