QDM: Quadtree-Based Region-Adaptive Sparse Diffusion Models for Efficient Image Super-Resolution
Donglin Yang, Paul Vicol, Xiaojuan Qi, Renjie Liao, Xiaofan Zhang · 2025
Read QDM: Quadtree-Based Region-Adaptive Sparse Diffusion Models for Efficient Image Super-Resolution by Donglin Yang, Paul Vicol, Xiaojuan Qi, Renjie Liao, Xiaofan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12015.txt.
Introduction
Yang and co-authors present QDM, a diffusion model that sharpens images only where detail matters. Standard super-resolution methods spend equal compute on every pixel. Flat areas get needless refinement while busy areas need more. QDM reads the low-quality input and builds a quadtree over it. Leaf blocks mark detail-rich zones that deserve work. Flat zones get minimal touch. Two streams, guided by that mask, balance quality against cost.
Tests covered many image types at high resolution. Medical scans, with their large flat regions, showed the method's strength. Outputs stayed faithful while wasted compute fell. The design suits any task where detail clusters in patches.
Catalog record
| CREATOR | Donglin Yang, Paul Vicol, Xiaojuan Qi, Renjie Liao, Xiaofan Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-12015
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
QDM: Quadtree-Based Region-Adaptive Sparse Diffusion Models for Efficient Image Super-Resolution Donglin Yang, Paul Vicol, Xiaojuan Qi, Renjie Liao, Xiaofan Zhang Abstract Deep learning-based super-resolution (SR) methods often perform pixel-wise computations uniformly across entire images, even in homogeneous regions where high-resolution refinement is redundant. We propose the Quadtree Diffusion Model (QDM), a region-adaptive diffusion framework that leverages a quadtree structure to selectively enhance detail-rich regions while reducing computations in homogeneous areas. By guiding the diffusion with a quadtree derived from the low-quality input, QDM identifies key regions—represented by leaf nodes—where fine detail is essential and applies minimal refinement elsewhere. This mask-guided, two-stream architecture adaptively balances quality and efficiency, producing high-fidelity outputs with low computational redundancy. Experiments demonstrate QDM’s effectiveness in high-resolution SR tasks across diverse image types, particularly in medical imaging (e.g., CT scans), where large homogeneous regions are prevalent. Furthermore, QDM outperforms or is comparable to state-of-the-art SR methods on standard benchmarks while significantly reducing computational costs, highlighting its efficiency and suitability for resource-limited environments. Our code is available at https://github.com/linYDTHU/QDM . 1 Introduction Super-resolution (SR) seeks to reconstruct high-resolution (HR) images from low-resolution (LR) inputs, an inherently ill-posed inverse problem due to the unknown and complex image formation processes in real-world scenarios. Existing approaches mitigate this challenge by designing empirical image formation pipelines and leveraging generative adversarial networks (GANs) [ 9 ] to approximate the LR-to-HR mapping [ 45 , 35 , 18 ] . Recent advances in diffusion models [ 11 , 30 , 31 ] have demonstrated superior image generation capabilities compared to GANs [ 6 ] , positioning them as promising tools for SR. Contemporary studies achieve this either by adapting diffusion frameworks for SR tasks [ 28 , 26 , 44 , 37 ] or exploiting pretrained text-to-image (T2I) models like Stable Diffusion [ 26 , 24 ] as priors to regularize the ill-posed reconstruction [ 20 , 40 , 34 , 39 , 43 , 41 ] . Despite recent advances, most SR methods apply uniform pixel-wise computations across the entire image, regardless of local content variations. While this type of approach effectively enhances details, it incurs unnecessary computational overhead, especially in homogeneous regions where pixel intensities are similar, i.e . , exhibiting redundant information. As illustrated in Fig. 1 , fine-grained details benefit the most from the SR process, whereas homogeneous regions require minimal processing. However, existing SR methods largely overlook this spatial adaptivity, leading to redundant computations. Figure 1 : The original high-resolution (HR) natural images collected from DIV2K [ 1 ] and medical images from SegRap2023 [ 22 ] , are downsampled to lower resolutions. The difference heatmaps reveal that only sparse detail-rich regions—such as edges and textures—require significant refinement, while large homogeneous areas remain largely unaffected. To address these inefficiencies, we propose an adaptive computational framework that selectively allocates computations to detail-rich regions while minimizing computation in homogeneous areas. Our approach resolves two key challenges: 1) identifying homogeneous regions that require minimal processing while preserving structures in detail-rich areas, and 2) designing an architecture capable of spatially adaptive computation . For the first challenge, we employ a quadtree decomposition strategy [ 29 ] . This approach recursively subdivides regions until they reach a predefined homogeneity threshold, resulting in variable-sized homogeneous regions at the leaf nodes. We can then leverage this quadtree prior to refining regions only where necessary, thus reducing redundant computations. For the second challenge, existing diffusion-based SR methods [ 11 , 31 , 26 , 44 ] typically rely on architectures, such as U-Net [ 27 ] or Transformer-based models [ 23 ] , which uniformly process all spatial regions without adaptivity. We instead propose a dual-stream architecture guided by the quadtree mask. Our design processes coarse-grained patches in upstream blocks to capture global context, while dynamically routing detail-rich regions to downstream blocks for enhancing fine-grained patches. Our approach provides two main advantages: (1) it localizes computationally intensive operations to detail-rich regions, and (2) it partitions selected tokens into parallelizable chunks, eliminating the need for full-feature-map storage. In summary, the main contributions of this work are twofold. First, we propose a hierarchical quadtree-based algorithm to efficiently identify detail-rich regions, and design mask-guided diffusion processes that selectively refine these regions while leveraging homogeneous areas as spatial-contextual priors. Second, we introduce a dual-stream architecture with dynamic resource allocation, enabling spatial-adaptive computation to improve processing efficiency. Extensive experimental results show that our QDM outperforms or matches state-of-the-art SR methods on both natural and medical imaging SR benchmarks, while reducing computational costs. 2 Related Works Diffusion-based Image Super-Resolution. Diffusion-based models for super-resolution (SR) have been widely explored and can be roughly categorized into two approaches: train-from-scratch and pretrained-prior-based methods. Train-from-scratch techniques, initiated by SR3 [ 28 ] , pioneered the application of diffusion models to SR. LDM [ 26 ] further enhanced computational efficiency by operating in a latent space, while ResShift [ 44 ] further streamlined the process to 15 diffusion steps via residue shifting. These methods are easy to retrain, and inherently versatile, making them broadly applicable across diverse scenarios. Another research direction leverages the generative power of pretrained text-to-image (T2I) models like Stable Diffusion [ 26 , 24 ] . Works like DiffBIR [ 20 ] , SeeSR [ 40 ] , and StableSR [ 34 ] , along with others [ 39 , 43 , 41 ] , employ various fine-tuning strategies to exploit diffusion priors for SR, achieving state-of-the-art performance. However, such methods rely heavily on large-scale T2I foundation models, limiting their adaptability to domains beyond natural images. Our approach falls within the train-from-scratch paradigm, prioritizing flexibility and broad applicability without using external pretrained models. Model Architecture for Diffusion Models. The rapid advancement of diffusion models has established U-Net architectures [ 27 ] as foundational to diffusion-based image synthesis. These architectures have demonstrated exceptional performance across tasks spanning denoising, super-resolution, and high-definition image generation [ 11 , 31 , 26 , 44 ] . Early frameworks like DDPMs [ 11 ] augmented U-Nets with self-attention layers at coarse resolutions to enhance feature interaction. Later studies [ 12 , 24 ] emphasize the critical role of additional self-attention in low-resolution stages for scaling to high-dimensional outputs. Concurrently, the Swin Transformer [ 21 ] introduces shifted-window self-attention, which confines attention to localized windows while preserving global context through window shifting. This approach has been successfully integrated into diffusion frameworks like ResShift [ 44 ] and SinSR [ 37 ] to balance scalability and efficiency. Recently, DiT [ 23 ] has emerged as a milestone, replacing U-Nets entirely with pure Transformer backbones, showcasing unprecedented scalability for large-scale generative tasks. Quadtree Representations. Quadtrees are hierarchical data structures with a long history in image processing [ 29 ] . Early works, such as [ 2 , 8 ] , applied quadtrees to semantic segmentation tasks using graphical models. However, in the deep learning era, their use has been primarily focused on reducing computation and memory costs. For instance, [ 13 ] presents a quadtree-based convolutional network for tasks involving sparse, binary input images. Similarly, [ 5 ] leverages an efficient quadtree representation of segmentation masks to reduce memory consumption with minimal accuracy loss. Quadtree attention [ 32 ] computes attention in a