Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution

Jingkai Wang, Yixin Tang, Jue Gong, Jiatong Li, Shu Li, Libo Liu, Jianliang Lan, Yutong Liu, Yulun Zhang · 2026

Read Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution by Jingkai Wang, Yixin Tang, Jue Gong, Jiatong Li, Shu Li, Libo Liu, Jianliang Lan, Yutong Liu, Yulun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06275.txt.

Introduction

Wang and his team put forward StrSR, a one step adversarial distillation method with spectral and path regularization for real world image super resolution with diffusion transformers. Sampling is costly and step by step, and past one step methods suffer trajectory mismatch plus severe grid like periodic artifacts. An asymmetric telling-apart distillation architecture bridges that gap, while a frequency matching plan curbs such flaws from high frequency leakage. Tests show top scores across number metrics and visual perception in the real world.

Catalog record

CREATORJingkai Wang, Yixin Tang, Jue Gong, Jiatong Li, Shu Li, Libo Liu, Jianliang Lan, Yutong Liu, Yulun Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39284
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06275

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution Jingkai Wang, Yixin Tang, Jue Gong, Jiatong Li, Shu Li, Libo Liu, Jianliang Lan, Yutong Liu, Yulun Zhang Abstract Diffusion transformer (DiT) architectures show great potential for real-world image super-resolution (Real-ISR). However, their computationally expensive iterative sampling necessitates one-step distillation. Existing one-step distillation methods struggle with Real-ISR on DiT. They suffer from fundamental trajectory mismatch and generate severe grid-like periodic artifacts. To tackle these challenges, we propose StrSR, a novel one-step adversarial distillation framework featuring spectral and trajectory regularization. Specifically, we propose an asymmetric discriminative distillation architecture to bridge the trajectory gap. Additionally, we design a frequency distribution matching strategy to effectively suppress DiT-specific periodic artifacts caused by high-frequency spectral leakage. Extensive experiments demonstrate that StrSR achieves state-of-the-art performance in Real-ISR, across both quantitative metrics and visual perception. The code and models will be released at https://github.com/jkwang28/StrSR . 1 Introduction Real-world image super-resolution (Real-ISR) is a typical and highly practical problem in low-level vision. It originates from the classic super-resolution (SR), i.e. , the input low-resolution (LR) image is upscaled to a high-resolution (HR) image, with only Bicubic degradation [ 15 , 83 , 12 , 11 ] . However, in real-world scenarios, the degradation is much more complex and unknown, making Real-ISR a highly challenging and ill-posed problem [ 80 , 61 , 31 ] . Substantial efforts have been devoted to Real-ISR, with a particular focus on generative approaches like GAN-based methods [ 27 , 62 , 80 , 61 ] . These methods continuously improve and deliver steadily better performance. With the emergence and rapid development of diffusion models [ 51 , 48 , 18 ] , Real-ISR reaches a new stage. Recent studies [ 69 , 33 , 76 ] increasingly adopt diffusion models for Real-ISR, leveraging their strong generative capability and scalability. Very recently, extensive research [ 20 ] has highlighted the scaling laws of diffusion models. These studies demonstrate that diffusion transformer (DiT) [ 47 ] architectures significantly outperform traditional UNets as model size and data volume grow. Given this superior capacity and predictable scaling behavior, adopting DiT for image SR is an essential and inevitable trend for the future of low-level vision. However, generating high-quality images with diffusion models usually requires dozens or hundreds of steps, which is computationally expensive. To tackle this issue, many works [ 75 , 77 , 59 , 16 ] distill pretrained text-to-image (T2I) models for image-to-image tasks. They use either score distillation [ 67 , 16 ] , or adversarial diffusion distillation [ 34 , 29 ] , and achieve significant performance. (a) Input (b) TSD-SR [ 16 ] (c) FluxSR [ 28 ] (d) StrSR (ours) Figure 1 : For a self-captured cat image with dense fur, TSD-SR [ 16 ] and FluxSR [ 28 ] suffer from heavy grid-like artifacts, whereas our StrSR restores realistic details. Despite achieving promising results, we observe that current methods still suffer when combining DiT and one-step distillation. As shown in Fig. 1 , those methods [ 16 , 28 ] suffer from severe artifacts and inferior performance. To analyze the reason, there is a fundamental trajectory mismatch. It is difficult to quickly and stably align the new generation trajectory (from LR to HR images) with the pretrained trajectory (from pure noise to HR images). Furthermore, during this forced trajectory alignment, models struggle to maintain photo-realistic outputs. Forcing a single large evaluation step heavily exacerbates grid-like artifacts, a problem that is uniquely severe in DiT architectures. To tackle these challenges, we propose spectral and trajectory regularization for diffusion transformer super-resolution (StrSR). This one-step adversarial distillation framework unlocks photo-realistic generation while alleviating the severe periodic artifacts inherent to DiT models. Our contributions are as follows: 1. We propose asymmetric discriminative distillation to bridge the gap between multi-step and one-step generation. To bypass heavy progressive distillation and prevent model collapse, we employ a pretrained CLIP-ConvNeXt as a lightweight, texture-sensitive discriminator. Combined with a relativistic average GAN loss and approximated R1 regularization, this asymmetric design ensures stable training and superior semantic-aligned texture recovery. 2. We design frequency distribution matching for artifact suppression. To address the grid-like periodic artifacts arising from high-frequency spectral leakage in DiT patches, we introduce the frequency distribution loss (FDL). By minimizing the sliced Wasserstein distance between the predicted and target features across both amplitude and phase components, FDL provides a spectral constraint that effectively suppresses these artifacts. 3. We develop a comprehensive dual-encoder architecture that seamlessly integrates spatial adversarial distillation and spectral domain optimization. Empowered by this unified framework, StrSR achieves state-of-the-art (SOTA) performance in one-step Real-ISR. Extensive experiments confirm its superiority across both quantitative metrics and visual quality. 2 Related Works 2.1 Diffusion-based Real-ISR Recently, the success in pretrained diffusion models [ 51 , 48 , 5 , 65 ] enables researchers to leverage pretrained weights for downstream tasks like real-world image super-resolution. Built upon stable diffusion models [ 51 , 48 ] based on UNet architecture, StableSR and DiffBIR inject low-resolution information into the model through controlnet-like methods. OSEDiff [ 67 ] directly uses LR image as input and adopts variational score distillation (VSD) [ 64 , 45 ] to distill a pretrained diffusion model for one-step super-resolution. InvSR [ 77 ] treats SR as a diffusion inversion problem to better guide image priors in large pretrained diffusion models. HYPIR [ 34 ] harnesses adversarial diffusion distillation (ADD) to distill SD-XL [ 48 ] into one-step image SR model. With the prevalence of diffusion transformer (DiT) [ 47 ] in the text-to-image (T2I) task, some works also explored the possibility to utilize DiT architecture in image SR. TSD-SR [ 16 ] proposes target score distillation (TSD) and a distribution-aware sampling module to handle the artifacts caused by VSD in training. DiT4SR [ 17 ] trains DiT model from scratch based on SD3 [ 18 ] MM-DiT architecture with additional LR-residual. FluxSR [ 28 ] explores one-step DiT-based SR models in terms of flow trajectory [ 37 , 35 ] . 2.2 Diffusion Model Acceleration Diffusion models have strong generative abilities. However, they suffer from high computational costs during inference. This issue limits their practical deployment. Many methods have been proposed to speed up this process. One major category is deterministic methods using a deterministic probability flow. They aim to predict the exact output of a teacher model using fewer steps. Examples include progressive distillation [ 53 ] , consistency distillation [ 57 , 58 , 41 , 42 ] , and rectified flow [ 37 ] , and they are easy to train. However, they usually need more steps ( e.g. , 8 steps) to generate high-quality results. Another category is distributional methods. Instead of predicting exact outputs, they approximate the data distribution of the teacher model. This group includes adversarial training [ 10 , 24 , 44 , 54 , 70 ] and score distillation [ 43 , 74 , 64 ] . Some approaches even combine both techniques [ 73 , 55 , 8 ] . Recently, these acceleration techniques have been applied to diffusion-based SR tasks. These works directly fine-tune pretrained diffusion models using score or adversarial distillation. They achieve superior performance. Notable examples in image

Read the complete text