Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessment
Guanglu Dong Xiangyu Liao Mingyang Li Guihuan Guo Chao Ren College of Electronics and Information Engineering, Sichuan University, China {dongguanglu, liaoxiangyu1, 2024222055110, guoguihuan}@stu.scu.edu.cn, [email protected] · 2025
Read Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessment by Guanglu Dong Xiangyu Liao Mingyang Li Guihuan Guo Chao Ren College of Electronics and Information Engineering, Sichuan University, China {dongguanglu, liaoxiangyu1, 2024222055110, guoguihuan}@stu.scu.edu.cn, [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19295.txt.
Introduction
Guanglu Dong and co-authors use meaning-level traits to sharpen image upscaling, in this 2025 paper. Adversarial upscalers often judge whole images crudely and miss fine texture tied to content. Their method instead judges the middle semantic features of an image model.
One discriminator compares those features pixel by pixel between upscaled and true high-quality images. A second uses learned text prompt pairs to judge the model's top-level output features. Together they push the upscaler toward lifelike, content-true textures. As a bonus, the trained parts yield a quality scorer that needs no human ratings and beat prior such scorers with no extra training.
Catalog record
| CREATOR | Guanglu Dong Xiangyu Liao Mingyang Li Guihuan Guo Chao Ren College of Electronics and Information Engineering, Sichuan University, China {dongguanglu, liaoxiangyu1, 2024222055110, guoguihuan}@stu.scu.edu.cn, [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19295
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessment Guanglu Dong Xiangyu Liao Mingyang Li Guihuan Guo Chao Ren College of Electronics and Information Engineering, Sichuan University, China {dongguanglu, liaoxiangyu1, 2024222055110, guoguihuan}@stu.scu.edu.cn, [email protected] Abstract Generative Adversarial Networks (GANs) have been widely applied to image super-resolution (SR) to enhance the perceptual quality. However, most existing GAN-based SR methods typically perform coarse-grained discrimination directly on images and ignore the semantic information of images, making it challenging for the super resolution networks (SRN) to learn fine-grained and semantic-related texture details. To alleviate this issue, we propose a semantic feature discrimination method, SFD, for perceptual SR. Specifically, we first design a feature discriminator (Feat-D), to discriminate the pixel-wise middle semantic features from CLIP, aligning the feature distributions of SR images with that of high-quality images. Additionally, we propose a text-guided discrimination method (TG-D) by introducing learnable prompt pairs (LPP) in an adversarial manner to perform discrimination on the more abstract output feature of CLIP, further enhancing the discriminative ability of our method. With both Feat-D and TG-D, our SFD can effectively distinguish between the semantic feature distributions of low-quality and high-quality images, encouraging SRN to generate more realistic and semantic-relevant textures. Furthermore, based on the trained Feat-D and LPP, we propose a novel opinion-unaware no-reference image quality assessment (OU NR-IQA) method, SFD-IQA, greatly improving OU NR-IQA performance without any additional targeted training. Extensive experiments on classical SISR, real-world SISR, and OU NR-IQA tasks demonstrate the effectiveness of our proposed methods. Code is available at https://github.com/GuangluDong0728/SFD . 1 Introduction Figure 1 : Perception-distortion trade-off comparison between our proposed SFD and other SOTA GAN-based SISR methods. Figure 2 : Relative PLCC comparation between the proposed SFD-IQA and other OU NR-IQA methods. Figure 3 : (a) Vanilla GAN-based SISR methods: performing discrimination on images; (b) Our proposed SFD: performing discrimination on the semantic features. Vanilla GAN-based SISR typically uses an image-wise, patch-wise, or pixel-wise discriminator to determine if the distribution of SR images align with that of high-quality real-world images, ignoring the semantic information of images. Such coarse-grained discrimination makes it challenging for the SR network to reconstruct fine-grained, semantic-relevant textures. In contrast, we propose to perform discrimination on the semantic features, encouraging the SR network to generate more realistic semantic textures. As one of the most important tasks in low-level computer vision [ 33 , 52 , 36 , 60 , 19 , 14 , 50 , 67 ] , single image super-resolution (SISR) [ 11 , 62 , 57 , 80 , 18 , 34 , 56 , 28 ] aims to accurately reconstruct high-resolution (HR) images from the given low-resolution (LR) counterparts. In recent years, with advancements in deep learning, SISR has achieved rapid and substantial progress. Since Dong et al. [ 11 ] first introduce deep learning to SISR task, numerous studies have emerged to explore more advanced network architectures, including CNN-based models [ 82 , 81 , 57 , 34 , 56 , 28 ] and Transformer-based models [ 80 , 31 , 72 , 13 , 7 , 38 ] , achieving continuous improvements in fidelity-oriented performance. However, these works primarily employ pixel-wise L1 or MSE losses for the training process, which often produce overly smooth results and fail to generate realistic texture details [ 29 , 4 ] . To overcome the above limitations, a series of generative adversarial networks (GANs) [ 17 ] based SISR models [ 65 , 32 , 30 , 40 , 15 , 63 , 48 , 6 , 47 , 29 ] have been proposed to generate more realistic images that align more closely with human visual perception by effectively balancing perceptual quality and fidelity. As shown in Figure 3 (a), most existing GAN-based methods perform discrimination on the images using an image-wise [ 54 ] , patch-wise [ 23 ] , or pixel-wise [ 53 ] discriminator to distinguish between the distributions of SR images and HR images. However, directly discriminating on the images is coarse-grained and lacks semantic awareness. This leads to that the textures of generated images do not match its semantic and often come with artificial artifacts, making it challenging to produce realistic semantic textures. In this paper, different from existing works, we propose to perform discrimination on the semantic features, aiming to align the semantic feature distributions of SR images more closely with that of HR images. Considering that CLIP [ 51 ] can extract more interpretable, semantic-aware and quality-relevant features [ 61 , 30 , 9 , 20 ] , we select CLIP’s image encoder as the semantic feature extractor and propose two methods to perform joint discrimination on the semantic features. Specifically, as shown in Figure 3 (b), as for the pixel-wise middle semantic features, we first design a feature discriminator, Feat-D, to distinguish between the feature distributions of SR images and HR images, encouraging the semantic feature distributions of SR images to match that of HR images. As for the final output feature which is more abstract and global, we propose a text-guided discrimination method, TG-D, by introducing learnable prompt pairs (LPP) into an adversarial training process to make them sensitive to global image quality and thus able to discriminate the abstract semantic features, further enhancing our method’s overall discriminative capability. With both Feat-D and TG-D, the SR network is able to learn more fine-grained semantic feature distributions, ultimately producing more realistic and semantic-related texture details. Additionally, existing researches [ 83 , 39 , 40 ] have shown that during the training process where the discriminators in GAN-based SISR learn to distinguish the realness of image distributions, they simultaneously and implicitly learn to assess image quality, making them highly applicable to opinion-unaware no-reference image quality assessment (OU NR-IQA). Simultaneously, recent studies [ 20 , 44 ] have also shown the advantage of CLIP for OU NR-IQA. In this paper, to further improve OU NR-IQA performance, we propose a novel OU NR-IQA method, SFD-IQA, by integrating the well trained Feat-D and TG-D which conveniently combines both of the above advantages. Thanks to the superior semantic feature discrimination capability of Feat-D and TG-D and the inherent image quality correlation of CLIP, we greatly improve the OU NR-IQA performance without any additional task-specific training on IQA datasets. As shown in Fig. 1 , our SFD demonstrates a better perception-distortion (PD) trade-off on the PSNR-LPIPS-DISTS plane. Compared to other state-of-the-art (SOTA) GAN-based SISR methods, our LPIPS and PSNR scores are the best among all methods with competitive DISTS scores. Additionally, Fig. 2 illustrates a comparison between our SFD-IQA and other SOTA OU NR-IQA methods, where SFD-IQA outperforms other methods across both SR IQA datasets and authentically distorted IQA datasets. The main contributions of this work are summarized below: • We propose a semantic feature discrimination (SFD) method for perceptual SISR, which consists of a feature discriminator (Feat-D) based discrimination and a text-guided discrimination (TG-D). Our SFD can effectively distinguish between the CLIP semantic feature distributions of SR and HR images, encouraging SR networks to learn more fine-grained and semantic-related textures, and achieving better perception-distortion (PD) trade-off. • We propose a novel OU NR-IQA method, SFD-IQA, by effectively integrating the two well trained feature discrimination methods.