Bridging the Perception Gap in Image Super-Resolution Evaluation

Shaolin Su, Josep M. Rocafort, Danna Xue, David Serrano-Lozano, Lei Sun, Javier Vazquez-Corral · 2025

Read Bridging the Perception Gap in Image Super-Resolution Evaluation by Shaolin Su, Josep M. Rocafort, Danna Xue, David Serrano-Lozano, Lei Sun, Javier Vazquez-Corral (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13074.txt.

Introduction

Software that sharpens blurry photos keeps getting better, yet the scores used to grade it often disagree with what people actually prefer. Su and co-authors audit the common image quality metrics against human judgments on recent super-resolution models. Some metrics track human taste only weakly, and a few even run against it.

They trace the trouble to built-in flaws in both kinds of grading, those that compare against a reference image and those that do not. Their fix, a Relative Quality Index, grades the gap between pairs of images instead. Plugged into existing metrics it sharpened agreement, and used in training it guided models toward images with truer detail.

Catalog record

CREATORShaolin Su, Josep M. Rocafort, Danna Xue, David Serrano-Lozano, Lei Sun, Javier Vazquez-Corral
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13074

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging the Perception Gap in Image Super-Resolution Evaluation Shaolin Su, Josep M. Rocafort, Danna Xue, David Serrano-Lozano, Lei Sun, Javier Vazquez-Corral Abstract As super-resolution (SR) techniques advance, we observe a growing distrust of evaluation metrics in recent SR research. An inconsistency often emerges between certain evaluation criteria and human perceptual preference. Although current SR research employs varying metrics to evaluate SR performance, it remains underexplored how robust and reliable these metrics actually are. To bridge this gap, we conduct a comprehensive analysis of widely used image quality metrics, examining their consistency with human perception when evaluating state-of-the-art SR models. We show that some metrics exhibit only limited—or even negative—correlation with human preferences. We further identify several intrinsic challenges in SR evaluation that compromise the effectiveness of both full-reference (FR) and no-reference (NR) image quality assessment (IQA) frameworks. To address these issues, we propose a simple yet effective Relative Quality Index (RQI) framework, which assesses the relative quality discrepancy between image pairs. Our framework enables easy integration and notable improvements for existing IQA metrics in SR evaluation. Moreover, it can be utilized as a valuable training guide for SR models, enabling the generation of images with more realistic details while maintaining structural fidelity. 1 Introduction Super-resolution aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts, thereby recovering fine textures and structural details that are often lost during image degradation. In recent years, SR techniques [ 60 , 27 , 53 , 26 , 43 , 38 ] have witnessed rapid progress, with models showing substantial enhancement in visual performance. However, as SR models continue to evolve, a critical issue has emerged: the commonly used evaluation metrics, such as PSNR, SSIM [ 40 ] and LPIPS [ 58 ] , often fail to reflect the true perceptual quality of the reconstructed images [ 20 , 37 , 43 ] . In many cases, models that achieve higher scores on these conventional metrics may not actually produce images that appear more realistic or visually pleasing to human observers. To address this inconsistency, researchers have either conducted extensive user studies [ 37 , 43 , 34 , 17 ] – which are costly and time-consuming – or combined multiple quantitative metrics [ 49 , 44 , 11 , 56 ] in an attempt to provide a more comprehensive evaluation. Yet, this situation reveals a deeper contradiction: while SR models have evolved rapidly, the evaluation criteria used to assess them have remained largely unchanged . Whether these long-standing metrics are still capable of distinguishing the true perceptual performance of modern SR models remains an open and underexplored question. This gap motivates a re-evaluation of existing SR evaluation practices and calls for a deeper understanding of how well current metrics align with human visual perception. To better understand the reliability of existing evaluation criteria, we first conduct a large-scale user study to thoroughly analyze the alignment between commonly used distortion-oriented FR, perception-oriented FR, NR quality metrics and human perceptual judgments. Specifically, we examine the performance of seven state-of-the-art (SOTA) SR models across five benchmark datasets and compare their metric-based scores with subjective human ratings. The results reveal that the consistency between current quantitative metrics and human perception is rather limited, with some metrics even exhibiting negative correlations with human preference. We further analyze the challenging aspects that lead to this inconsistency. We identify that each existing evaluation framework suffers from inherent limitations. NR metrics [ 31 , 5 , 36 , 48 ] , which lack reference images, struggle to capture textural or structural fidelity. Distortion-oriented FR metrics [ 40 , 54 , 47 ] tend to regress toward the average solution across multiple plausible reconstructions, thereby underestimating perceptually superior results. Perception-oriented FR metrics [ 58 , 14 ] , on the other hand, may fail when the ground-truth images themselves are of suboptimal quality. In addition, as the perceptual quality of algorithm-generated images continues to improve, metrics that perform well in assessing clearly degraded images become less effective in distinguishing subtle differences among high-quality results. Together, these factors highlight the inadequacy of existing evaluation metrics in accurately reflecting human visual perception of modern SR outputs. Building upon these observations, we propose a remarkably simple but effective IQA training framework termed Relative Quality Index (RQI), which provides a more reliable and perceptually aligned evaluation for SR. Specifically, the proposed framework (1) accurately assesses fidelity based on reference images, (2) effectively handles cases where the ground-truth images are of suboptimal quality, and (3) enables fine-grained evaluation of subtle perceptual differences by constructing dense training pair comparisons. Moreover, since the proposed method is designed as a general framework rather than a specific metric, it can be seamlessly integrated into any existing quality assessment model or dataset for training. Extensive experiments demonstrate that the proposed framework not only provides more accurate evaluations of SR models, but it can also be employed as a loss function to guide SR training, further enhancing the performance of advanced SR models. Note that due to the ill-posed nature of the SR problem, we focus on the perceptual evaluation of SR models rather than pixel-wise distortion-based measurements. Nevertheless, our approach proves effective in assessing and preserving textural and structural fidelity. The contributions of this work are threefold: • We conduct an in-depth investigation into how accurately current full-reference and no-reference quality metrics reflect human perception in SR evaluation, and provide detailed insights into the underlying causes of their limited effectiveness. • We propose a general and lightweight framework that addresses the key issues limiting existing metrics, while being easily adaptable for integration with various quality assessment models and datasets. • We demonstrate through comprehensive experiments that the proposed framework achieves more reliable and perceptually consistent SR evaluation results. Moreover, the metric can also serve as a loss function guiding SR model optimization toward improved visual performance. 2 Related Work 2.1 Image Super-Resolution Early SR approaches [ 15 , 27 , 60 , 13 , 16 , 10 ] operate under the assumption of a predefined degradation process, such as bicubic downsampling or blurring with a known prior. However, their effectiveness is often constrained in real-world scenarios where noise or compression artifacts may occur. To handle practical SR, BSRGAN [ 53 ] and RealESRGAN [ 38 ] assume complex degradation in LR images and adversarially train models to remove multiple degradations. Transformer-based methods further improve HR quality by capturing long dependencies between pixels: SwinIR [ 26 ] utilizes window attention in Swin Transformer [ 29 ] for global dependencies, while HAT [ 12 ] optimizes hierarchical feature integration. With the emergence of diffusion techniques [ 18 ] , diffusion-based SR models [ 37 , 43 , 49 , 42 , 39 , 34 , 11 ] have demonstrated powerful ability in reconstructing image details and generating high perceptual quality outputs. StableSR [ 37 ] utilizes rich prior from the Stable Diffusion model conditioned by the LR input, SeeSR [ 43 ] explores the roles of tag-style prompts in guiding semantic robust image generation, FaithDiff [ 11 ] proposes to align the noisy latents with the LR features for maintaining structural consistency, and PiSA-SR [ 34 ] balances pixel-level fidelity and perceptual quality by

Read the complete text