FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images

Pengwei Wang ( ), José Morano, Virginia Mares, Hrvoje Bogunović · 2026

Read FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images by Pengwei Wang ( ), José Morano, Virginia Mares, Hrvoje Bogunović (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25915.txt.

Introduction

Pengwei Wang and colleagues address a practical problem in eye screening: judging the quality of colour fundus photographs, images of the back of the eye used to spot retinal disease. Existing quality labels judge the whole image at once, yet damage like blur or poor lighting often ruins only part of a photo. Wang's team argues that quality should be scored pixel by pixel, based on whether anatomical structures stay visible.

FunPiQ, the benchmark they introduce, is the first fundus dataset with pixel-level quality annotations. Their companion method, EFIQA-CP, trains a neural network on quality pseudo-labels derived from anatomical visibility, using a learning scheme built for partly labelled data. Across their evaluations, explainable-by-design methods beat ordinary classifiers with after-the-fact explanations and anomaly detectors, with EFIQA-CP performing best overall.

Catalog record

CREATORPengwei Wang ( ), José Morano, Virginia Mares, Hrvoje Bogunović
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25915

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images Pengwei Wang ( ), José Morano, Virginia Mares, Hrvoje Bogunović Abstract Color fundus photography (CFP) is the most common ophthalmic imaging modality for large-scale screening. However, it is highly susceptible to degradations, making robust fundus image quality assessment (FIQA) crucial. The criteria for what constitutes high-quality at the image level vary across clinical tasks, making FIQA dependent on expert knowledge. This motivated the development of automated methods and datasets. While existing datasets aim to standardize image-level quality, their criteria often differ. Furthermore, image-level labels preclude the quantitative evaluation of localized degradations, which is essential for trustworthy FIQA. We argue that pixel-level FIQA based on anatomical visibility represents a more task-agnostic, explainable approach. In this work, we introduce FunPiQ, the first FIQA benchmark to provide pixel-level quality annotations. In addition, we propose EFIQA-CP, an explainable-by-design (EBD) method that uses quality pseudo-labels based on anatomical visibility to train a CNN via Non-Negative Positive-Unlabeled learning. Extensive evaluations of classification methods with post-hoc explanations, anomaly detection methods, and EBD methods demonstrate the superior performance of the last and, particularly, of EFIQA-CP. Our code, weights, and dataset are publicly available at https://github.com/penway/FunPiQ 1 Introduction Figure 1: Overview of our FunPiQ dataset. Color fundus photography (CFP) is one of the most widely used imaging modalities in ophthalmology [ 7 ] , especially for screening. CFP enables the discovery not only of retinal diseases but also of systemic health conditions [ 32 , 11 ] . In addition, it is arguably the most accessible ophthalmic modality, with devices ranging from professional desktop systems to portable, mobile-phone-based cameras, making large-scale screening feasible in remote areas [ 23 ] . However, CFP is highly susceptible to degradations, such as inconsistent illumination and poor contrast [ 8 ] . These issues often affect local regions rather than the entire image, meaning certain areas remain of high quality while others are degraded (see Fig. 1 ). Such degradations are frequent even in professional settings. For example, in the EyePACS benchmark [ 6 ] , 42% of images had quality issues, with 19% deemed unsuitable for diagnosis [ 8 ] . These challenges are further exacerbated when using portable fundus cameras in uncontrolled environments. Thus, Fundus Image Quality Assessment (FIQA) represents a crucial step. Nonetheless, FIQA faces important challenges, primarily due to the lack of standardized criteria. The definition of image-level quality is inherently tied to specific downstream tasks, as the visibility of certain anatomical structures may be critical for diagnosing some diseases while remaining less relevant, or irrelevant, for others. This task-dependency makes manual, image-level FIQA partly subjective, time-consuming, and dependent on expert knowledge, rendering it unfeasible for large-scale screening. Consequently, there is a growing need for automated methods and datasets. Table 1 summarizes the most relevant datasets for FIQA. While these benchmarks provide various quality-related labels, ranging from global classes to specific degradation types, they implement distinct criteria for what constitutes “quality”. As previously discussed, a universal, image-level standard remains elusive because diagnostic requirements vary by disease; consequently, some images labeled as bad quality in these datasets are actually usable for specific clinical tasks. For example, in MSHF [ 14 ] , images flagged as bad quality were still deemed suitable for glaucoma screening. Furthermore, the separate standardization efforts have led to fairly dissimilar classifications across benchmarks, evidenced by the suboptimal generalization of models on external tests [ 29 ] , and by differences in quality labels assigned to the same images by independent teams—for instance, an 11% difference in a subset of EyePACS [ 8 , 31 ] . Given the inherently local nature of degradations, we argue that defining quality locally, through the visibility of underlying anatomy, offers a more task-agnostic, explainable alternative. This approach can be coupled with specific anatomical requirements (e.g., optic disc visibility) to implement dynamic image-level quality criteria. Table 1: Summary of current CFP datasets with quality labels, including classes (C) and degradation types (DT). 2C includes good and bad , while 3C adds usable . FunPiQ is the first evaluation dataset providing pixel-level quality annotations. Granularity Dataset Year Size Country Quality Labels Image-level DRIMDB [ 26 ] 2014 216 Turkey 2C and outlier class EyePACS-Q [ 31 ] 2018 88,702 US 2C EyeQ [ 8 ] 2019 30,000 US 3C DeepDRiD [ 17 ] 2022 2,256 China 2C and 0–10 score with 3DT MSHF [ 14 ] 2023 1,302 China 2C and 3DT BRSET [ 21 ] 2024 16,266 Brazil 2C and 4DT mBRSET [ 30 ] 2025 5,164 Brazil 2C FQS [ 10 ] 2025 2,246 China 3C and mean opinion score Pixel-level FunPiQ 2026 300 US, Brazil 3C Most FIQA methods rely on supervised deep learning on the aforementioned datasets, incorporating various architectural innovations. For instance, MCF-Net utilizes multi-color space fusion [ 8 ] ; FGR-Net, reconstruction-based representations [ 15 ] , and Shen et al. use semi-tied adversarial domain adaptation for domain-invariant features [ 27 ] . However, these approaches depend on image-level labels, inheriting the biases of their training datasets. Furthermore, while post-hoc visualization techniques like Grad-CAM [ 25 ] are frequently used [ 27 , 15 ] , the explainability provided by these tools is limited, and the localization of degradations is generally imprecise. To address the latter, EyeQual [ 4 ] proposes an explainable-by-design architecture that outputs local-level quality maps via patch-level scoring. Aiming to address both issues, EFIQA [ 29 ] proposed using vessel unsupervised anomaly detection (VUAD) as a prior for producing local-level quality maps without the need for quality-related labels. While it showed superior performance on external tests, EFIQA still presents some limitations. First, it relies on a single linear adapter for patch-level quality scores from foundation model features, with no global context. This leads to incorrect predictions in areas with naturally low vessel density, such as the fovea. Second, it is trained on the noisy pseudo-labels from the VUAD model via a simple Binary Cross-Entropy (BCE) loss, leading to pseudo-label overfitting. Finally, despite the recognized importance of explainability and localization [ 15 , 27 ] , no benchmark currently exists to quantitatively evaluate the precision of these outputs. In this context, the contributions of this work are threefold: 1. We introduce the Fundus Pixel Quality (FunPiQ) dataset, the first public benchmark dedicated to pixel-level quality assessment for CFP. The benchmark comprises 300 images sourced from three public datasets and annotated by expert readers supervised by a board-certified ophthalmologist. 2. We propose EFIQA-CP, a method that mitigates EFIQA’s limitations by using a shallow CNN with a large receptive field as feature adapter and Non-Negative Positive-Unlabeled (nnPU) learning [ 16 ] to handle pseudo-labels. 3. We conduct a thorough evaluation of existing and proposed methods on our benchmark, demonstrating the superior performance of EFIQA-CP and highlighting the potential of explainable-by-design approaches for implementing robust, local-level quality assessment systems. 2 Materials and Methods 2.1 Dataset Data Selection. Fig. 1 shows an overview of the dataset. To maximize data diversity in terms of degradation types, ethnicity, and acquisition devices, we sampled from three existing benchmarks from two different countries: EyeQ [ 8 ] (US), BRSET [ 21 , 20 , 9 ] , and mBRSET [

Read the complete text