Performance Estimation for Supervised Medical Image Segmentation Models on Unlabeled Data Using UniverSeg
Jingchen Zou, Jianqiang Li, Gabriel Jimenez, Qing Zhao, Daniel Racoceanu, Matias Cosarinsky, Enzo Ferrante, Guanghui Fu · 2025
Read Performance Estimation for Supervised Medical Image Segmentation Models on Unlabeled Data Using UniverSeg by Jingchen Zou, Jianqiang Li, Gabriel Jimenez, Qing Zhao, Daniel Racoceanu, Matias Cosarinsky, Enzo Ferrante, Guanghui Fu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15667.txt.
Introduction
Jingchen Zou and co-authors present, in 2025, a way to estimate how well a medical image segmentation model performs on data nobody has labeled. In clinics, new scans rarely come with expert outlines, so true accuracy stays unknown. Their evaluator uses a general segmentation model to help guess the target model's scores.
The framework adapts to many quality measures and model designs. Across six public datasets and six measures, its estimates tracked the real overlap scores closely, with small average error. It plugs into training without adding overhead. The authors offer it as a safety check for deploying segmentation tools where labeling every case is impossible.
Catalog record
| CREATOR | Jingchen Zou, Jianqiang Li, Gabriel Jimenez, Qing Zhao, Daniel Racoceanu, Matias Cosarinsky, Enzo Ferrante, Guanghui Fu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15667
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Performance Estimation for Supervised Medical Image Segmentation Models on Unlabeled Data Using UniverSeg Jingchen Zou, Jianqiang Li, Gabriel Jimenez, Qing Zhao, Daniel Racoceanu, Matias Cosarinsky, Enzo Ferrante, Guanghui Fu Abstract The performance of medical image segmentation models is usually evaluated using metrics like the Dice score and Hausdorff distance, which compare predicted masks to ground truth annotations. However, when applying the model to unseen data, such as in clinical settings, it is often impractical to annotate all the data, making the model’s performance uncertain. To address this challenge, we propose the Segmentation Performance Evaluator (SPE), a framework for estimating segmentation models’ performance on unlabeled data. This framework is adaptable to various evaluation metrics and model architectures. Experiments on six publicly available datasets across six evaluation metrics including pixel-based metrics such as Dice score and distance-based metrics like HD95, demonstrated the versatility and effectiveness of our approach, achieving a high correlation (0.956 ± \pm 0.046) and low MAE (0.025 ± \pm 0.019) compare with real Dice score on the independent test set. These results highlight its ability to reliably estimate model performance without requiring annotations. The SPE framework integrates seamlessly into any model training process without adding training overhead, enabling performance estimation and facilitating the real-world application of medical image segmentation algorithms. The source code is publicly available at: https://anonymous.4open.science/r/SPE . 1 Introduction Automatic medical image segmentation is a critical task in image analysis frameworks [ 1 , 4 ] . Segmenting lesions or anatomical structures supports clinical decision-making, such as surgical planning [ 13 ] or disease characterization [ 18 ] . Currently, supervised deep learning method serves as the foundation for constructing segmentation models, which require training on annotated images of target regions [ 17 , 9 ] . Representative architectures like UNet [ 12 ] can achieve high accuracy when provided with sufficient image-label pairs during training [ 1 ] . However, estimating model performance on unseen clinical data remains challenging [ 6 ] . While visual inspection or annotating additional image-label pairs is a common approach, it is impractical for large cohorts due to the high cost and time requirements of manual annotation. Even with extensive annotations, models will inevitably encounter unseen data. Therefore, an automatic estimation framework is essential for real-world applications. Vanya et al. [ 16 ] proposed a novel framework named as reverse classification accuracy (RCA). It trains a reverse classifier using the predicted segmentation from a new image and evaluates it on reference images with ground truth. A high-quality prediction leads to good reverse classifier performance on some reference images. However, RCA is primarily suited for atlas-based segmentation models or anatomical structures with minimal variation, making it ineffective for lesion segmentation due to altered anatomy. The development of UniverSeg [ 2 ] , a foundation model for medical imaging, offers a flexible tool that can be explored for performance estimation. UniverSeg segments new images by referencing a support set of image-label pairs and adapts to varying anatomical and pathological conditions. Leveraging the strong correlation between support set quality and segmentation performance, this approach provides a promising direction for estimating model performance, extending beyond the limitations of RCA. In this paper, we propose a flexible performance estimation framework called the Segmentation Performance Evaluator (SPE). In the SPE framework, segmentation models are saved at different training epochs to capture their varying performance levels on the test set, which are considered as the real performance. These models generate predicted masks on the test set at each epoch. UniverSeg then uses these predicted masks as the support set to define its tasks and performs inference on images from the training set. Pseudo-performance is calculated by comparing the newly generated predictions with the ground truth in the training set. A simple linear function is fitted to map the pseudo-performance to the corresponding real performance. This linear mapping is used during real-world applications to estimate the model’s actual performance based on pseudo-performance. We conducted experiments on six datasets across five imaging modalities, covering lesions and anatomical structures, to estimate six pixel- and distance-level metrics. The results shown its effectiveness, with a high correlation and low MAE on the independent dataset, highlighting its ability to estimate model performance without annotations. This framework integrates seamlessly into the training process without restrictions on model architecture, enabling performance estimation for clinical applications of medical image segmentation algorithms. All source codes are publicly available. 2 Methods The SPE framework consists of four stages: the training stage, inference stage, pseudo-metric computation, and fitting stage. The framework process is illustrated in Figure 1 . Figure 1: An overview of our proposed SPE framework. 2.1 Model development and evaluation Training In a standard deep learning-based segmentation model development process, the annotated dataset ( I for images and L for corresponding labels) is typically divided into three subsets: a training set ( I_{\text{train}} and L_{\text{train}} ), a validation set, and a test set ( I_{\text{test}} and L_{\text{test}} ). The training and validation sets are used for model development, while the test set is reserved for evaluation. The segmentation model is not restricted within our framework; we use UNet ( M_{u} ) as an example since it is a standard model in the field. We assume the model is trained for K epochs, with the model trained at the i -th epoch denoted as M_{u}^{i} . Evaluation During the evaluation stage, the model M_{u}^{i} is used to infer segmentation results P_{\text{test}}^{M_{u}^{i}} from the test set images I_{\text{test}} . Given the annotated labels L_{\text{test}} in the test set, the real performance of M_{u}^{i} can be calculated as: \phi_{r}^{i}=F(L_{\text{test}},P_{\text{test}}^{(M_{u}^{i})}) (1) where F denotes any segmentation evaluation metric function, such as the Dice score or Hausdorff distance. 2.2 Reverse pseudo metric calculation UniverSeg [ 2 ] is a Foundation Model for medical image segmentation that addresses unseen tasks without additional training, making it suitable for clinical use. It relies on a support set of image-label pairs to infer query image segmentation. Segmentation performance is directly tied to the quality of the support set: good support sets enhance performance, while poor ones lead to degradation. In the reverse metric computation stage, we use the UniverSeg model M_{\text{useg}}^{S} , configured with a support set S=(I_{\text{test}},P_{\text{test}}^{M_{u}^{i}}) . The reason we use the test set rather than the validation set is that the validation set is typically used for tuning model performance, whereas the reverse pseudo-metric calculation process requires an independent dataset to ensure unbiased mapping. This model is then used to perform inference on the training set I_{\text{train}} , which has annotated labels available. The UniverSeg model M_{\text{useg}}^{S} generates segmentation results P_{\text{train}}^{M_{\text{useg}}} . These results are compared with the true labels L_{\text{train}} of the training set, and the pseudo performance metric \phi_{p}^{i} is calculated as follows: \phi_{p}^{i}=F(L_{\text{train}},P_{\text{train}}^{M_{\text{useg}}^{i}}) (2) Since the model M_{u} is trained for K epochs during the training stage, these above stages can produce K pairs of (\phi_{r}^{i},\phi_{p}^{i}) at epoch i . Here, \phi_{r}^{i} represents the real performance of M_{u}^{i} on the test set, and \phi_{p}^{i} represents the pseudo performance metric from the reverse evaluation stage. The pair of real performance and reverse pseudo performance is denoted as \Psi and it represents performance of an entire group of images, rather than the individual