Open-Set Semi-Supervised Learning for Long-Tailed Medical Datasets

Daniya Najiha A. Kareem, Jean Lahoud, Mustansar Fiaz, Amandeep Kumar, Hisham Cholakkal · 2025

Read Open-Set Semi-Supervised Learning for Long-Tailed Medical Datasets by Daniya Najiha A. Kareem, Jean Lahoud, Mustansar Fiaz, Amandeep Kumar, Hisham Cholakkal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14846.txt.

Introduction

Daniya Kareem and co-authors tackle three tangled problems in medical image AI at once. Real clinic data are lopsided, with rare classes drowned by common ones. New, unseen classes also show up in practice. Labels, meanwhile, are scarce and dear. Their method learns from a mix of labeled and unlabeled scans while staying open to the unknown.

A feature level rule fights the pull of the long tail, and a tweak to the classifier evens its aim. Tests on skin and tissue image sets, with varied label counts, show gains on both known and unseen classes. The team plans to release code and trained models for others to build on.

Catalog record

CREATORDaniya Najiha A. Kareem, Jean Lahoud, Mustansar Fiaz, Amandeep Kumar, Hisham Cholakkal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14846

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Open-Set Semi-Supervised Learning for Long-Tailed Medical Datasets Daniya Najiha A. Kareem, Jean Lahoud, Mustansar Fiaz, Amandeep Kumar, Hisham Cholakkal Abstract Many practical medical imaging scenarios include categories that are under-represented but still crucial. The relevance of image recognition models to real-world applications lies in their ability to generalize to these rare classes as well as unseen classes. Real-world generalization requires taking into account the various complexities that can be encountered in the real-world. First, training data is highly imbalanced, which may lead to model exhibiting bias toward the more frequently represented classes. Moreover, real-world data may contain unseen classes that need to be identified, and model performance is affected by the data scarcity. While medical image recognition has been extensively addressed in the literature, current methods do not take into account all the intricacies in the real-world scenarios. To this end, we propose an open-set learning method for highly imbalanced medical datasets using a semi-supervised approach. Understanding the adverse impact of long-tail distribution at the inherent model characteristics, we implement a regularization strategy at the feature level complemented by a classifier normalization technique. We conduct extensive experiments on the publicly available datasets, ISIC2018, ISIC2019, and TissueMNIST with various numbers of labelled samples. Our analysis shows that addressing the impact of long-tail data in classification significantly improves the overall performance of the network in terms of closed-set and open-set accuracies on all datasets. Our code and trained models will be made publicly available at https://github.com/Daniyanaj/OpenLTR . † † address: 1 Mohamed bin Zayed University of AI, UAE 2 IBM Research, UAE 3 Johns Hopkins University, US 1 Introduction Medical image recognition includes classifying images, such as pathological images, X-rays, MRI scans, and CT scans into different classes to aid in diagnosis, disease monitoring, and treatment planning [ 1 , 2 ] . In the practical scenarios, the classification task faces numerous challenges. Often, labelled data for categories of interest is limited, where high-quality annotations are scarce due to privacy, ethical concerns, and the requirement for expert annotations. In addition, medical images are complex due to the fine-grained nature which often contain subtle features, thus requiring advanced analysis techniques and high-resolution imaging to accurately interpret and diagnose conditions. Imbalanced class distribution poses another hurdle, as disease-positive cases are relatively scarce within the broader population, leading to a large variation in the occurrence rate of different diseases. To overcome the challenge of limited annotations, semi-supervised learning (SSL) has been introduced as a promising approach [ 3 , 4 ] , [ 5 ] , and [ 6 , 7 ] . While SSL approaches offer potential solutions to the limited data availability, they do not address other challenges, such as the class imbalance and the need to detect novel classes not seen in the training phase. Effectively managing this class imbalance is of paramount importance to prevent false negatives in the minority class (i.e., the disease-positive cases), as overlooking them could lead to potentially fatal outcomes. In many real-world medical scenarios, it’s often impractical to manually acquire such an extensive dataset with equal distribution among all disease classes. The imbalance in dataset affects the natural equilibrium state of the classifier feature representations. When training with imbalanced data, it is important to carefully balance the distributions at both the feature and classifier levels to ensure effective learning [ 8 , 9 , 10 ] . A more complex scenario emerges, wherein unlabelled data may contain outliers representing unseen classes not present in the labelled dataset. To this end, we introduce a novel recognition system that accurately classifies both prevalent and less common classes, generalizes from limited examples of known instances, and correctly identifies novel instances that have not been encountered before. We test our method in a practical real-world scenario, where data annotation is limited and unbalanced, and novel classes might appear at inference time. In summary, this paper has the following contributions: • An open-set SSL framework is adopted for medical image classification that can classify known categories and can distinguish between known and unknown classes, in a unified paradigm. • To rule out the adverse effect of data imbalance on classification, we implement a regularization strategy at the feature level in addition to weight normalization at the classifier layer. • Extensive experiments are conducted on three distinct datasets to showcase the efficacy of our techniques aimed at tackling class imbalance issues in both supervised learning and SSL contexts. 2 Methodology We propose an integrated framework to classify known categories in the long tail medical data from a few samples and identify the unknown classes within the dataset. The input images, denoted by \mathcal{Y}\in\mathcal{R}^{H\times W\times D} , are processed to form labelled batch \mathcal{I_{C}} and unlabelled batch \mathcal{I_{U}} . Our network is organized into two main branches, a closed-set classification branch and an open-set recognition branch, as shown in Figure 1 . Both the closed-set and open-set classification branches are trained simultaneously to classify the known classes from a few samples, as well as to identify unknown classes. Figure 1 : Overall Architecture : The framework is organized into two branches, namely, a closed-set classification branch and an open-set recognition branch. The black and red arrows correspond to the flow of labelled data and unlabelled data respectively. In the closed-set classification branch, the backbone features of the labelled batch are passed to a closed-set classifier and to a multi-binary classifier, which are optimised by the cross-entropy and multi-binary losses respectively. In parallel, for feature regularization, another classifier initialised as a random simplex equiangular tight frame (ETF) is applied on the feature centers of the labelled batch and optimised using the regularization loss. Further, a classifier weight normalization technique is employed on the closed-set classifier weights to overcome the adverse effect of data imbalance on training. In the open-set recognition branch, features from the unlabelled batch are projected into a low dimensional feature embedding on which a set of multi-binary classifiers are employed. The multi-binary classifier along with closed-set classifier produces open-set targets. These open-set targets optimize the closed-set and open-set classifiers to achieve joint inliers and outliers utilization. 2.1 Closed-Set Classification Branch In the closed-set classification branch, features extracted from the labelled batch with seen-class samples are fed to a multi-class classifier, which is then optimized with the cross-entropy loss \text{H}(\cdot,\cdot) : \mathcal{L}_{sup}(\mathcal{I_{C}})=\frac{1}{M}\sum_{i=1}^{M}\text{H}(\bm{z}_{i},y_{i}). (1) where \bm{M} is the batch size, z_{i} is the classifier output, and {y}_{i} is the corresponding ground-truth label. Feature-Center Regularization: In a training setup with class-balanced data, during the final stage of training, the features of the final layer converge towards the mean values within their respective classes [ 11 ] . These within-class means, combined with the classifier weights, will then collapse to the vertices of a simplex that forms an equiangular tight frame (ETF). For a dataset with L classes, let v_{i}\in{R}^{d} be the d-dimensional feature of the final classifier layer. Simplex equiangular tight frame (ETF) can be defined as a collection of vectors namely, {n}_{l}\in\mathbb{R}^{d} , l=1,2,...,L , d\geq L , which upon satisfying the below condition is called a simplex ETF. If {N}=[{n}_{1},...,{n}_{K}]\in\mathbb{R}^{d\times L} , {U}\in\mathbb{R}^{d\times L} allows a rotation and satisfies {U}^{\top}{U}={I}_{L} , {I}_{L} is the identity matrix, and {1}_{L} is a vector with

Read the complete text