OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning
Shengjie Niu, Lifan Lin, Jian Huang, Chao Wang · 2024
Read OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning by Shengjie Niu, Lifan Lin, Jian Huang, Chao Wang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01833.txt.
Description
Abstract Semi-supervised learning (SSL) offers a robust framework for harnessing the potential of unannotated data. Traditionally, SSL mandates that all classes possess labeled instances. However, the emergence of open-world SSL (OwSSL) introduces a more practical challenge, wherein unlabeled data may encompass samples from unseen classes.
Catalog record
| CREATOR | Shengjie Niu, Lifan Lin, Jian Huang, Chao Wang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01833
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning Shengjie Niu, Lifan Lin, Jian Huang, Chao Wang Abstract Semi-supervised learning (SSL) offers a robust framework for harnessing the potential of unannotated data. Traditionally, SSL mandates that all classes possess labeled instances. However, the emergence of open-world SSL (OwSSL) introduces a more practical challenge, wherein unlabeled data may encompass samples from unseen classes. This scenario leads to the misclassification of unseen classes as known ones, consequently undermining classification accuracy. To overcome this challenge, this study revisits two methodologies from self-supervised and semi-supervised learning, self-labeling and consistency, tailoring them to address the OwSSL problem. Specifically, we propose an effective framework called OwMatch , combining conditional self-labeling and open-world hierarchical thresholding. Theoretically, we analyze the estimation of class distribution on unlabeled data through rigorous statistical analysis, thus demonstrating that OwMatch can ensure the unbiasedness of the self-label assignment estimator with reliability. Comprehensive empirical analyses demonstrate that our method yields substantial performance enhancements across both known and unknown classes in comparison to previous studies. Code is available at https://github.com/niusj03/OwMatch . 1 Introduction Deep learning has made remarkable success in various tasks by leveraging substantial labeled training data [ 22 , 21 , 13 ] . However, the costly and time-consuming labeling process limits their application in practical scenarios. Semi-supervised learning (SSL) significantly reduces the dependency on labeled data by exploring the inherent structure of unlabeled data [ 16 ] . Despite promising results, SSL methods assume a closed-world scenario where, though limited, all classes possess labeled instances. This assumption may be violated due to difficulties in data collection, such as in medical diagnostics, where it is common to encounter new symptoms or fail to annotate due to technical constraints. As a result, only a subset of the categories can be precisely labeled during the annotation process. Recently, numerous studies have sought to identify such novel classes effectively. Open-world SSL (OwSSL) is innovative in promoting dual objectives: classifying instances of seen classes and discovering instances of novel classes [ 5 ] . A notable challenge in OwSSL is the confirmation bias of model: model tends to predict instances as seen classes owing to the lack of ground-truth supervision of novel-class instances. To eliminate this bias, existing works utilize unsupervised clustering methods, including contrastive loss and binary cross-entropy (BCE) loss, to group pairs identified by similarity metrics [ 5 , 15 ] . Among these unsupervised techniques, self-labeling [ 1 , 6 ] has shown remarkable success, which involves assigning self-labels to unlabeled data, with the generation of high-quality self-labels being the key factor. Previous studies utilize optimal transport to align the self-labels for unlabeled data with a given distribution. However, this self-label generation fully relies on the accurate prior distribution and lack of consideration of the supervision of labeled data. In TRSSL [ 34 ] , the unlabeled data are assigned with a soft self-label based on the inaccurate class distribution, which raises a biased estimation. Moreover, the confirmation bias still exists even if we use the ground-truth distribution to align the unlabeled data in the same process. In addition to confirmation bias, a new issue called clustering misalignment arises when self-labeling depends solely on unlabeled data: without proper guidance, the self-labeling process may adopt varying criteria for clustering. For example, it might cluster data based on superficial features like color rather than high-level semantic information. This misalignment can lead to results that deviate from expected outcomes and even contradict the classification criteria established by labeled data. Consequently, we propose a new self-labeling scheme, conditional self-labeling, designed to address the challenges of OwSSL, particularly targeting issues related to confirmation bias and misalignment. This scheme limits self-labels for each class and incorporates labeled data to generate debiased and informative self-label assignments for all training data, further mitigating the confirmation bias as shown in Figure 1(a) . Additionally, as illustrated in Figure 1(b) , seen classes typically exhibit higher predictive confidence, while novel clusters demonstrate variability in their internal learning progresses. The disparities in learning paces between seen and novel classes, coupled with their distinct behaviors, necessitate the selection of appropriate thresholds to facilitate cluster learning. To address these challenges and ensure a balanced learning process across classes, we propose a hierarchical thresholding scheme. We demonstrate our contributions as follows: 1) We introduce a novel conditional self-labeling method to incorporate labeled data into the clustering process, reducing confirmation bias and misalignment. 2) We design a hierarchical thresholding strategy that balances learning difficulties across different classes, helping unstable clusters gradually form. 3) Our theoretical analysis rigorously discusses the unbiasedness and reliability of conditional self-labeling estimator from population-level statistics. To the best of our knowledge, this is the first work proposing the expectation of chi-square statistics (ECS) to evaluate the reliability of self-label assignment estimation. 4) We conduct extensive experiments on various datasets, demonstrating the effectiveness of our approach, OwMatch, through detailed comparisons. On CIFAR-10, OwMatch significantly outperforms FixMatch [ 36 ] by up to 47.3% in all-class accuracy, while on CIFAR-100, it enhances TRSSL [ 34 ] , the state-of-the-art model in OwSSL, by up to 14.6% in novel-class and 7.2% in all-class accuracy. (a) (b) Figure 1 : Experimental results on the OwSSL problem. (a) Self-label assignment of seen classes (1-5) and novel classes (6-10) with or without conditional component in self-labeling. (b) Predictive confidence and hierarchical threshold for each class. 2 Related work Traditional semi-supervised learning (SSL). Traditional SSL assumes that labeled and unlabeled data share an identical distribution. Extensive researches on SSL have spanned a considerable duration. The commonly employed strategies in SSL consist of entropy minimization [ 29 , 14 ] , consistency regularization [ 44 , 30 ] and holistic methods [ 4 , 3 , 36 ] . The latest progresses in SSL include adaptive thresholding strategies [ 42 , 45 , 47 ] , which enhance model performance by accounting for varying difficulties and learning conditions across classes, alongside other innovative techniques [ 19 , 41 , 2 ] that employ self-SL approaches to facilitate extracting the semantic information from unlabeled data. However, traditional SSL algorithms typically struggle to tackle the open-world problem in the presence of novel-class instances within unlabeled data. Open-set semi-supervised learning (OSSL). OSSL expands the traditional SSL boundaries by allowing novel-class instances or outliers within unlabeled data. A variety of OSSL approaches have emerged in recent years [ 46 , 16 , 35 , 8 , 23 ] . A common solution among these methods is the optimization of the SSL objective exclusively for unlabeled samples deemed inliers. For instance, MTC [ 46 ] optimizes the network and estimates the anomaly score of unlabeled data alternately. OpenMatch [ 35 ] and T2T [ 23 ] train one-vs-all (OVA) classifiers for each known class to detect outliers. Subsequently, standard SSL objective [ 36 ] are applied to the remaining training data, excluding detected outliers. Furthermore, DS3L [ 16 ] leverages a bi-level optimization technique to train a weighting function, which mitigates the passive impact of out-of-distribution (OOD) samples. Nonetheless, these approaches are designed for classifying seen classes, thereby failing to learn from the novel class instances. Open-world semi-supervised learning (OwSSL). OwSSL [ 5 ] has been proposed to