Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data
Weiran Pan, Wei Wei, Feida Zhu, Yong Deng · 2025
Read Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data by Weiran Pan, Wei Wei, Feida Zhu, Yong Deng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17474.txt.
Introduction
Weiran Pan and co-authors tackle training image models on data with wrong labels, in a 2025 study. A common fix keeps only samples the model finds easy, since errors often look hard. Yet some correct samples are truly hard to learn and get thrown out by that rule, which trades one loss for another.
The team instead watches how the model's confidence in the given label grows against all other classes. For correct samples, that lead tends to rise fastest, even for hard ones. A trend test on the gap over training picks out likely correct samples with more care. The method aims to keep hard but honest samples in training, which should lift final accuracy on noisy real-world data.
Catalog record
| CREATOR | Weiran Pan, Wei Wei, Feida Zhu, Yong Deng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17474
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data Weiran Pan, Wei Wei, Feida Zhu, Yong Deng Abstract We propose a novel sample selection method for image classification in the presence of noisy labels. Existing methods typically consider small-loss samples as correctly labeled. However, some correctly labeled samples are inherently difficult for the model to learn and can exhibit high loss similar to mislabeled samples in the early stages of training. Consequently, setting a threshold on per-sample loss to select correct labels results in a trade-off between precision and recall in sample selection: a lower threshold may miss many correctly labeled hard-to-learn samples (low recall), while a higher threshold may include many mislabeled samples (low precision). To address this issue, our goal is to accurately distinguish correctly labeled yet hard-to-learn samples from mislabeled ones, thus alleviating the trade-off dilemma. We achieve this by considering the trends in model prediction confidence rather than relying solely on loss values. Empirical observations show that only for correctly labeled samples, the model’s prediction confidence for the annotated labels typically increases faster than for any other classes. Based on this insight, we propose tracking the confidence gaps between the annotated labels and other classes during training and evaluating their trends using the Mann-Kendall Test. A sample is considered potentially correctly labeled if all its confidence gaps tend to increase. Our method functions as a plug-and-play component that can be seamlessly integrated into existing sample selection techniques. Experiments on several standard benchmarks and real-world datasets demonstrate that our method enhances the performance of existing methods for learning with noisy labels. 1 School of Computer Science and Technology, Huazhong University of Science and Technology, China 2 Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL), China 3 School of Computing and Information Systems, Singapore Management University, Singapore 4 State Grid Fujian Electric Power Co. {panwr, weiw}@hust.edu.cn, [email protected], [email protected] Code — https://github.com/Aliinton/ConfidenceTracking 1 Introduction The remarkable success of deep learning methods in classification tasks can largely be attributed to high-quality datasets. However, collecting such datasets through manual labeling can be both time-consuming and expensive in many applications. Acquiring data via online queries ( Li et al. 2017 ) or crowdsourcing ( Xiao et al. 2015 ) can construct large-scale datasets at a lower cost but inevitably introduces noise labels. Existing research ( Zhang et al. 2021 ) has shown that deep neural networks can easily fit noisy data, resulting in poor generalization. Therefore, developing algorithms robust to noisy labels is of great practical importance ( Natarajan et al. 2013 ) . Figure 1: Illustration of Confidence Tracking. We train a PreActResNet-18 model using cross-entropy loss and an SGD optimizer on CIFAR-10N-Worst (CIFAR-10 dataset with human-annotated real-world noisy labels, its noise rate is 40.21%). The left graph presents the average per-sample loss distribution in the first 30 epochs. We regard samples with an average loss greater than 1.2 (indicated by the dotted vertical line) as hard-to-learn ones and show the model prediction confidence trajectories on hard-to-learn dogs’ images (middle graph) and mislabeled cats’ images (right graph). Sample selection methods aim to identify correct labels from noisy data, which is increasingly crucial for current deep learning models to effectively learn from noisy labels. There is a general consensus that the small-loss criterion is an effective approach, which assumes samples with small losses are more likely to have correct labels. In this context, the Co-teaching family ( Han et al. 2018 ; Yu et al. 2019 ; Wei et al. 2020 ; Xia et al. 2022 ) and other state-of-the-art methods ( Li, Socher, and Hoi 2020 ; Li et al. 2023 ) have been proposed. Typically, these methods select samples with losses below a threshold for training. So a higher threshold introduces more incorrect labels while a lower threshold excludes more correct labels, creating a trade-off dilemma between precision and recall in sample selection. To alleviate this issue, one possible solution is establishing another sample selection criterion that can distinguish correct labels from incorrect ones in high-loss data. Then we can combine it with the small-loss criterion to improve recall while maintaining precision in sample selection. To achieve this, we propose considering the changing trends in model predictions to identify correct labels rather than relying solely on loss values. This is motivated by our empirical observations that although some correctly labeled samples may obtain similar loss values to mislabeled ones, their training dynamics are still distinguishable. As shown in Figure 1 , some correctly labeled data can be hard to fit by the model and exhibit similar high loss to mislabeled data in the early training stage. It is difficult to distinguish them by setting a threshold on loss values. But we also observe that, only for the correctly labeled samples, the model’s prediction confidence for annotated labels tends to rise more quickly than for other classes. For instance, with correctly labeled dog images, the model’s prediction confidence for the “dog” class ( i.e., the posterior probability of the image belonging to the “dog” class predicted by the model) increases more rapidly than for any other class. Conversely, for cat images mislabeled as “dog”, the model’s prediction confidence for the “cat” class rises slightly faster than for the “dog” class. This observation suggests it’s possible to identify correctly labeled samples by considering training dynamics even when they are indistinguishable from mislabeled samples in terms of loss values. Motivated by this finding, we propose a novel sample selection criterion that selects correct labels by monitoring how model predictions change during training, dubbed Confidence Tracking (CT). Specifically, we track confidence gaps in model predictions between annotated labels and other classes during training. If all confidence gaps of a sample tend to increase, we regard it as a potentially correctly labeled sample. Unlike the small loss criterion which mainly considers the loss values, our method focuses on the changing trend in model predictions, allowing us to distinguish correctly labeled yet hard-to-learn samples from mislabeled ones within high-loss data. In practice, our method functions as a plug-and-play component that can be combined with popular sample selection methods ( Arazo et al. 2019 ; Kim et al. 2021 ; Pleiss et al. 2020 ; Li et al. 2023 ) to enhance their performance. Our method is related to AUM ( Pleiss et al. 2020 ) which also considers confidence gaps but selects samples with relatively large average logit margins as correctly labeled. Similar to the small-loss criterion, setting a threshold on average logit margins to select correct labels still faces the trade-off dilemma between precision and recall. Our experiments in Section 4 demonstrate that CT accurately selects correct labels from samples rejected by AUM or the small-loss criterion, improving recall while maintaining precision in sample selection, and bringing performance gains to various benchmarks. To sum up, our key contributions are as follows: • We analyze why correctly labeled and mislabeled samples exhibit different training dynamics from the perspective of coherent gradients ( Chatterjee 2020 ) and provide supporting evidence. • We propose a novel sample selection method based on monitoring changes in model predictions during training, termed Confidence Tracking (CT), which is a plug-and-play component that can