Realistic Evaluation of Deep Partial-Label Learning Algorithms
Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama · 2025
Read Realistic Evaluation of Deep Partial-Label Learning Algorithms by Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10184.txt.
Description
Some training data come with several candidate labels per item, only one of them right. Wei Wang and co-authors examine how deep learning methods for this partial-label problem have been tested, and they find the testing itself is flawed. Settings differ from paper to paper, and model selection, a key step, was never studied in a systematic way.
Surprisingly, some early, simple methods beat newer complex ones under fair conditions. To fix the comparison problem, the authors build PLENCH, a benchmark suite for the field. It adds principled rules for choosing models, backed by theory. It also adds a new image data set with human-given partial labels gathered through a crowdsourcing platform. The authors hope the suite makes future claims in this area easier to check and trust.
Catalog record
| CREATOR | Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10184