Realistic Evaluation of Deep Partial-Label Learning Algorithms

Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama · 2025

Read Realistic Evaluation of Deep Partial-Label Learning Algorithms by Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10184.txt.

Description

Some training data come with several candidate labels per item, only one of them right. Wei Wang and co-authors examine how deep learning methods for this partial-label problem have been tested, and they find the testing itself is flawed. Settings differ from paper to paper, and model selection, a key step, was never studied in a systematic way.

Surprisingly, some early, simple methods beat newer complex ones under fair conditions. To fix the comparison problem, the authors build PLENCH, a benchmark suite for the field. It adds principled rules for choosing models, backed by theory. It also adds a new image data set with human-given partial labels gathered through a crowdsourcing platform. The authors hope the suite makes future claims in this area easier to check and trust.

Catalog record

CREATORWei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, Masashi Sugiyama
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10184

Browse the full library · About the Tower of Babel