Revisiting Active Learning under (Human) Label Variation
Cornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher · 2025
Read Revisiting Active Learning under (Human) Label Variation by Cornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-02593.txt.
Description
Cornelia Gruber and co-authors rethink active learning when human labelers honestly disagree, in work from 2025. Active learning spends a tight labeling budget on the most useful cases. Most setups assume one true label per case, yet people often split for good reasons.
Disagreement can be signal, not just noise. The paper separates true human variation from plain mistakes and surveys how two research camps handled, or missed, the split. A framework threads label variation through the whole loop: which cases to pick, which labeler to ask, and how to store split labels. Language models as labelers get a look too. The aim is groundwork, not a new algorithm.
Catalog record
| CREATOR | Cornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-02593