Revisiting Active Learning under (Human) Label Variation

Cornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher · 2025

Read Revisiting Active Learning under (Human) Label Variation by Cornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-02593.txt.

Description

Cornelia Gruber and co-authors rethink active learning when human labelers honestly disagree, in work from 2025. Active learning spends a tight labeling budget on the most useful cases. Most setups assume one true label per case, yet people often split for good reasons.

Disagreement can be signal, not just noise. The paper separates true human variation from plain mistakes and surveys how two research camps handled, or missed, the split. A framework threads label variation through the whole loop: which cases to pick, which labeler to ask, and how to store split labels. Language models as labelers get a look too. The aim is groundwork, not a new algorithm.

Catalog record

CREATORCornelia Gruber, Helen Alber, Bernd Bischl, Göran Kauermann, Barbara Plank, Matthias Aßenmacher
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-02593

Browse the full library · About the Tower of Babel