Active learning with human heuristics: an algorithm robust to labeling bias
Sriram Ravichandran, Nandan Sudarsanam, Balaraman Ravindran, Konstantinos V. Katsikopoulos · 2024
Read Active learning with human heuristics: an algorithm robust to labeling bias by Sriram Ravichandran, Nandan Sudarsanam, Balaraman Ravindran, Konstantinos V. Katsikopoulos (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11611880.txt.
Description
Sriram Ravichandran and co-authors study what happens when the teacher in active learning is a human with biases, in this 2024 paper. Active learning lets a model pick which examples to ask about, speeding its training. When people supply the labels, quick mental rules can skew their answers, and the team asked whether that skew breaks the method.
They paired two known human rule models with four active learning methods and three classifiers across fifteen data sets from health, marketing, and transport. With biased labels, performance dropped sharply, sometimes below random picking. Their fix, a method that seeks out low-density regions in a psychology-inspired way, beat the best rival by a wide margin overall. The authors urge designers to model the human labeler from the start.
Catalog record
| CREATOR | Sriram Ravichandran, Nandan Sudarsanam, Balaraman Ravindran, Konstantinos V. Katsikopoulos |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |