An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations

Varun Totakura, Ankita Singh, Yushun Dong, Shayok Chakraborty · 2026

Read An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations by Varun Totakura, Ankita Singh, Yushun Dong, Shayok Chakraborty (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23290.txt.

Description

Totakura and three colleagues test active learning with real crowd-sourced annotations instead of simulated oracles. Standard setups assume labelers never err, but crowd workers can give wrong labels or refuse to label at all. Collecting annotations on three benchmark text datasets through a crowd platform, they study eight common active learning techniques with deep neural networks. Their results show how each technique fares under genuine annotation noise and refusal, offering guidance for deploying deep active learning in real settings.

Catalog record

CREATORVarun Totakura, Ankita Singh, Yushun Dong, Shayok Chakraborty
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70518
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23290

Browse the full library · About the Tower of Babel