Human-in-the-Loop Annotation for Image-Based Engagement Estimation: Assessing the Impact of Model Reliability on Annotation Accuracy

Sahana Yadnakudige Subramanya, Ko Watanabe, Andreas Dengel, Shoya Ishimaru · 2025

Read Human-in-the-Loop Annotation for Image-Based Engagement Estimation: Assessing the Impact of Model Reliability on Annotation Accuracy by Sahana Yadnakudige Subramanya, Ko Watanabe, Andreas Dengel, Shoya Ishimaru (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07404.txt.

Description

Sahana Subramanya and three co-authors tested how a model's reliability steers the humans who label images with its help, in a paper published in 2025. Labeling teams now pair people with a face-reading model to mark engagement in photos. The team asked what trust, workload, and care the paired work creates. Twenty-nine people took part in three settings: a sound model, a model with planted flaws, and a sound model presented with a negative frame.

Sound predictions bred high trust and steady labels. Planted flaws made labelers sharper but more frustrated and less consistent. The negative frame, oddly, made people rate the flawed model as more relatable and accurate, though they had been misled about its quality. The authors say paired work rests on two legs: true machine quality and the frame in the human mind. Their setup scales to wider labeling tasks.

Catalog record

CREATORSahana Yadnakudige Subramanya, Ko Watanabe, Andreas Dengel, Shoya Ishimaru
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-07404

Browse the full library · About the Tower of Babel