Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain

Joshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo · 2026

Read Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain by Joshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28772.txt.

Description

Muhumuza, Agaba Ezra and Amiyo examine how hate speech datasets are labelled, focusing on HateXplain, a collection of social media posts judged by human annotators. Their concern is the common step of keeping only the majority vote, meaning the label most annotators chose, and throwing away the rest. They ask whether that step quietly erases the views of annotators in the minority.

Disagreement, they find, is not spread evenly. About 42.6 percent of it sits right on the border between hateful and merely offensive posts, which suggests judges use different thresholds for where hate starts. Three kinds of trained model all lost about 22 points of accuracy on disputed posts compared with agreed ones. Attempts to fix the problem afterwards failed, so the authors argue the fix must come earlier, when annotation itself is designed.

Catalog record

CREATORJoshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28772

Browse the full library · About the Tower of Babel