Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain
Joshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo · 2026
Read Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain by Joshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28772.txt.
Description
Muhumuza, Agaba Ezra and Amiyo examine how hate speech datasets are labelled, focusing on HateXplain, a collection of social media posts judged by human annotators. Their concern is the common step of keeping only the majority vote, meaning the label most annotators chose, and throwing away the rest. They ask whether that step quietly erases the views of annotators in the minority.
Disagreement, they find, is not spread evenly. About 42.6 percent of it sits right on the border between hateful and merely offensive posts, which suggests judges use different thresholds for where hate starts. Three kinds of trained model all lost about 22 points of accuracy on disputed posts compared with agreed ones. Attempts to fix the problem afterwards failed, so the authors argue the fix must come earlier, when annotation itself is designed.
Catalog record
| CREATOR | Joshua Muhumuza, Joab Agaba Ezra, Mercy Amiyo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28772