Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations

David Hartmann, Amin Oueslati, Dimitri Staufer, Lena Pohlmann, Simon Munzert, Hendrik Heuer · 2025

Read Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations by David Hartmann, Amin Oueslati, Dimitri Staufer, Lena Pohlmann, Simon Munzert, Hendrik Heuer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01623.txt.

Introduction

David Hartmann and co-authors audit five commercial content moderation services for bias in policing hate speech. Such services promise scale, but they can silence fair speech and miss coded hate at the same time. The team sent five million test queries drawn from four datasets.

All five services leaned hard on group identity words when flagging hate. OpenAI and Amazon did slightly better than the rest. Every service under-flagged implied hate, above all against LGBTQIA+ people. Every one also over-flagged counter-speech and reclaimed slurs from the targeted groups. The authors urge clearer guidance and candor about limits from the providers.

Catalog record

CREATORDavid Hartmann, Amin Oueslati, Dimitri Staufer, Lena Pohlmann, Simon Munzert, Hendrik Heuer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE153 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01623

Browse the full library · About the Tower of Babel