HateBuffer : Safeguarding Content Moderators’ Mental Well-Being through Hate Speech Content Modification Journal: PACMHCI Volume: 9 7 CSCW428 11 DOI: 10.1145/3757609 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Empirical studies in collaborative and social com

Subin Park, Jeonghyun Kim, Jeanne Choi, Joseph Seering, Uichin Lee, Sung-Ju Lee · 2025

Read HateBuffer : Safeguarding Content Moderators’ Mental Well-Being through Hate Speech Content Modification Journal: PACMHCI Volume: 9 7 CSCW428 11 DOI: 10.1145/3757609 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Empirical studies in collaborative and social com by Subin Park, Jeonghyun Kim, Jeanne Choi, Joseph Seering, Uichin Lee, Sung-Ju Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00439.txt.

Introduction

Subin Park and five co-authors build a buffer between hate speech and the people paid to review it, in a 2025 paper. The catalog title is padded with journal and subject metadata; the printed title is clean. Moderators absorb abuse all day, and the mental cost is real.

HateBuffer hides the targets of slurs, softens the wording, and lets moderators reveal the original on demand. Eighty people tried it on a mock news platform. They rated comments as less severe with the tool, yet their measured mood and fatigue did not improve. Interviews told another story: users felt shielded from emotional contagion and creeping bias. Accuracy held, and recall of violations slightly rose.

Catalog record

CREATORSubin Park, Jeonghyun Kim, Jeanne Choi, Joseph Seering, Uichin Lee, Sung-Ju Lee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE108 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00439

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HateBuffer : Safeguarding Content Moderators’ Mental Well-Being through Hate Speech Content Modification Journal: PACMHCI Volume: 9 7 CSCW428 11 DOI: 10.1145/3757609 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Empirical studies in collaborative and social computing Subin Park, Jeonghyun Kim, Jeanne Choi, Joseph Seering, Uichin Lee, Sung-Ju Lee Abstract . Hate speech remains a persistent and unresolved challenge in online platforms. Content moderators, working on the front lines to review user-generated content and shield viewers from hate speech, often find themselves unprotected from the mental burden as they continuously engage with offensive language. To safeguard moderators’ mental well-being, we designed HateBuffer , which anonymizes targets of hate speech, paraphrases offensive expressions into less offensive forms, and shows the original expressions when moderators opt to see them. Our user study with 80 participants consisted of a simulated hate speech moderation task set on a fictional news platform, followed by semi-structured interviews. Although participants rated the hate severity of comments lower while using HateBuffer , contrary to our expectations, they did not experience improved emotion or reduced fatigue compared with the control group. In interviews, however, participants described HateBuffer as an effective buffer against emotional contagion and the normalization of biased opinions in hate speech. Notably, HateBuffer did not compromise moderation accuracy and even contributed to a slight increase in recall. We explore possible explanations for the discrepancy between the perceived benefits of HateBuffer and its measured impact on mental well-being. We also underscore the promise of text-based content modification techniques as tools for a healthier content moderation environment. † † cc-license: by CONTENT WARNING : This paper contains hate speech examples, including pejorative terms, that readers may find disturbing. 1. Introduction As the volume of user-generated content continues to grow on social media ( Influencer Marketing Hub 2024 ) , the prevalence of harmful content, particularly hate speech, has become a significant concern ( UNESCO 2023 ) . Hate speech is defined as any form of communication in speech or writing that attacks or discriminates against a person or group based on their identity (e.g., religion, ethnicity, nationality, race, gender, or other factors) ( United Nations 2019 ) , and is a long-established issue. For example, users on X reported 66.9M instances of hate speech ( X (2024) formerly Twitter ) , and Facebook took action on 14.6M instances of hate speech in the first half of 2024 ( Meta (2024) Facebook ) . Hate speech can elicit strong negative emotions ( Heung et al. 2024 ) , resulting in serious psychological effects (e.g., trauma ( Scott et al. 2023 ; Thomas et al. 2022 ) or depression ( Wachs et al. 2022 ; Tynes et al. 2008 ) ) for readers, particularly the target and onlookers who identify with the targets. Given the significant harm that hate speech can cause to individuals and society at large by reinforcing harmful stereotypes ( Buerger 2021 ; Castaño-Pulgarín et al. 2021 ) , many platforms have employed commercial content moderators to review and moderate them ( Roberts 2016 ) . Many efforts have been developed for automated hate speech moderation, mostly based on heuristic rules ( Meta (nd) Facebook ; Meta (nd) Instagram ; Twitch nd ) and artificial intelligence (AI) ( Microsoft nd ; Google nd ) . However, these tools often fail to grasp the context and subtle nuances of hate speech ( AlKhamissi et al. 2023 ; Masud et al. 2021 ; Warner and Hirschberg 2012 ; Breitfeller et al. 2019 ) . In fact, X’s 2024 transparency report indicates that of the over 2M removed hate speech posts, 99.75% required human moderators to address nuanced content, with only 0.25% managed solely by automated systems ( X (2024) formerly Twitter ) . Given the persistent need for human judgment in hate speech moderation, platforms continue to recruit human moderators ( Indeed 2025 ; ZipRecruiter 2025 ; Amazon Jobs 2025 ) . These content moderators are at risk for a variety of detrimental impacts to their mental well-being due to regular exposure to harsh and offensive content, creating a challenging and emotionally demanding work environment ( Steiger et al. 2021 ; Roberts 2019 ) . Recent studies have shown that modifying the style of image- and video-based content that moderators review using approaches such as grayscaling, blurring, and adding cartooning ( Das et al. 2020a ; Karunakaran and Ramakrishan 2019 ; Lee et al. 2024 ) can reduce the emotional burden of moderating harmful content without compromising accuracy. However, the potential for similar modifications to text content remains underexplored. To safeguard the mental well-being of moderators who review hate speech, adapting content modification techniques for text content could be highly beneficial; however, unlike visual content, where immediate sensory stimuli often drive emotional impact, text-based hate speech derives its detrimental effect from the semantic meaning, making it challenging to directly apply visual content modification techniques. To address this, we designed HateBuffer , a text content modification system for hate speech moderation to alleviate moderators’ mental burdens while preserving moderation performance. HateBuffer consists of four features to modify the hate speech content. First, target_anonymization anonymizes the target group of potential hate speech to reduce the negative emotional impact on moderators caused by feeling attacked ( Cowan and Hodge 1996 ) or experiencing vicarious trauma ( Winchcomb 2019 ) . Second, paraphrasing_offensive paraphrases offensive expressions to less offensive versions to prevent the emotional contagion from reviewing offensive language ( Cheshin et al. 2011 ; Hancock et al. 2008 ; Ferrara and Yang 2015 ) . Lastly, revealing_target and revealing_original allow moderators to optionally view the original target and offensive expressions by clicking, providing control over whether to access the full content. To investigate whether HateBuffer can reduce moderators’ emotional burden without harming their performance, we aimed to answer the following research questions (RQs): • RQ1 : How does each feature of HateBuffer contribute to moderators’ mental well-being during hate speech moderation? • RQ2 : How does each feature of HateBuffer influence moderation strategies and contribute to moderators’ performance in hate speech moderation? To address these RQs, we conducted a between-subjects study with 80 participants. We distributed them into four groups: the control group as the baseline, the anonymizing group using target_anonymization , the paraphrasing group using paraphrasing_offensive , and the revealing group using HateBuffer with all features. For the user study, we selected 100 comments from the K-HATERS dataset ( Park et al. 2023 ) and applied text content modification, utilizing a Large Language Model (LLM) for paraphrasing_offensive . Participants were assigned the role of moderators to perform simulated hate speech moderation for a fictional news platform. Through surveys and semi-structured interviews, we investigated HateBuffer ’s impact on participants’ mental well-being and moderation performance. In contrast to our expectations, we did not detect a difference in post-study fatigue or negative emotion levels between the study conditions. However, participants in the text modification conditions rated the severity of hate speech lower than those in the control condition, and participants noted many positive aspects of the system in post-study interviews; they perceived HateBuffer as a buffer, providing time to prepare themselves to face the hateful content. Participants also noted that HateBuffer helped to protect

Read the complete text