PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages

2025

Read PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04377.txt.

Introduction

Priyanshu Kumar and six co-authors release a safety checker for large language models that works across many tongues, in a paper from 2025. Past checkers leaned on English and a few big tongues, and used a narrow sense of harm, leaving gaps for the wider world.

Their model, PolyGuard, learned from the largest such training set yet, with samples in seventeen tongues. A new test set pairs prompts and replies with harm and refusal labels. Across safety and toxicity tests, PolyGuard beat both open and paid rival checkers. The team shared the model and both data sets for public use.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-04377

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages Abstract Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard , a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the corresponding training and evaluation datasets. PolyGuard is trained on PolyGuardMix , the largest multilingual safety training corpus to date containing 1.91M samples across 17 languages (e.g., Chinese, Czech, English, Hindi). We also introduce PolyGuardPrompts , a high quality multilingual benchmark with 29K samples for the evaluation of safety guardrails. Created by combining naturally occurring multilingual human-LLM interactions and human-verified machine translations of an English-only safety dataset ( Han et al., 2024 , WildGuardMix;) , our datasets contain prompt-output pairs with labels of prompt harmfulness , response harmfulness , and response refusal . Through extensive evaluations across multiple safety and toxicity benchmarks, we demonstrate that PolyGuard outperforms existing state-of-the-art open-weight and commercial safety classifiers by 5.5%. Our contributions advance efforts toward safer multilingual LLMs for all global users. PolyGuard Collection kpriyanshu256/polyguard Priyanshu Kumar ♡ 1 1 1 Equal contributors, correspondence at [email protected] . Devansh Jain ♡ 1 1 1 Equal contributors, correspondence at [email protected] . Akhila Yerukola ♡ Liwei Jiang ♠ Himanshu Beniwal △ ♢ Thomas Hartvigsen ♢ Maarten Sap ♡ ♣ ♡ Carnegie Mellon University ♠ University of Washington △ IIT Gandhinagar ♢ University of Virginia ♣ Allen Institute for AI 1 Introduction Recent advances in large language models (LLMs), especially their multilingual capabilities, have led to their deployment to a diverse global user base that spans multiple languages. Despite this global reach, safety research has focused primarily on the English language ( Ghosh et al., 2024 ; Ghosh et al., ; Han et al., 2024 ) , exposing global users to potential safety risks such as harmful content and privacy violations. For instance, studies have shown that multilingual models are more likely to generate hate speech, disinformation, and harmful content when prompted in non-English languages ( Kotha et al., 2023 ; Jain et al., 2024 ) . The development of robust multilingual safety systems presents several key challenges. First, building multilingual systems is inherently difficult due to challenges such as the lack of comprehensive datasets, the “curse of multilinguality” ( Aharoni et al., 2019 ; Conneau et al., 2020 ; Gurgurov et al., 2024 ) , and the inherent biases embedded in training corpora ( Xu et al., 2024 ) . Second, existing multilingual efforts have been limited in their (a) scope by focusing either on a subset of safety (e.g., PerspectiveAPI covering only toxicity, ignoring other unsafe content) and/or on a narrow set of language coverage ( Inan et al., 2023 ; Jain et al., 2024 ; Deng et al., 2025 , e.g., Llama-Guard-1 only covering English safety, ignoring toxicity and DuoGuard being evaluated on 4 very high resource languages only;) , or (b) performance ( Dubey et al., 2024 ; PatronusAI, 2024 , e.g., Llama-Guard-3-8B which struggles on multilingual benchmarks;) . Finally, most existing safety frameworks address only the single task of classifying safety and often rely on simplistic binary settings ( safe / unsafe ), which fail to capture the complex spectrum of harmful content that can manifest differently across cultural and linguistic contexts ( Sap et al., 2020 ; Zhou et al., 2023 ) . To address these gaps, we release PolyGuard ( PG ), a new state-of-the-art fine-tuned language model for multi-task safety detection and moderation. As Figure 1 highlights, PG can classify a multilingual input of a user prompt and an LLM response on five dimensions. Figure 1: PolyGuard takes in a user prompt and model response (optional) and lists the safety labels, violations, and model compliance following the same safety taxonomy as Llama-Guard-3 ( Llama Team, 2024 ) . Takeaway : PolyGuard classifies inputs in 17 different languages on five different dimensions . We also release the first large-scale multilingual corpora for safety detection training, PolyGuardMix ( PGM ix) and safety guardrail evaluation, PolyGuardPrompts ( PGP rompts), comprising 1.91M and 29K user prompt - LLM output pairs, respectively, across 17 languages. Our datasets contain binary and categorical labels for prompt harmfulness and response harmfulness , and response refusal (if the LLM response complies with the user request). We use a systematic labeling process that leverages a panel of English safety classifiers and LLM-as-a-judge (proprietary and open-weight LLM) to obtain these labels. We create our PGM ix dataset by combining both: (a) naturally occurring multilingual human-LLM interactions from In-The-Wild (ITW) datasets, and (b) machine translations of WildGuardMix ( Han et al., 2024 ) , to ensure data diversity which is crucial for improved model performance ( Davani et al., 2024 ) . We utilize multiple LLMs to ensure high-quality translations of WildGuardMix, verified by a high average translation score of 81.15 as rated by human annotators. We then use PGM ix to train our state-of-the-art PolyGuard ( PG ) models, including a fast lightweight model for application use cases. Our empirical results show that PG outperforms existing open-source and proprietary safety detectors on English-only as well as multilingual safety and toxicity benchmarks. Furthermore, we find that the incorporation of ITW samples in the training datasets makes PG models more robust to various data distributions, including code-switched and translated data. Overall, our datasets and models 2 2 2 Model, code, and data are available under the ODC-BY license. serve as a starting point for building powerful and robust multilingual safety detectors and advance efforts towards multilingual safe AI systems. 2 Dataset To address the critical need for multilingual safety detection, we introduce PolyGuardMix ( PGM ix) and PolyGuardPrompts ( PGP rompts), multilingual datasets specifically designed to train and evaluate robust safety classifiers. PGM ix comprises 1.91M human-LLM interactions, including 1.47M machine-translated samples from WildGuardMix and 0.43M naturally-occurring samples from In-The-Wild datasets, whereas PGP rompts comprises 29K translated samples. Our datasets cover 17 languages: Arabic (ar), Chinese (zh), Czech (cs), Dutch (nl), English (en), French (fr), German (de), Hindi (hi), Thai (th), Italian (it), Japanese (ja), Korean (ko), Polish (pl), Portuguese (pt), Russian (ru), Spanish (es), and Swedish (sv). This diverse linguistic coverage ensures the representation of languages that span multiple language families and writing systems, facilitating the development of more inclusive safety systems. Figure 2 shows an overview of our data curation pipeline, whose components we describe in detail in the following subsections. Figure 2: Data curation process for PGM ix (safety detection training) and PGP rompts (safety guardrail evaluation). Takeaway : PGM ix combines machine-translated and naturally occurring data to improve data diversity and, consequently, model performance. 2.1 Data Sources Both PGM ix and PGP rompts are constructed from the train and test samples of WildGuardMix ( Han et al., 2024 ) , a dataset of synthetic and natural single-turn human-LLM interactions with fine-grained annotations, respectively. In addition, PGM ix also contains samples from In-The-Wild datasets: LMSys-Chat-1M ( Zheng et al., 2023 ) and WildChat ( Zhao et al., 2024 ) 3 3 3 WildChat-1M is available for modifications under the

Read the complete text