Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore’s Low-Resource Languages
Yujia Hu, Ming Shan Hee, Preslav Nakov, Roy Ka-Wei Lee · 2025
Read Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore’s Low-Resource Languages by Yujia Hu, Ming Shan Hee, Preslav Nakov, Roy Ka-Wei Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15260.txt.
Introduction
Yujia Hu, Ming Shan Hee, Preslav Nakov, and Roy Ka-Wei Lee probe the safety of language models in the tongues of Singapore in a 2025 paper. Safety tests focus on English and a few big languages, which leaves low-resource settings exposed. The mix in Singapore spans Singlish, Chinese, Malay, and Tamil, a hard test for guards that span tongues.
Their data set and frame, SGToxicGuard, probe models in a red-team style across chat, question answering, and writing tasks. Broad runs with top models that span tongues expose key gaps in their guards. The gaps grow where local norms shape what counts as toxic. The authors draw lessons for cultural care and harm cuts, and share the bench so builders can test safety where users live.
Catalog record
| CREATOR | Yujia Hu, Ming Shan Hee, Preslav Nakov, Roy Ka-Wei Lee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-15260
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore’s Low-Resource Languages Yujia Hu, Ming Shan Hee, Preslav Nakov, Roy Ka-Wei Lee Abstract The advancement of Large Language Models (LLMs) has transformed natural language processing; however, their safety mechanisms remain under-explored in low-resource, multilingual settings. Here, we aim to bridge this gap. In particular, we introduce SGToxicGuard , a novel dataset and evaluation framework for benchmarking LLM safety in Singapore’s diverse linguistic context, including Singlish, Chinese, Malay, and Tamil. SGToxicGuard adopts a red-teaming approach to systematically probe LLM vulnerabilities in three real-world scenarios: conversation , question-answering , and content composition . We conduct extensive experiments with state-of-the-art multilingual LLMs, and the results uncover critical gaps in their safety guardrails. By offering actionable insights into cultural sensitivity and toxicity mitigation, we lay the foundation for safer and more inclusive AI systems in linguistically diverse environments. 1 1 1 Link to the dataset: https://github.com/Social-AI-Studio/SGToxicGuard . Disclaimer: This paper contains sensitive content that may be disturbing to some readers. 1 Introduction Figure 1: Examples of toxicity red-teaming tasks. Motivation. The rapid proliferation of large language models (LLMs) has introduced unprecedented capabilities in natural language processing, enabling Artificial Intelligence (AI) applications ranging from multi-turn conversational dialogues to multilingual content generation Dwivedi et al. 2023 ; Gottlieb et al. 2023 ; Deng et al. 2024 . However, alongside these advancements, concerns regarding AI safety, ethical deployment, and content moderation have grown, particularly in managing harmful or toxic content Wang and Kim 2023 ; Anjum and Katarya 2024 . As LLMs are increasingly integrated into real-world applications, it is paramount to ensure their responsible operation in diverse linguistic and cultural settings. Red-teaming approaches have emerged as a prominent method for systematically probing LLM vulnerabilities Perez et al. 2022 ; Ganguli et al. 2022 . Techniques such as adversarial input crafting Papernot et al. 2016 , scenario-based evaluations Carroll 1997 , and boundary condition testing Papanastasiou et al. 1992 have proven effective for detecting biases, toxic content, and other risks. However, these efforts have been largely concentrated on high-resource languages, such as English, thus leaving critical gaps in low-resource multilingual settings. Limited data availability, combined with unique linguistic variability and cultural nuances, exposes models to new vulnerabilities, increasing the risk of biased and toxic content generation in low-resource multilingual contexts. Such vulnerabilities can be exploited to perpetuate social biases and exacerbate social divides within marginalized communities. To address these challenges, there is a growing need for red-teaming methodologies that are explicitly tailored to low-resource, multilingual environments. Singapore presents a uniquely relevant testbed for evaluating LLM safety in low-resource, multilingual contexts. The country’s linguistic landscape encompasses English, Singlish (a creole-like vernacular), Chinese, Malay, and Tamil, with frequent code-switching, culturally specific expressions, and sociolectal variations. These local linguistic factors present new challenges for AI safety evaluation that go beyond traditional toxicity detection frameworks, which were primarily designed for English. While recent efforts, such as SeaLLM Zhang et al. 2024 and SEA-LION Singapore 2024 , have developed LLMs tailored to Southeast-Asian languages, the effectiveness of their safety mechanisms in minority languages remains underexplored. Existing benchmarks, such as SGHateCheck Ng et al. 2024 , provide valuable functional tests for hate speech detection, but do not systematically evaluate LLM vulnerabilities under adversarial attacks. Given the increasing adoption of AI-driven productivity tools in Singapore and similar multilingual societies, developing a rigorous red-teaming evaluation framework is essential for advancing inclusive and contextually aware AI safety research. Research Objectives. To address gaps in AI safety evaluation, we introduce SGToxicGuard , a novel dataset and evaluation framework for benchmarking LLM safety in Singapore’s low-resource languages. Unlike SGHateCheck, which focuses on hate speech detection, SGToxicGuard adopts an adversarial red-teaming approach to systematically probe LLM vulnerabilities in exacerbating social biases. Our study targets Singlish, Malay, and Tamil as primary low-resource languages, alongside English and Chinese to reflect Singapore’s multilingual society. We assess LLM safety through three real-world-inspired tasks: (1) conversational safety, evaluating responses to toxic dialogue prompts; (2) toxic bias detection in question-answering; and (3) content moderation in composition, testing LLM susceptibility to generating disseminable toxic content. Examples of each task are shown in Figure 1 . By addressing linguistic and cultural toxicity challenges, our work provides critical insights into LLM vulnerabilities in diverse, multilingual contexts. Contributions. Our work makes the following key contributions: ( i ) We present SGToxicGuard , the first multilingual dataset designed for red-teaming LLMs in low-resource environments. ( ii ) We propose a three-task evaluation framework, Conversational Safety , Toxic QA , and Content Composition , that systematically assesses LLMs’ ability to handle Singapore-specific toxic content in real-world applications. ( iii ) Our study offers the first large-scale benchmarking of multilingual LLMs across Singapore’s low-resource languages, highlighting strengths and limitations in existing AI safety implementations. By demonstrating the vulnerabilities of LLMs for multilingual toxicity detection, we advance AI safety for linguistically diverse societies. Our findings contribute to ongoing discussions on AI ethics, content moderation policies, and robustness in multilingual AI deployments. 2 Related Work 2.1 Red-Teaming for AI Safety Red-teaming systematically identifies vulnerabilities in AI systems by simulating adversarial scenarios to test model robustness and ethical reliability Ganguli et al. 2022 ; Perez et al. 2022 ; Zhuo et al. 2023 . In LLMs, it plays a crucial role in uncovering biases and harmful content generation Feffer et al. 2024 ; Teichmann and Boticiu 2023 . Previous studies demonstrate the effectiveness through adversarial input crafting, boundary condition testing, and scenario-based evaluations Feffer et al. 2024 ; Teichmann and Boticiu 2023 ; Zhuo et al. 2023 . However, research has focused mainly on English, with limited exploration of other languages Ropers et al. 2024 . Singapore’s linguistic landscape, which includes English, Singlish, Mandarin, Malay, and Tamil, introduces unique challenges such as code switching and cultural specificity, which existing frameworks often overlook. We address this gap by proposing a Singapore-focused red-teaming methodology to benchmark LLM safety across multilingual, real-world scenarios. 2.2 Multilingual Toxicity Detection As digital interactions grow increasingly multilingual, toxicity detection across languages has become critical. Early work focused largely on English Waseem and Hovy 2016 ; Davidson et al. 2017 ; Alkomah and Ma 2022 ; Wang et al. 2024a , with cultural factors often overlooked and annotated resources scarce for many languages Aluru et al. 2020 ; Corazza et al. 2020 ; Röttger et al. 2022 ; Ashraf et al. 2025 ; Wang et al. 2024b . Recent advances in multilingual embeddings and LLMs, such as mBERT Pires et al. 2019 , XLM-RoBERTa Conneau et al. 2020 , GPT-3 Achiam et al. 2023 , LLaMA Dubey et al. 2024 , and Mistral Jiang et al. 2023 , have improved cross-lingual transfer, but also amplified concerns about bias Tedeschi et al. 2024 . We extend this line of research by evaluating LLM toxicity detection using SGHateCheck Ng et al. 2024 , a curated dataset offering parallel toxic content across Singlish, Chinese, Malay, and Tamil. Unlike prior Southeast Asian datasets Maity et al. 2023 ; Lu et al. 2023 ; Chakravarthi et al. 2021 , SGHateCheck enables structured multilingual comparisons, revealing disparities in LLM safety and informing the development of culturally sensitive AI moderation frameworks. Examples of localized hateful content containing local slurs and expressions across Singlish,