Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation
Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee · 2025
Read Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation by Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11966.txt.
Introduction
Ziyu Ge and co-authors translate Singlish slang without scrubbing its bite in a 2025 paper. Singlish mixes English with local tongues, and standard translators flatten its slang and miss its harmful expressions. Safety filters also sanitize insults that moderators need to see accurately. Their two-stage framework aims to keep tone and toxicity intact across translation.
First, human-verified example prompts teach the model local phrasing, ranked by annotators. Second, model and prompt pairs are benchmarked with meaning checks that run translation both ways. Human judges confirmed the pipeline's quality and speed. The authors frame Singlish as a testbed for fairer language technology. Keeping nuance, even rude nuance, makes moderation tools work in the cultures they police.
Catalog record
| CREATOR | Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-11966
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee Abstract As online communication increasingly incorporates under-represented languages and colloquial dialects, standard translation systems often fail to preserve local slang, code-mixing, and culturally embedded markers of harmful speech. Translating toxic content between low-resource language pairs poses additional challenges due to scarce parallel data and safety filters that sanitize offensive expressions. In this work, we propose a reproducible, two-stage framework for toxicity-preserving translation, demonstrated on a code-mixed Singlish safety corpus. First, we perform human-verified few-shot prompt engineering: we iteratively curate and rank annotator-selected Singlish–target examples to capture nuanced slang, tone, and toxicity. Second, we optimize model–prompt pairs by benchmarking several large language models using semantic similarity via direct and back-translation. Quantitative human evaluation confirms the effectiveness and efficiency of our pipeline. Beyond improving translation quality, our framework contributes to the safety of multicultural LLMs by supporting culturally sensitive moderation and benchmarking in low-resource contexts. By positioning Singlish as a testbed for inclusive NLP, we underscore the importance of preserving sociolinguistic nuance in real-world applications such as content moderation and regional platform governance. 1 Introduction Recent advances in large language models (LLMs) have significantly improved machine translation, achieving strong performance on many language pairs with only a few carefully selected examples Vilar et al. 2023 ; Brown et al. 2020 . Prompt-based approaches allow LLMs to rapidly adapt to new domains and languages, delivering high fluency and adequacy Haddow et al. 2022 . These developments are based on a rich history of machine translation research: from early statistical methods Lopez 2008 ; Wang et al. 2017 and neural sequence-to-sequence models Stahlberg 2020 , to multilingual NMT systems that enable zero-shot translation Johnson et al. 2017 and unsupervised approaches that bypass the need for parallel corpora Lample et al. 2017 ; Artetxe et al. 2017 . Although LLMs now rival traditional systems in formal high-resource languages Hendy et al. 2023 ; Karpinska & Iyyer 2023 , their performance remains limited in inputs rooted in low-resource, informal, or culturally embedded Robinson et al. 2023 ; Haddow et al. 2022 . Recent work shows that code-mixed languages like Singlish, characterized by slang, emotive tone, and loanwords, challenge generic prompting strategies Ng & Chan 2024 . Similarly, Enis & Hopkins 2024 demonstrates that even top-performing models like Claude 3 struggle with translation fidelity on low-resource pairs, though their outputs can be distilled into smaller systems. Translating toxic or harmful content in such settings presents further challenges: safety filters in LLMs often sanitize offensive expressions, and standard translation pipelines lack sensitivity to sociolinguistic cues Costa-jussà et al. 2022 . This problem is compounded in low-resource contexts where parallel corpora and annotated toxicity benchmarks are scarce. Singlish, a creole blend of English, Malay, Hokkien, and other regional languages, exemplifies these issues: it features rich code-mixing and culturally embedded markers of harm that often fall outside the representational scope of standard multilingual embeddings Pratapa et al. 2018 . When these nuances are not preserved, translations risk diluting critical signals, undermining downstream tasks such as content moderation or sentiment analysis. In this work, we propose a two-stage, human-in-the-loop framework for toxicity-preserving translation, aimed at enhancing multicultural LLM safety. The workflow is shown in Figure 1 . Our approach is demonstrated on a Singlish safety corpus constructed from LionGuard Foo & Khoo 2024 , and applied to Chinese, Malay, and Tamil—languages with varying degrees of institutional and digital support in Singapore’s multilingual society. We treat Singlish not merely as a linguistic artifact, but as a testbed for inclusive NLP, offering insight into how LLMs handle culturally situated expressions of harm. The pipeline begins with curating a balanced set of Singlish–target examples through iterative annotator selection and ranking, capturing variations in slang, tone, and toxicity. We then evaluate different LLMs and prompt configurations using semantic similarity metrics computed through direct and back-translation, enabling reference-free assessment at scale. Human evaluations confirm that our pipeline effectively preserves both semantic content and harmful tone with minimal annotation overhead. By foregrounding culturally sensitive translation in low-resource contexts, this work contributes not only to inclusive model evaluation but also to practical applications such as multilingual content moderation, trust and safety tools, and regional platform governance. Figure 1: The proposed framework for toxicity-preserving translation. 2 Methodology Unlike standard multilingual benchmarks, our objective is to preserve both the semantic content and the expressive level of harmfulness in each input. This dual objective introduces challenges: Most translation models either sanitize toxic content, due to embedded safety filters, or mistranslate culturally embedded phrases, particularly in informal or slang-heavy languages like Singlish. To address these limitations, we propose a two-stage, human-in-the-loop framework designed to maintain multicultural fidelity in low-resource translation. 2.1 Human-Curated Few-Shot Examples Standard LLM-based translation often neutralizes harmful language or fails to capture the expressive tone of informal slang. To mitigate this, we curated a compact but diverse set of high-quality translation examples to guide model outputs toward faithful, toxicity-preserving behavior. We selected 20 Singlish sentences, balanced between benign and harmful content, and subjected them to a structured, three-round human verification process to construct the final few-shot prompt pool. 2.1.1 Annotation Procedure Each sentence underwent the following iterative refinement steps: Round 1 – Broad Candidate Selection. We generated three zero-shot translations per sentence using GPT-4o mini OpenAI 2024a , DeepSeek-R1 DeepSeek-AI & Others 2025 , and Gemini 2.0 Flash Google 2025 . Annotators reviewed all outputs and could (a) select any number of acceptable translations, or (b) submit a custom translation if none captured the intended tone and meaning. Round 2 – Focused Comparison. The top two LLM-generated outputs from Round 1, along with any human-provided alternatives, were reviewed. Annotators selected up to two preferred candidates to refine the pool. Round 3 – Final Selection. The remaining candidates were ranked, and annotators selected the single best translation for inclusion. The version receiving the most votes across annotators was adopted for the final prompt set. This multi-stage design enables efficient human oversight while minimizing manual translation workload. It ensures both fidelity and tone preservation through controlled iteration. The proportion of LLM-generated outputs retained in the final pool also serves as a proxy for model reliability in culturally sensitive translation tasks. Additional interface details and annotation screenshots are provided in Appendix C.1 . 2.1.2 Results and Analysis In the final few-shot prompt pool, both Chinese and Tamil retained nine LLM-generated translations, whereas Malay retained only two. Correspondingly, we observed more custom (i.e., human-provided) translations for Malay—averaging 8.8 per sentence—compared to 6.4 for Chinese and 5.6 for Tamil. Based on annotator feedback and output inspection, this lower retention rate for Malay appears to stem from orthographic variability: annotators frequently substituted standard lexical forms with colloquial spellings that better captured the expressive tone of the original Singlish. To assess the surface-level alignment between the selected final examples and the original LLM outputs, we computed character-level substring overlap. The results yielded a median overlap score of 0.47 and an average of 0.54, indicating moderate textual similarity between curated examples and candidate translations. Additional statistics, including custom submission counts in language and interannotator agreement between rounds, are reported in the Appendix C.2 . Notably, inter-annotator agreement improved steadily from Round