Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South \conffull ( \confshort ), \confdate , \confloc Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3812296 ISBN: 979-8-4007-2596-8/2026/06 CCS: Human-centered computing Empirical studies in collaborative and social computing CCS: Computing methodologies Computer vision CCS: Social and professional topics Geographic characteristics CCS: Social and professional topics Cultural characteristics
Charvi Rastogi, Mukul Bhutani, Minsuk Kahng, Shamsuddeen Hassan Muhammad, Evgeniia Razumovskaia, Priyanka Suresh, Ibrahim Said Ahmad, Charu Kalia, Yaaseen Mahomed, Madhurima Maji, Minjae Lee, Alicia Parrish, Jessica Quaye, Vijay Janapa Reddi, Aishwarya Verma, Lora Aroyo · 2026
Read Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South \conffull ( \confshort ), \confdate , \confloc Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3812296 ISBN: 979-8-4007-2596-8/2026/06 CCS: Human-centered computing Empirical studies in collaborative and social computing CCS: Computing methodologies Computer vision CCS: Social and professional topics Geographic characteristics CCS: Social and professional topics Cultural characteristics by Charvi Rastogi, Mukul Bhutani, Minsuk Kahng, Shamsuddeen Hassan Muhammad, Evgeniia Razumovskaia, Priyanka Suresh, Ibrahim Said Ahmad, Charu Kalia, Yaaseen Mahomed, Madhurima Maji, Minjae Lee, Alicia Parrish, Jessica Quaye, Vijay Janapa Reddi, Aishwarya Verma, Lora Aroyo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19190.txt.
Introduction
Charvi Rastogi and 15 colleagues conduct localised red teaming for text-to-image safety in the Global South, where frameworks follow a Western-centric default. They gather local norms and contexts through workshops in secondary urban centres. PLACES holds over 26,000 model-failure examples from universities in Ghana, Nigeria, Karnataka, and Punjab. Prompts show socio-cultural and linguistic diversity and unique adversarial patterns from local nuances, plus theme clusters such as religion in India. The authors identify novel harms from normative dissonance, arguing safety needs localised, participatory methods.
Catalog record
| CREATOR | Charvi Rastogi, Mukul Bhutani, Minsuk Kahng, Shamsuddeen Hassan Muhammad, Evgeniia Razumovskaia, Priyanka Suresh, Ibrahim Said Ahmad, Charu Kalia, Yaaseen Mahomed, Madhurima Maji, Minjae Lee, Alicia Parrish, Jessica Quaye, Vijay Janapa Reddi, Aishwarya Verma, Lora Aroyo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90544 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19190
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South \conffull ( \confshort ), \confdate , \confloc Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3812296 ISBN: 979-8-4007-2596-8/2026/06 CCS: Human-centered computing Empirical studies in collaborative and social computing CCS: Computing methodologies Computer vision CCS: Social and professional topics Geographic characteristics CCS: Social and professional topics Cultural characteristics Charvi Rastogi, Mukul Bhutani, Minsuk Kahng, Shamsuddeen Hassan Muhammad, Evgeniia Razumovskaia, Priyanka Suresh, Ibrahim Said Ahmad, Charu Kalia, Yaaseen Mahomed, Madhurima Maji, Minjae Lee, Alicia Parrish, Jessica Quaye, Vijay Janapa Reddi, Aishwarya Verma, Lora Aroyo Abstract . Despite the global deployment of text-to-image (T2I) models, their safety frameworks are largely calibrated to a Western-centric default, creating significant vulnerabilities for the rest of the world. To embrace cultural pluralism and bring historically under-represented perspectives in T2I safety, we conduct localised community-centered red teaming studies in the Global South. Our two-fold approach prioritizes localization and participation, by focusing on secondary urban centers in these regions, and conducting community engagement and training workshops to contextualize local norms. As a result, we present PLACES, a dataset comprising over 26,000 examples of T2I model failures collected in partnership with universities in Ghana, Nigeria, and two regions of India (Karnataka and Punjab). Analysis of prompts collected reveals a wide-ranging diversity in socio-cultural and linguistic attributes, when compared to existing geography-agnostic crowdsourced red-teaming data. We observe unique adversarial patterns enabled by local cultural and linguistic nuances, and distinct clusters within region around specific themes, such as religion in India. Moreover, we uncover structural contextual gaps in existing safety frameworks by identifying novel harms showing normative dissonance (e.g., violating religious norms, ignoring local customs, and ominous symbolism). This work argues that expanding T2I safety requires moving beyond mere scale to incorporate deeply localized, participatory methodologies for data collection and contextualization. Content warning: This paper includes examples containing potentially harmful or offensive content. † † cc-license: by † † authorsaddresses: Corresponding author’s contact information: Charvi Rastogi, Google DeepMind, New York, USA, [email protected]. 1. Introduction While text-to-image generation (T2I) models have demonstrated remarkable capabilities, they remain vulnerable to a wide array of harmful behaviours, such as generation of offensive and inappropriate content, reinforcement of social biases, cultural erasure, and misinformation ( 100 ; 13 ; 26 ) . Research has heralded the “dangers of a single story” ( 2 ) , as T2I models with limited worldview and a purported Western gaze, flatten rich, pluralistic narratives of global populations ( 71 ; 60 ; 47 ) . Their widespread use across the globe has led to growing calls for geographically-diverse, community-centered, locally contextualised evaluations ( 22 ; 53 ) . This is particularly pertinent in the Global South where existing evaluation strategies and safety frameworks, optimized for WEIRD settings ( 15 ; 82 ; 25 ) fail to capture local context-specific harms unique to the regions’ social, cultural, and linguistic diversity ( 7 ; 37 ; 61 ; 45 ) , further amplifying existing disparities in AI safety ( 85 ; 8 ; 64 ) . Red teaming has emerged as a standard method in AI safety ( 65 ; 27 ) - probing models with adversarial prompts to uncover unknown model failures - often mandated by regulatory bodies ( 23 ) . Red teaming is often done by model developers, safety researchers and expert testers ( 23 ) at sites of model development, reflecting a similar geographical bias ( 42 ; 71 ) . More recently, in an effort to scale and diversify red teaming, open crowdsourcing methods have been piloted ( 72 ; 87 ) . However, without participatory engagement that respects local expertise and encourages community-driven definitions of harms, such methods produce data mostly tethered to popular (and Western) perspective on safety ( 69 ; 55 ) . Community-centered participatory methods grounded in the rich cultural and linguistic expertise and lived experiences of the community members have shown immense promise, contributing high-quality evaluation data from under-represented groups ( 55 ; 37 ; 31 ) . Thus, in this work, our red teaming approach bridges the reach of crowdsourcing with the diversity of community-centered methods, thereby combining broad scale with cultural depth. We present PLACES 1 1 1 https://huggingface.co/spaces/CharviRastogi/PLACES , a dataset obtained from novel large-scale community-driven T2I red teaming effort in partnership with four universities across India, Nigeria, and Ghana. We argue that surfacing novel locale-specific AI harms must go hand-in-hand with participatory methods. Specifically, we demonstrate a two-fold approach to red teaming with focus on localization and community-centered participation: (1) We situate our research in secondary urban centres in the Global South - Mangalore, Karnataka and Phagwara, Punjab in India, Cape Coast in Ghana, and Kano in Nigeria, which represent diverse users whose realities remain under-represented in AI safety; (2) We conduct educational workshops and focus groups on AI safety to contextualize community values and implement a scaffolded data collection with community ambassadors to enable bottom-up surfacing of region-specific, contextualised harms. To validate our methodology and examine the outcomes of the red teaming efforts, we conduct an in-depth analysis with two intentions: (1) comparing and analysing data from each of the four regions to highlight locale-specific insights, and (2) comparing the localized data against geography-agnostic data ( 72 ) collected in an online crowdsourced challenge to investigate the different affordances of each red teaming approach. Our analysis is guided by three research questions: • Cultural specificity: Each of the regions and communities in our red teaming studies has a distinct culture and demography. We investigate how references to region and community-specific cultural artifacts, such as food, art forms, social dynamics, and material culture, occur in localized red teaming. • Language use: Many regions in the Global South utilize code-mixing, specifically by blending local languages with English, in everyday communication. While code-mixing vulnerabilities are already documented in large language models (LLMs) ( 61 ; 62 ) , their impact on visual generation remains underexplored. Next, we ask how such localized linguistic patterns effectively bypass safety guardrails optimized for standard English and result in harmful outcomes. • Region-specific harms and attacks: Different regions may have different conceptualizations of model failures that do not fit in existing safety frameworks. We examine what are the unique failure modes and corresponding attack strategies revealed by this data in each region. Figure 1 . Visualization of prompts in red teaming dataset, PLACES, by projecting their embedding representations into two dimensions using UMAP. Prompts collected from four regions—Ghana, Nigeria, Karnataka (India), and Punjab (India)—exhibit distinctive clusters. While some themes such as professional roles, violence and sexually explicit content are shared across regions, several clusters capture region-specific cultural contexts, contributing to greater diversity in the dataset. A scatter plot showing a two-dimensional UMAP projection of dataset prompts, represented as individual dots colored by their region of origin: Ghana is blue, Nigeria is green, Karnataka, India is red, and Punjab, India is yellow. The main large plot on the left is annotated with text labels pointing to distinct thematic clusters. Clusters related to shared themes, such as "violence, blood,