Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation
Chuancheng Shi, Shangze Li, Shiming Guo, Simiao Xie, Wenhua Wu, Jingtong Dou, [2pt] Chao Wu, Canran Xiao, Cong Wang, Zifeng Cheng, Fei Shen, Tat-Seng Chua, [4pt] The University of Sydney · 2025
Read Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation by Chuancheng Shi, Shangze Li, Shiming Guo, Simiao Xie, Wenhua Wu, Jingtong Dou, [2pt] Chao Wu, Canran Xiao, Cong Wang, Zifeng Cheng, Fei Shen, Tat-Seng Chua, [4pt] The University of Sydney (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17282.txt.
Introduction
Chuancheng Shi and 11 co-authors document a cultural blind spot in text-to-image models. Asked in different languages for culturally specific scenes, current models often return culturally neutral or English-biased pictures, even though language carries cultural connotations that the image should reflect. Studying two representative multilingual models, the authors find the problem is not missing knowledge: the cultural information is present in the network but too weakly activated.
A probing method locates the culture-sensitive signal in a small set of neurons within a few layers. Two fixes follow from that map. One amplifies those neurons at inference time, with no retraining; the other fine-tunes only the culturally relevant layers. On their CultureBench benchmark, both strategies improved cultural consistency over strong baselines while preserving image fidelity and diversity.
Catalog record
| CREATOR | Chuancheng Shi, Shangze Li, Shiming Guo, Simiao Xie, Wenhua Wu, Jingtong Dou, [2pt] Chao Wu, Canran Xiao, Cong Wang, Zifeng Cheng, Fei Shen, Tat-Seng Chua, [4pt] The University of Sydney |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-17282
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation Chuancheng Shi, Shangze Li, Shiming Guo, Simiao Xie, Wenhua Wu, Jingtong Dou, [2pt] Chao Wu, Canran Xiao, Cong Wang, Zifeng Cheng, Fei Shen, Tat-Seng Chua, [4pt] The University of Sydney Abstract Multilingual text-to-image (T2I) models have advanced rapidly in terms of visual realism and semantic alignment, and are now widely utilized. Yet outputs vary across cultural contexts: because language carries cultural connotations, images synthesized from multilingual prompts should preserve cross-lingual cultural consistency. We conduct a comprehensive analysis showing that current T2I models often produce culturally neutral or English-biased results under multilingual prompts. Analyses of two representative models indicate that the issue stems not from missing cultural knowledge but from insufficient activation of culture-related representations. We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers. Guided by this finding, we introduce two complementary alignment strategies: (1) inference-time cultural activation that amplifies the identified neurons without backbone fine-tuned; and (2) layer-targeted cultural enhancement that updates only culturally relevant layers. Experiments on our CultureBench demonstrate consistent improvements over strong baselines in cultural consistency while preserving fidelity and diversity. 1 Introduction Ensuring cultural fairness and representation in generative AI 5 ; 36 ; 42 ; 17 ; 13 ; 10 ; 20 ; 23 ; 40 is essential for global accessibility and cultural diversity, aligning with the United Nations’ principles of inclusiveness and universality. Yet, when prompted in different languages, many state-of-the-art methods 1 ; 48 ; 22 ; 34 ; 7 frequently produce culturally neutral or English-biased images, which weakens cross-lingual cultural correspondence. Here, we use cultural consistency to denote the extent to which generated images exhibit visual elements that are statistically associated with the target language’s cultural context, beyond mere semantic correctness. To avoid conflating culturally typical elements with stereotypes, we require that cultural cues be both contextually appropriate and roughly consistent with real-world cultural statistics. Figure 1 : Cultural alignment in local languages. (a) LLMs/recommenders keep cultural consistency, but T2I models falter with “noun-only” prompts. (b) Adding a “culture-style modifier + noun” restores consistency. Figure 2 : Overview of the CultureBench pipeline. First, manually collect and rigorously quality-control datasets from 15 linguistic regions; annotate “culture-style modifier noun” captions using GPT5-Nano 30 and, through human annotation, “noun-only” captions; convert annotated content into local languages via translation tools, supplemented by manual review. Despite strong gains in semantic and visual fidelity, diffusion-based T2I models 35 ; 37 lag in cultural generalization. Prior work largely targets cross-lingual encoder alignment 27 ; 48 while overlooking cross-cultural grounding, defined as generating visuals that reflect each language’s socio-cultural context. As shown in Figure 1 (a), models often capture only literal meanings, for example, “ a traditional building ” in Portuguese or Turkish, while missing culture-specific cues. In contrast, language-based systems such as LLMs 41 ; 30 ; 1 and recommender engines 32 ; 16 produce localized responses, revealing a cross-modal gap in cultural grounding. We argue that the issue is insufficient activation rather than missing knowledge. Large-scale training corpora already contain diverse cultural attributes, and explicit prompts can elicit them. As shown in Figure 1 (b), adding culture-style modifiers, for example, “ people wearing Chinese clothes ” or “ an Italian architecture ,” yields images with clear country-specific characteristics. The inconsistency happens because “noun-only” prompts don’t strongly trigger cultural knowledge, so the model gives literal but not cultural interpretations. To validate this hypothesis and mitigate cultural inconsistency in T2I models, we first examine two representative systems 48 ; 27 and show that the failure arises from insufficient activation of culture-related knowledge rather than its absence, and that the effect is observed across two architecturally different diffusion methods. We then introduce a two-stage probing method. We begin by comparing attention distributions between culture-style modifiers and nouns to localize culture-sensitive layers. Next, we use the Top-K SAE 4 to quantify activation differences between prompts with explicit cultural cues and “noun-only” prompts, revealing that culture-relevant representations cluster in a few fixed layers and a small set of neurons. Guided by these findings, we propose two complementary strategies: (1) a zero-training activation scheme that amplifies the responses of the identified neurons at inference time, and (2) a layer-specific fine-tuned scheme that updates only culturally relevant layers to improve consistency. Finally, experiments on our CultureBench show consistent improvements over strong baselines in cultural consistency while preserving fidelity and diversity. We highlight the following contributions: • We empirically show that multilingual T2I models often produce culturally neutral or subtly English-biased outputs under multilingual prompts, thereby hindering cross-lingual cultural consistency. • We develop a probing framework that localizes culture-sensitive signals to a few fixed layers and neurons by contrasting attention patterns and Top-K SAE activations, indicating that failures stem from insufficient activation rather than missing knowledge. • We propose two alignment strategies: a zero-training inference-time activation scheme and a layer-specific fine-tuning scheme that updates only culturally relevant layers, improving cultural consistency while preserving fidelity and diversity. • We curate and release CultureBench, a 15-country benchmark with multilingual prompts and images, enabling evaluation of cross-cultural consistency and culturally adaptive training. 2 Related Work Cultural Text-to-Image Generation. Prior work has examined cultural bias, fairness, and stereotyping in T2I systems 9 ; 26 ; 18 . For example, SCoFT 25 expands cultural coverage through CCUB to mitigate stereotyping, and ViSAGe 19 quantifies visual stereotypes. However, existing studies remain largely focused on fairness and debiasing, with a strong English-centric orientation and limited coverage of low-resource languages and diverse cultural contexts 6 ; 39 . Clear definitions and systematic evaluation of cross-lingual cultural consistency are still lacking. Noun-only or short prompts often collapse to culturally neutral or implicitly English-biased generations in multilingual settings, yet a unified diagnostic framework is absent 24 . At the representational level, prior work provides little insight into where culture-sensitive features reside within the model or how to control them at the layer or neuron level 21 ; 31 . Methodologically, there is still a lack of lightweight, plug-and-play interventions and corresponding benchmarks that do not require large-scale retraining. Figure 3 : Data distribution of the proposed CultureBench dataset across 15 languages. The dataset is divided into train, test, and neuron-detection subsets with a ratio of 7:2:1. Neuron Interpretability. Neuron-level interpretability links model activations to learned concepts in vision and language 29 ; 15 ; 2 ; 49 . Recent work further distinguishes shared and language-specific semantics at the neuron or direction level, clarifying how abstract meanings distribute across layers 14 ; 41 ; 45 . In multimodal settings, causal probing shows that individual neurons can be driven by visual concepts via textual proxies and directly manipulated to control semantics 38 ; 33 ; 11 . For concreteness, FEMN 33 locates sparse neuron groups mediating specific concepts (e.g., “smile,” “striped”) in a CLIP-style model and demonstrates that directly patching or amplifying those units causally increases or suppresses the target concept in zero-shot classification and retrieval, establishing neuron-level semantic control. However, prior studies still focus mainly on object- or attribute-level notions and provide limited diagnostics for cultural representations, especially those that vary under multilingual prompts 28 ; 8 ; 43 . 3 CultureBench Dataset Data Collection. As