Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen · 2026
Read Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency by Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22137.txt.
Introduction
Soegeng and co-authors propose cross-lingual consensus for aligning multilingual cultural knowledge via self-consistency. Large language models perform unevenly across languages, and English prompting induces a Western-centric bias that hinders reflection of diverse cultural knowledge. The authors hypothesize models already hold rich cultural knowledge in local languages, with a mechanism to transfer it to weaker languages. On the BLEnD benchmark, their approach improves cultural alignment, boosting English query performance by an average of 5.03 percent using only their own data.
Catalog record
| CREATOR | Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19699 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22137
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang Nguyen Abstract Although Large Language Models (LLMs) demonstrate strong capabilities across various tasks, they exhibit significant performance discrepancies across languages. While prompting LLMs in English typically yields the highest general performance, it often induces a Western-centric bias, hindering the model’s ability to accurately reflect diverse cultural knowledge. We hypothesize that LLMs already possess rich cultural knowledge embedded within local-language representations, but fail to retrieve it when prompted in English. To bridge this cross-lingual knowledge gap, we propose a novel self-supervised framework. Our method leverages multilingual self-consistency to identify the most reliable cultural responses across languages, combined with a self-critique mechanism to transfer this knowledge to the weaker language. Evaluations on the BLEnD benchmark demonstrate that our approach significantly improves cultural alignment—boosting performance on English queries by an average of 5.03%—relying entirely on self-generated data. Ultimately, our work demonstrates that latent cultural knowledge can be successfully surfaced and propagated across languages, enabling more culturally equitable and consistent LLMs. rmTeXGyreTermesX [*devanagari]rmLohit Devanagari [*arabic]rmNoto Sans Arabic \IfFontExistsTF Noto Sans CJK SC[chinese]rm[AutoFakeSlant=0.15,SmallCapsFont=Noto Sans CJK SC]Noto Sans CJK SC 1 Introduction Large Language Models (LLMs) have achieved remarkable progress across diverse natural language processing tasks, including logical reasoning Mondorf and Plank 2024 , question answering Brown et al. 2020 , and multilingual understanding Workshop et al. 2023 . Nevertheless, despite their strong overall performance, these models often exhibit uneven behavior across different languages Huang et al. 2023 and struggle to generate culturally appropriate responses when applied beyond dominant linguistic and cultural contexts Naous et al. 2024 . A key challenge for multilingual LLMs is their inability to access knowledge across languages consistently. Although models often achieve the strongest performance when prompted in English, this can introduce Western-centric bias and limit their ability to reflect diverse cultural knowledge. Interestingly, the same models may produce more culturally appropriate responses when prompted in local languages Ying et al. 2025 ; Myung et al. 2024 , suggesting that relevant knowledge is already present but not effectively retrieved or transferred across languages. This mismatch limits the reliability of LLMs in diverse real-world settings. To address these limitations, prior work has explored several directions for improving cross-lingual and cultural alignment in LLMs. Prompt-based methods attempt to inject cultural knowledge at inference time Wang et al. 2024 , but often require careful design and fail to capture deeper cultural understanding DURMUS et al. 2024 ; Kovač et al. 2023 . Training-based approaches instead rely on curated datasets from surveys, social media, or multilingual sources to enhance cultural awareness Li et al. 2024a ; Shi et al. 2024 ; Adilazuarda et al. 2025 , though these methods are costly and difficult to scale. More recent work leverages critique-based data synthesis and self-consistency to improve cultural knowledge and model reliability Feng et al. 2025 ; Wang et al. 2025 . However, these approaches either depend on stronger external models or are primarily evaluated in structured settings, leaving open challenges for scalable and robust alignment in open-ended generation. In this work, we propose a novel framework that leverages multilingual self-consistency to improve cultural knowledge alignment across languages. Instead of relying on external annotations, our method exploits the model’s own responses across multiple languages to identify reliable knowledge. By comparing response consistency across languages, we can determine which language yields more stable and coherent answers and use this signal to construct self-supervised training data. Our contributions can be summarized as follows: • We propose a self-supervised multilingual self-consistency framework to generate reliable training signals without human annotation. • We introduce a cross-lingual knowledge transfer mechanism that leverages stronger-language responses to improve weaker-language performance. 2 Related Works Cross-Lingual Performance Disparities Multilingual Large Language Models (LLMs) have been shown to yield different answers when a query is posed in different languages, leading to high performance variance across languages ( Xuan et al. 2025 ; Bandarkar et al. 2024 ; Ponti et al. 2020 ) . This discrepancy is primarily attributed to English-skewed training data, which causes models to route their internal reasoning through English ( Weihua et al. 2026 ; Schut et al. 2025 ; Wendler et al. 2024 ) . Consequently, as the target language drifts further from English, models suffer significant performance degradation, particularly in low-resource languages ( Huang et al. 2024 ) . Conversely, in some cases, LLMs may possess cultural knowledge in a local language but fail to retrieve or translate it when prompted in English ( Myung et al. 2024 ) . Cultural Bias and Western-Centricity Cultural bias has emerged as a significant concern in current LLMs as these models tend to reflect the dominant values of their pretraining corpora, often marginalizing other demographic groups ( Li et al. 2025 ) . Consequently, LLMs predominantly exhibit a bias toward Western values ( Mushtaq et al. 2025 ; Li et al. 2024b ) . Moreover, simply prompting these models to adopt a specific cultural perspective often yields answers grounded in superficial stereotypes rather than a deep understanding of the underlying cultural nuances ( DURMUS et al. 2024 ) . Such biases raise serious safety concerns, particularly regarding the deployment of LLMs in underrepresented or localized cultural contexts ( Azmi et al. 2025 ) . Approaches to Cultural Alignment Various strategies attempt to mitigate cultural bias in LLMs. One approach aims to directly inject cultural knowledge via prompting ( Wang et al. 2024 ) . However, such methods often yield only a shallow understanding and require extensive domain expertise to design ( DURMUS et al. 2024 ; Kovač et al. 2023 ) . Other approaches fine-tune models to improve their awareness of specific cultures. These methods often involve curating training datasets from surveys ( Li et al. 2024a ) , social media ( Shi et al. 2024 ) , or a combination of diverse sources ( Adilazuarda et al. 2025 ) . Recent work also leverages stronger LLMs to generate critiques that improve cultural data quality ( Feng et al. 2025 ) . Another line of research demonstrates that self-supervision via self-consistency can enhance cultural knowledge, even without relying on stronger models ( Wang et al. 2025 ; Zhang et al. 2025a ) . Building on these foundations, our work demonstrates how to further enhance cultural alignment in open-ended generation by combining self-critique and self-consistency across languages, eliminating the reliance on stronger LLMs. 3 Methodology We adapt CulFiT Feng et al. 2025 by omitting the ineffective Direct Preference Optimization (DPO) phase and introducing self-supervised ground truth generation. The pipeline has two stages: (1) Bilingual Question Generation for synthesizing English and local-language query pairs, and (2) Self-Supervised Ground Truth Generation to distill reliable cultural knowledge from a base model \mathcal{M} via multilingual self-consistency. Figure 1: Overview of the self-supervised ground truth generation via multilingual self-consistency. The model generates N responses per language; the language with higher intra-language consistency is selected as the stronger language, and the most consistent answer in the stronger language is translated to the weaker language and set as the ground truth. 3.1 Bilingual Question Generation We convert assertive statements s_{i} from the CANDLE Nguyen et al. 2023 and CultureAtlas Fung et al. 2024 datasets into coherent