Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs
Ariba Khan ∗, Stephen Casper ∗, Dylan Hadfield-Menell · 2025
Read Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs by Ariba Khan ∗, Stephen Casper ∗, Dylan Hadfield-Menell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08688.txt.
Introduction
Ariba Khan, Stephen Casper, and Dylan Hadfield-Menell test how culture alignment in language models is judged in two thousand twenty-five. Survey based tests borrow tools from social science to score such fit. The team checks three claims that sit under those tests. Claims cover stability, spread to new issues, and steer by prompt.
Their tests find high sway from mere format shifts. Scores on one set of issues fail to predict held out issues. Prompt steering acts in erratic ways as well. A case study shows how narrow tests can paint a false bright picture. They urge firm robustness checks and red teaming for such claims.
Catalog record
| CREATOR | Ariba Khan ∗, Stephen Casper ∗, Dylan Hadfield-Menell |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08688
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs Ariba Khan ∗, Stephen Casper ∗, Dylan Hadfield-Menell Abstract . Research on the ‘cultural alignment’ of Large Language Models (LLMs) has emerged in response to growing interest in understanding representation across diverse stakeholders. Current approaches to evaluating cultural alignment through survey-based assessments that borrow from social science methodologies often overlook systematic robustness checks. Here, we identify and test three assumptions behind current survey-based evaluation methods: (1) Stability : that cultural alignment is a property of LLMs rather than an artifact of evaluation design, (2) Extrapolability : that alignment with one culture on a narrow set of issues predicts alignment with that culture on others, and (3) Steerability : that LLMs can be reliably prompted to represent specific cultural perspectives. Through experiments examining both explicit and implicit preferences of leading LLMs, we find a high level of instability across presentation formats, incoherence between evaluated versus held-out cultural dimensions, and erratic behavior under prompt steering. We show that these inconsistencies can cause the results of an evaluation to be very sensitive to minor variations in methodology. Finally, we demonstrate in a case study on evaluation design that narrow experiments and a selective assessment of evidence can be used to paint an incomplete picture of LLMs’ cultural alignment properties. Overall, these results highlight significant limitations of current survey-based approaches to evaluating the cultural alignment of LLMs and highlight a need for systematic robustness checks and red-teaming for evaluation results. Data and code are available at akhan02/cultural-dimension-cover-letters and ariba-k/llm-cultural-alignment-evaluation , respectively. CCS: Social and professional topics Cultural characteristics CCS: General and reference Evaluation Figure 1 . Core assumptions about LLM cultural alignment fail under systematic evaluation. Our experiments reveal that cultural alignment in LLMs is: (1) not stable ( Section 5.1 , Figure 3 , Figure 4 ) - response variations from trivial format changes often exceed real-world cultural differences; (2) not extrapolable ( Section 5.2 , Figure 5 ) - extrapolation from limited dimensions produces near-random clustering results, with strong sensitivity to which dimensions are included; and (3) not steerable ( Section 5.3 , Figure 6 ) - even optimized prompting techniques produce erratic, un-humanlike response patterns that fail to align with cultural perspectives. 1. Introduction Despite advancements in aligning AI with human preferences through methods like RLHF ( Kovač et al., 2023 ; Casper et al., 2023 ) , evaluating and controlling AI systems’ alignment with different cultures remains challenging ( Prabhakaran et al., 2022 ; Ożegalska-Łukasik and Łukasik, 2023 ) . Most existing research evaluating cultural alignment in LLMs uses survey-based assessments that merely analyze how models respond when asked about their values and preferences ( Pawar et al., 2024 ) . For example, the CDEval benchmark evaluates cultural dimensions in LLMs using questions designed to assess six cultural dimensions ( Wang et al., 2024b ) . However, modern LLMs – and their expressed ‘preferences’ – are complex ( Liu et al., 2023 ) . This leads us to ask whether current survey-based evaluation methods can adequately characterize the cultural alignment of LLMs. Current methods for assessing cultural alignment borrow frameworks from social science methodologies ( Lindgren and Holmström, 2020 ) but typically reduce to asking LLMs questions about their ‘preferences’ without necessarily implementing systematic validation practices. This oversight is potentially concerning because LLMs, unlike humans whose values often show consistent patterns, are known to express inconsistent ideas across different contexts ( Sclar et al., 2023 ; Zheng et al., 2023 ; Errica et al., 2024 ; Zhuo et al., 2024 ; Anagnostidis and Bulian, 2024 ) . Here, we investigate the reliability of current survey-based cultural alignment evaluations by identifying and testing three key assumptions from prior work (see Figure 1 ): • Stability: Cultural alignment manifests as a property of LLMs rather than an artifact of evaluation design. • Extrapolability: Alignment with one culture on a narrow set of issues predicts alignment with it on others. • Steerability: LLMs can consistently be made to embody specific cultural perspectives through prompting. To test these assumptions, we use both explicit evaluation methods, using established cultural assessments ( Hofstede and Minkov, 2013 ; Durmus et al., 2023 ) , and implicit evaluations that examine LLM behavior through a simulated hiring scenario. We make three contributions to understanding and evaluating cultural alignment in LLMs: (1) We identify and formally characterize three critical assumptions in survey-based cultural alignment evaluation literature: stability , extrapolability , and steerability . (2) We provide comprehensive empirical evidence challenging these assumptions through experiments with both explicit surveys and implicit preference elicitation, quantifying significant inconsistencies across different cultural dimensions. (3) We demonstrate how subtle variations in evaluation methodology can greatly shift conclusions about LLMs’ cultural alignment, presenting a case study that demonstrates how high-level findings from Mazeika et al. (2025) do not replicate when an LLM is evaluated with the option of selecting indifference between alternatives. Our findings suggest that current, popular survey-based methods for evaluating cultural alignment in LLMs require critical re-examination, as they risk oversimplifying or misrepresenting results. The high sensitivity of LLM responses to subtle methodological choices indicates that narrow experiments or a selective assessment of evidence may paint an incomplete picture of these systems’ cultural alignment properties. We release data and code at akhan02/cultural-dimension-cover-letters and ariba-k/llm-cultural-alignment-evaluation , respectively. 2. Related Work 2.1. Cultural Alignment in Language Models An LLM’s ‘alignment’ with a culture describes the degree to which its behaviors reflect common beliefs within that culture regarding what is considered desirable and proper ( Pawar et al., 2024 ) . As language models have become increasingly capable of generating human-like text, researchers have analyzed how the behaviors of these models reflect or diverge from different cultural perspectives ( Cao et al., 2023 ) . Early work in cultural alignment has highlighted that LLMs tend to reflect values aligned with WEIRD (Western, Educated, Industrialized, Rich, and Democratic) societies ( Johnson et al., 2022 ) , leading to the perpetuation of cultural biases and the misrepresentation of non-WEIRD worldviews. 2.2. Evaluating Cultural Alignment Two main methodological approaches have emerged for assessing cultural alignment. The more common paradigm has been discriminative assessment , which adapts traditional survey techniques by requiring models to select from predetermined options to evaluate their preferences and biases ( AlKhamissi et al., 2024 ; Arora et al., 2023 ; Cao et al., 2023 ; Masoud et al., 2023 ; Moore et al., 2024 ) . For example, recent work by Mazeika et al. (2025) studies LLM ‘preferences’ by having them select a binary preference between two outcomes (e.g., “saving 10 lives in the United States” versus “saving 10 lives in Nigeria”). In contrast, generative approaches analyze free-form model outputs, similar to qualitative methods in social science research ( Adilazuarda et al., 2024 ; Tao et al., 2024 ) . These can be implemented through both single-turn assessment, where cultural context and probe are given in one prompt, and multi-turn assessment, which evaluates responses over several interactions ( Adilazuarda et al., 2024 ) . 2.3. Survey-Based Assessments Building on discriminative assessment, researchers have developed benchmarks to evaluate model responses across cultural contexts, from politics to religion ( Durmus et al., 2023 ) . These include