Culture is Not Trivia: Sociocultural Theory for Cultural NLP

Naitian Zhou, David Bamman, Isaac L. Bleaman · 2025

Read Culture is Not Trivia: Sociocultural Theory for Cultural NLP by Naitian Zhou, David Bamman, Isaac L. Bleaman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12057.txt.

Introduction

Naitian Zhou, David Bamman, and Isaac Bleaman write a 2025 position paper on culture in language technology. Tools that work with human language now serve users from many cultures, yet the field lacks a shared idea of what culture is. Studies often lean on rough stand-ins, such as nation labels, that miss much of real life.

Drawing on theory from sociocultural linguistics, the authors show how the weak stand-ins limit data and tests. A case study makes the limits concrete. They then sketch paths toward stronger cultural skill in models and data. One key claim is that localization, fitting tools to a real community, is a better goal than broad cultural labels.

Catalog record

CREATORNaitian Zhou, David Bamman, Isaac L. Bleaman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12057

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Culture is Not Trivia: Sociocultural Theory for Cultural NLP Naitian Zhou, David Bamman, Isaac L. Bleaman Abstract The field of cultural NLP has recently experienced rapid growth, driven by a pressing need to ensure that language technologies are effective and safe across a pluralistic user base. This work has largely progressed without a shared conception of culture, instead choosing to rely on a wide array of cultural proxies. However, this leads to a number of recurring limitations: coarse national boundaries fail to capture nuanced differences that lay within them, limited coverage restricts datasets to only a subset of usually highly-represented cultures, and a lack of dynamicity results in static cultural benchmarks that do not change as culture evolves. In this position paper, we argue that these methodological limitations are symptomatic of a theoretical gap. We draw on a well-developed theory of culture from sociocultural linguistics to fill this gap by 1) demonstrating in a case study how it can clarify methodological constraints and affordances, 2) offering theoretically-motivated paths forward to achieving cultural competence, and 3) arguing that localization is a more useful framing for the goals of much current work in cultural NLP. 1 Introduction Language and culture are closely linked: language can be conceptualized simultaneously as an artifact of culture as well as a process through which culture is created Ochs (2009) . As language technologies become increasingly integrated into the everyday lives of a diverse set of users, it is imperative that they are robust to cultural differences between user bases Hershcovich et al. (2022) . Cultural NLP, sometimes also known as cultural alignment, is a subfield within the NLP and ML communities that has experienced drastic growth in recent years to meet this challenge. Work in cultural NLP usually involves building or evaluating systems that 1) have knowledge of cultural facts and 2) apply this knowledge appropriately in specific situations where cultural knowledge is relevant Adilazuarda et al. (2024) ; Liu et al. (2024b) . This can include building new evaluation benchmarks or fine-tuning datasets that contain cultural knowledge of some kind, either manually Lee et al. (2024) ; Koto et al. (2024) or automatically from a large corpus Shi et al. (2024) ; Wang et al. (2024a) , or creating systems that generate culturally-relevant output Khanuja et al. (2024) . Most work relies on various proxies for defining both cultural boundaries and cultural objects. Proxies of cultural boundaries commonly include nationality, religion, ethnicity, or other demographic features. Proxies for cultural objects might include culture-specific knowlege of foods, values, or norms Zhou et al. (2024a) ; Sorensen et al. (2024) ; Dwivedi et al. (2023) . These works constitute an important step forward in understanding how to build fairer, more inclusive language technologies. However, the disparate array of cultural proxies being evaluated is symptomatic of a theoretical gap: to achieve culturally-competent NLP systems, we must make progress towards a clearer, more unified conception of culture, and what it means for the systems we build to be responsive to that. Fortunately, cultural NLP is not alone in the search for a useful notion of culture, and its theoretical challenges are not new. Dissatisfaction with the coarseness of demographic cultural boundaries led to the second wave of sociolinguistics, which refocused efforts on identifying local cultural meaning within communities of practice Eckert (2012) . Larger questions, like the utility of the culture concept, have been debated in fields like sociocultural anthropology, where some researchers have abandoned culture altogether as being essentializing and othering Vann (2013) . Indeed, epistemological and empirical tensions as they relate to the study of culture have been grappled with across such fields as anthropology, sociolinguistics, sociology, cultural studies, among many others. Contributions. In this paper, we draw on theoretical developments in sociocultural linguistics Bucholtz and Hall (2005) — itself a collection of several adjacent disciplines — to clarify the status of cultural knowledge in building culturally competent NLP systems. We first review the goals of cultural NLP, and enumerate specific desiderata for culturally-aware language technologies that current works pursue (§ 2 ). Then, we highlight recurring difficulties in cultural NLP (§ 3 ) by providing a survey of common self-stated limitations in existing papers. We introduce sociocultural linguistics as a field with a useful theoretical framework which we can apply to better understand culture as an object of study (§ 4 ), and provide a case study to illustrate how the theory of indexicality can be applied to clarify the distinction between learning cultural knowledge and learning stereotypes (§ 5 ). Finally, we discuss the broader implications granted by this understanding of culture, offering two main claims. First, we highlight existing methodological and theoretical gaps in achieving the ambitious goal of cultural competence, and provide theoretically-motivated suggestions for making progress on the task (§ 6.1 ). Then, we argue that the goal in cultural NLP might be reasonably understood as localization instead of cultural competence or understanding, providing a more tractable and situated framing with which to build useful NLP systems (§ 6.2 ). 2 The goals of cultural NLP Adaptive Discerning Inclusive Nuanced Figure 1: Two separate spaces of desiderata; the left represents aspects of competence , while the right represents aspects coverage . In each space, there often exists a trade-off between the axes, so that most cultural NLP work falls into the conceptual area that is shaded. Though the field of cultural NLP does not necessarily agree on a definition of culture, there is general agreement on the goal: to build culturally- competent NLP systems Bhatt and Diaz (2024) . Here, we break down this high-level goal into several more specific desiderata that are frequently mentioned in cultural NLP papers. We want our language technologies to be: Adaptive. A foundational premise of cultural NLP is that language technologies should be culturally sensitive . In other words, culturally competent language technologies should be responsive to specific cultural contexts when designing their outputs. It would be insufficient for an NLP system to produce the same output for all cultural contexts; many works on bias in NLP have shown and problematized the tendency of language technologies to represent, exaggerate, and perpetuate a hegemonic set of values and structures Voigt et al. (2018) ; Sheng et al. (2019) ; Bender et al. (2021) . A wide body of work focuses on assessing whether NLP systems can generate different, appropriate outputs in response to different cultural contexts — by answering value-oriented survey questions in a manner consistent with the target culture, for example ( Cao et al., 2024a ; Huang and Yang, 2023 , ) . Some works that center adaptation as a value focus on extrinsic evaluation: instead of probing whether language models know specific cultural facts, they test whether NLP systems respond in a way that demonstrates this knowledge Bhatt and Diaz (2024) . Discerning. At the same time, there is also a desire that NLP systems not perpetuate reductive stereotypes. Past work has demonstrated that users have differing expectations for cultural adaptation Lucy et al. (2024) , and that cultural adaptation is not equally desired in all settings. For example, users may want technologies to understand their regional or ethnic dialects, but not generate them

Read the complete text