Versteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language

Verena Platzgummer John McCrae Sina Ahmadi University of Galway University of Zurich {verena.platzgummer,john.mccrae}@universityofgalway.ie [email protected] · 2026

Read Versteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language by Verena Platzgummer John McCrae Sina Ahmadi University of Galway University of Zurich {verena.platzgummer,john.mccrae}@universityofgalway.ie [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28213.txt.

Introduction

Platzgummer, McCrae, and Ahmadi examine how generative AI treats non-standard language, from South Tyrolean dialects to Kurdish varieties. Language models favor standard forms, a bias rooted in older standardization drives. The authors pair sociolinguistic critique with a technical look at fixes. They ask what it would take for models to handle dialects well, and whether doing so can serve fairer, decolonial language policy. The paper blends both lenses to frame the stakes.

Catalog record

CREATORVerena Platzgummer John McCrae Sina Ahmadi University of Galway University of Zurich {verena.platzgummer,john.mccrae}@universityofgalway.ie [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49871
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Versteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language Verena Platzgummer John McCrae Sina Ahmadi University of Galway University of Zurich {verena.platzgummer,john.mccrae}@universityofgalway.ie [email protected] Abstract The design of Large Language Models (LLMs) and generative artificial intelligence (GenAI) has been shown to be “unfair” to less-spoken languages ( 61 ) and to deepen the digital language divide ( 13 ) . Critical sociolinguistic work has also argued that these technologies are not only made possible by prior socio-historical processes of linguistic standardisation, often grounded in European nationalist and colonial projects ( 56 ) , but also exacerbate epistemologies of language as “monolithic, monolingual, syntactically standardized systems of meaning” ( 69 , p. 5) . In our paper, we draw on earlier work on the intersections of technology and language policy ( 46 ) and bring our respective expertise in critical sociolinguistics and computational linguistics to bear on an interrogation of these arguments. We take two different complexes of non-standard linguistic varieties in our respective repertoires– South Tyrolean dialects, which are widely used in informal communication in South Tyrol, Italy ( 6 ) , as well as varieties of Kurdish –as starting points to an interdisciplinary exploration of the intersections between GenAI and linguistic variation and standardisation. We discuss both how LLMs can be made to deal with non-standard language from a technical perspective, and whether, when or how this can contribute to “democratic and decolonial digital and machine learning strategies” ( 56 , p. 12) , which has direct policy implications. 1 Introduction The rapid advancement of Large Language Models (LLMs) and Generative Artificial Intelligence (GenAI) has transformed digital communication, yet these technologies systematically privilege standardized and, in computational linguistic terms, high-resource languages while marginalizing millions of speakers who communicate primarily through non-standard varieties and dialects. Recent scholarship demonstrates that LLM design is fundamentally “unfair” to less-spoken languages ( 61 ) and deepens the digital language divide ( 13 ) . Critical sociolinguistic work argues that these technologies not only emerge from historical processes of linguistic standardization rooted in colonial and nationalist projects ( 56 ) , but also reinforce epistemologies of language as “monolithic, monolingual, syntactically standardized systems of meaning” ( 69 , p. 5) . When AI systems fail to process non-standard varieties, they exclude speakers from full participation in digital citizenship. Further, technical decisions made in the architecture of models impose a “tokenization tax” that increases costs and degrades performance for languages less represented in the input data to LLMs. This paper brings together critical sociolinguistics and computational linguistics to examine these issues through two complementary case studies: South Tyrolean German and Kurdish varieties. South Tyrolean dialect comprises non-standard forms of German widely used in informal communication in South Tyrol, Italy ( 6 ) , yet remains largely absent from LLM training data despite being the primary medium of everyday interaction. Kurdish represents a dialect continuum spoken by over 40 million people across multiple nation-states, characterized by orthographic diversity, systematic political suppression, and acute digital underrepresentation. While Central Kurdish (Sorani) and Northern Kurdish (Kurmanji) have achieved modest computational resources, varieties such as Southern Kurdish, Hawrami, and Zazaki remain almost entirely invisible to language technology. Through these cases, we analyze the technical barriers preventing LLMs from processing non-standard varieties, examine how evaluation frameworks reproduce standardization biases, and explore policy implications for diverse actors—from Big Tech and states to civil society and academia. Building on earlier work examining technology and language policy intersections ( 46 ) , we argue that addressing LLM marginalization of non-standard varieties requires more than technical fixes; it demands a reorientation toward “democratic and decolonial digital and machine learning strategies” ( 56 ) that center linguistic agency and treat variation as constitutive of human communication rather than noise to eliminate. This has direct policy implications, from requiring “dialect gap” reporting to ensuring community data sovereignty. Ultimately, whether LLMs should “handle” non-standard varieties is not primarily a technical question but a fundamentally political one, with implications for digital sovereignty, cultural preservation, and social justice in an age where the choice between enforcing standardization or embracing variation will shape the vitality of the world’s linguistic diversity in digital spaces. In the following sections, we will first address sociolinguistic (section 2 ) and computational linguistic perspectives on standard and non-standard language and language technologies (section 3 ), before giving an overview of the policy landscape in relation to linguistic variation and LLMs (section 4 ). We will proceed by examining the cases of South Tyrolean dialects (section 5 ) and varieties of Kurdish (section 6 ) and end with overarching conclusions and policy implications. 2 Sociolinguistic Perspectives on (Non-)Standard Language and LLMs The concept of standard language , as well as processes of linguistic standardisation, have been objects of sociolinguistic investigation since the inception of the field itself ( 41 ; 32 ) . The impetus for this type of work largely came from concurrent processes of standardization in newly independent states, often postcolonial realities. As such, it is closely linked to the beginnings of language planning and policy as a subfield of applied linguistics. As characteristics of standard languages, 10 lists their validity across regional borders, their being considered as the ‘best’ language within their realm of validity, and their being codified in norms. As 23 ; 24 notes, it is their institutional enforcement that distinguishes standard languages from other types of linguistic norms, both from “always emergent, variable, and never ‘fixed’ conventions” ( 24 , p. 244) and from language standards that go beyond such conventions in that they become morally imperative, but are not quite standard languages as they are not institutionally enforced. Processes of standardisation usually involve some degree of reduction in linguistic variability ( 23 ; 57 ) . As such, these processes can also be understood as linguistic hierarchisations ( 51 ; 22 ; 34 ) , whereby specific sets of linguistic features, as well as entire varieties conceived as bundles of such features, are imbued with legitimacy for public use, while others are considered acceptable only in private ( 22 ) . As 34 argues, this legitimacy and authority of linguistic forms stem the two ideological complexes of anonymity and authenticity that go back to rationalist and romantic philosophies of the 17th and 18th centuries. It is commonly recognised that “[s]tandardization is […] best approached as an ideological phenomenon” ( 34 , p. 222) . Processes of linguistic standardisation have tended to be part and parcel of processes of nation-building ( 24 ; 22 ; 27 ) and the concomitant creation of an apparently neutral public sphere, as well as of colonialism, and are quintessentially modernist projects ( 22 ) . Standard language and non-standard language – be it referred to as dialect, patois, etc. – are constructed in opposition to one another, whereby only standard language indexes modernist values like progress and development, and is associated with the future ( 34 ) . It is easy to see how, within such standardisation regimes, the majority of the world’s languages that do not have standardised forms ( 66 ) can become constructed as ‘lacking’ and ‘backwards’ ( 34 ) , and even more so the myriad of primarily oral

Read the complete text