A Computational Approach to Language Contact – A Case Study of Persian

Ali Basirat, Danial Namazifard, Navid Baradaran Hemmati · 2026

Read A Computational Approach to Language Contact – A Case Study of Persian by Ali Basirat, Danial Namazifard, Navid Baradaran Hemmati (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20592.txt.

Introduction

Ali Basirat, Danial Namazifard, and Navid Baradaran Hemmati probe language contact inside the internal layers of a monolingual language model. They ask whether structural traces of contact with other languages survive in the representations a model learns. Their case study is Persian, a language with a rich contact history.

They expose a Persian-trained model to languages with varying degrees and types of contact with Persian, then inspect its intermediate representations. If contact leaves a measurable trace, model internals could become a new source of evidence for historical linguistics, complementing texts and speaker data.

Catalog record

CREATORAli Basirat, Danial Namazifard, Navid Baradaran Hemmati
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20592

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Computational Approach to Language Contact – A Case Study of Persian Ali Basirat, Danial Namazifard, Navid Baradaran Hemmati Abstract We investigate structural traces of language contact in the intermediate representations of a monolingual language model. Focusing on Persian (Farsi) as a historically contact-rich language, we probe the representations of a Persian-trained model when exposed to languages with varying degrees and types of contact with Persian. Our methodology quantifies the amount of linguistic information encoded in intermediate representations and assesses how this information is distributed across model components for different morphosyntactic features. The results show that universal syntactic information is largely insensitive to historical contact, whereas morphological features such as Case and Gender are strongly shaped by language-specific structure, suggesting that contact effects in monolingual language models are selective and structurally constrained. 1 Introduction Language contact is a primary driver of language change and structural diversification 25 . When speakers of different languages come into sustained interaction through, for example, trade, migration, conquest, or cultural exchange, their languages influence one another in ways that can range from lexical borrowing to deep grammatical restructuring ( 23 ; 16 ) . This can appear as matter borrowing , where the morphological material and its phonological shape are replicated, and/or as pattern borrowing , where the organization, distribution, and mapping of grammatical or semantic meaning patterns are replicated ( 24 ) . The study of language contact requires a holistic perspective on linguistic structure that spans multiple levels of representation 16 , a perspective that is difficult to obtain solely through traditional analytical methods. Large language models (LLMs) have opened new avenues for investigating linguistic phenomena by enabling the study of their intermediate representations. Although LLMs are not designed to model diachrony, their internal representations often reflect patterns of cross-linguistic similarity and genealogical relatedness that emerge implicitly from large-scale training data 26 ; 6 . This study investigates how language change might have been reflected in the intermediate representation of a monolingual Persian language model. The case of Persian (Farsi) provides a particularly compelling testbed. Although genealogically Indo-European, Persian has undergone extensive contact with languages from multiple families, most notably Turkic languages, as well as Arabic, English, French, Russian, and Hindi ( 27 ) . These contact dynamics have influenced Persian at several linguistic levels. However, it remains unclear whether such changes are detectable, quantifiable, and linguistically interpretable within the internal representations of a monolingual language model, such as ParsBERT ( 9 ) . We study language change at the morpho-syntactic level based on linguistic annotations from Parallel UD treebanks ( 29 ) for a group of languages at various degrees and types of language contact. We adopt an information-theoretic approach to measure the amount of usable information ( 28 ) that intermediate representations of ParsBERT encode about the linguistic features of the target languages. We further perform an attribution analysis ( 21 ) to identify those components of the representations that contribute most strongly to this encoding. Our findings contribute to a growing body of research on typology-aware NLP ( 4 ; 18 ) , offering evidence that monolingual language models encode selective and structurally constrained traces of language contact. Specifically, we show that contact effects are more readily reflected in morphological and constructional patterns aligned with the training language, while universal syntactic categories remain largely robust to contact-induced variation. By combining information-theoretic probing with attribution analysis, this work provides a principled computational framework for investigating classical questions in contact linguistics and sheds light on the limits and possibilities of using neural language models as tools for studying language change. 2 Related Work Language contact has been extensively studied in functional, historical, and typological linguistics, with a particular focus on how sustained interaction between speech communities leads to lexical, morphological, and syntactic change ( 23 ; 16 ) . Persian constitutes a well-documented case of long-term language contact: centuries of interaction with Semitic, Turkic, Indic, and European languages have left pervasive traces across multiple linguistic levels ( 27 ; 13 ) . While these contact-induced effects have been described in detail from diachronic and descriptive perspectives, they have not yet been examined using computational models of language. In particular, it remains unclear how language contact phenomena are reflected in the internal representations of neural language models trained on Persian. To the best of our knowledge, this work presents the first computational study of Persian language contact at the level of neural representations. Our work is also related to the growing literature on probing neural language models for linguistic structure. Prior research has investigated how morphology ( 20 ) , syntactic structure ( 22 ; 12 ) , and cross-linguistic typological properties ( 18 ) are encoded in contextualized representations. Grammatical gender, in particular, has been widely used as a test case for cross-linguistic generalization in neural models ( 26 ; 19 ) . We extend this line of work by probing the intermediate representations of a monolingual Persian language model, with a focus on how linguistic features associated with contact languages are encoded. By doing so, we connect descriptive insights from language contact research with contemporary computational approaches to representation learning. 3 Methodology Our goal is to determine whether a monolingual Persian model encodes latent structural traces that reflect Persian’s long-standing contact with other languages. To this end, we examine whether representations learned exclusively from Persian data exhibit systematic affinities to linguistic characteristics of contact languages. We pass a set of sentences sampled from a contact language into a Persian language model. For each token, we extract the intermediate representations produced by each of the embedding and Transformer layers of the model. We, then, adopt two techniques to analyze these intermediate representations (embeddings): (i) an information-theoretic probe that quantifies how much linguistic information is encoded in the representations, and (ii) a complementary attribution analysis that identifies which components of the representations contribute most strongly to this encoding. The probing framework is based on variational usable information I_{\mathcal{V}}(X\rightarrow Y) ( 28 ) , which estimates the amount of information that a random variable X contains about a target random variable Y . In our study, X corresponds to embedding vectors extracted from a given intermediate layer of the language model when processing a token in context, and Y represents a linguistic property of the token (e.g., language identity, UPOS tag, Case , or Gender ). Following 3 , we normalize I_{\mathcal{V}}(X\rightarrow Y) by marginal entropy H(Y) of the target feature. This normalization yields a dimensionless metric \hat{I}_{\mathcal{V}}\in[0,1] that reflects the proportion of the total uncertainty in Y that can be recovered from the representation. High usable-information for a linguistic property in a test language indicate that ParsBERT encodes systematic cues relevant to the realization of that property, suggesting structural alignment between the test language and Persian for the corresponding feature. Conversely, low usable-information scores reflect limited overlap between the feature’s realization in the test language and Persian. The attribution analysis quantifies how linguistic information about a target variable Y is distributed across elements of X using Language Activation Probability Entropy (LAPE) ( 21 ) . LAPE assigns each element of X a score based on the entropy of its activation distribution across different

Read the complete text