Language Models as Artificial Learners: Investigating Crosslinguistic Influence

Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis · 2026

Read Language Models as Artificial Learners: Investigating Crosslinguistic Influence by Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21587.txt.

Introduction

Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, and Gerasimos Spanakis use language models as stand-in learners to study how one language shapes another. Crosslinguistic influence is central to research on bilingual minds. Human studies, though, often clash because no two groups of learners match.

Models offer a controlled learner that can be rerun at will. The authors simulate the influence in a systematic way and isolate its drivers. With noise and background held fixed, the true causes stand out. The approach gives language science a lab bench where classic disputes can be rerun cleanly.

Catalog record

CREATORAbderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21587

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Language Models as Artificial Learners: Investigating Crosslinguistic Influence Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis Abstract Despite the centrality of crosslinguistic influence (CLI) to bilingualism research, human studies often yield conflicting results due to inherent experimental variance. We address these inconsistencies by using language models (LMs) as controlled statistical learners to systematically simulate CLI and isolate its underlying drivers. Specifically, we study the effect of varying the L1 language dominance and the L2 language proficiency, which we manipulate by controlling the L2 age of exposure—defined as the training step at which the L2 is introduced. Furthermore, we investigate the impact of pretraining on L1 languages with varying syntactic distance from the L2. Using crosslinguistic priming, we analyze how activating L1 structures impacts L2 processing. Our results align with evidence from psycholinguistic studies, confirming that language dominance and proficiency are strong predictors of CLI. We further find that while priming of grammatical structures is bidirectional, the priming of ungrammatical structures is sensitive to language dominance. Finally, we provide mechanistic evidence of CLI in LMs, demonstrating that the L1 is co-activated during L2 processing and directly influences the neural circuitry recruited for the L2. More broadly, our work demonstrates that LMs can serve as a computational framework to inform theories of human CLI. 1 Introduction The study of crosslinguistic influence (CLI) is central to understanding how two languages coexist and interact within bilingual speakers 33 ; 34 . Despite its prominence, the underlying mechanisms of CLI remain debated. Evidence from crosslinguistic priming—in which processing a structure in one language facilitates the production of the same structure in another—points to the existence of shared syntactic representations between languages 17 ; 16 ; 12 . However, while priming is a well-attested phenomenon, the number of variables that can affect human language acquisition leaves many questions unanswered. Previous research have explored various predictors of CLI; yet, findings remain inconclusive due to significant variance in how these predictors are operationalized across studies 37 . Figure 1 : Setup for modeling CLI using LMs. We manipulate the age of exposure by introducing the L2 at specific training steps after pretraining on the L1. The second stage of training uses interleaved L1/L2 sequences to maintain L1 influence. We select 5 L1 languages with increasing syntactic distance from the L2. Language models (LMs) can simulate human-like grammatical judgments from unstructured data 40 ; 44 . Despite significant differences between LMs and human learning, they can still contribute to the debate on human language acquisition, especially when their learning environment is deliberately made plausible 39 . Their greatest potential lies in the ability to ablate the variables that affect CLI. For instance, we can vary the L1 and L2 language exposure ratios and study the effect on crosslinguistic priming. This degree of control is rarely possible in psycholinguistic studies, which often rely on non-standardized linguistic tests of language dominance, leading to contradictory conclusions 37 . Furthermore, while human studies are often limited in their scalability, artificial learners facilitate the broad evaluation of linguistic phenomena, and the extension to languages with few non-native speakers, offering a comprehensive quantitative account of CLI. While LMs cannot simulate the full human learning experience, we focus on studying the effects of the L1 language dominance and L2 language proficiency, which are straightforward to manipulate in LMs, and were shown to be predictive of CLI and crosslinguistic priming effects 37 ; 16 . As shown in Figure 1 , to manipulate dominance and proficiency, we vary the age of exposure, which is the training step at which we introduce the L2 language. For instance, introducing the L2 language at step=32K means the model was trained exclusively on the L1 language for 32K steps, leading it to be more entrenched in the model’s parameters, therefore increasing its dominance. To maintain the L1 influence and prevent catastrophic forgetting, we train on interleaved L1/L2 sequences to maintain the L1 language influence; consequently, late L2 introduction also results in lower L2 exposure, directly affecting its proficiency. Our hypothesis is that a distant L1 language will lead to more negative or less positive CLI effects compared to a less distant one. As shown in Figure 1 , we study the CLI effects of 5 languages that are increasingly distant from the L2 language. We measure CLI effects using the Benchmark of Linguistic Minimal Pairs for English (BLiMP) 40 —a dataset of 67k pairs of grammatical and ungrammatical sentences. More specifically, we measure the difference in BLiMP accuracy between the bilingual LM, and a monolingual counterpart. While studies of crosslinguistic priming have found that it affects both grammatical 6 ; 16 and ungrammatical 36 ; 5 structures, we similarly prime the model with the L1 translation of the grammatical target sentence, and we study its effect on BLiMP accuracy. Beyond grammatical accuracy, we investigate whether bilingual LMs exhibit preference for the writing of non-native speakers that share the same L1. Using the First Certificate in English (FCE) dataset 43 —which contains articles written by different L1 non-native speakers along with their corrections—we study whether LMs internalize L1 specific CLI effects. This analysis provides evidence that the nuances of human CLI can be modeled and differentiated computationally. Finally, we provide mechanistic evidence of L1 influence on L2 processing and learning. We use LogitLens 28 to decode LMs hidden states and quantify the L1 language presence during L2 processing. Furthermore, we investigate the effect of the L1 language on learning the L2 by correlating the L2 language neuron overlap between the L1 languages with their syntactic distance. In summary, this work provides a comprehensive investigation of the computational modeling of CLI effects. Our contributions are as follows: • We propose a methodology to manipulate the L1 language dominance and the L2 language proficiency by varying the age of exposure and training on interleaved L1/L2 sequences. • We perform a quantitative study and ablation of CLI and crosslinguistic priming on the BLiMP benchmark. • Using the FCE dataset, we investigate whether LMs internalize L1-specific nuances based on their preference of non-native text from the same L1. • We provide mechanistic evidence of L1 influence using LogitLens and language neuron analysis. 2 Related Works Crosslinguistic Influence The study of how two languages interact is a prominent research focus of both human bilinguals 20 ; 32 ; 37 ; 34 and artificial learners 30 ; 29 ; 42 ; 3 . While studies of language models (LMs) consistently show that syntactic distance negatively impacts target language performance, it remains unclear whether this effect stems from shared grammatical representations or alternative factors, such as interference resulting from limited model capacity 9 . In human CLI research, there is a stronger consensus that representations are either fully shared or separate-but-connected 16 ; 2 . Most of the evidence for this comes from studies of crosslinguistic priming 7 . We similarly employ crosslinguistic priming to study CLI in artificial models, discussing the implications for both artificial and human bilinguals. Crosslinguistic priming Crosslinguistic priming, first formulated by 7 , is a promising methodology for validating hypotheses regarding CLI 33 . For instance, 17 ; 16 proposed the the shared-syntax hypothesis, using crosslinguistic priming as evidence. Specifically, the hypothesis is

Read the complete text