Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring
Mina Almasi, Ross Deans Kristensen-McLachlan · 2025
Read Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring by Mina Almasi, Ross Deans Kristensen-McLachlan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08351.txt.
Introduction
Mina Almasi and Ross Deans Kristensen-McLachlan test whether large language models can hold a set reading level while tutoring Spanish, in a two thousand twenty-five paper. The levels come from the CEFR scale, from beginner A1 to advanced C1.
The authors simulated full teacher-student dialogues in Spanish with open-source models of seven to twelve billion parameters. One model played tutor and another played student, each with separate chat histories. Prompting did steer difficulty at first, but control decayed over long conversations, a failure they name alignment drift. The study offers a cheap, human-free way to evaluate tutoring models.
Catalog record
| CREATOR | Mina Almasi, Ross Deans Kristensen-McLachlan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08351
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring Mina Almasi, Ross Deans Kristensen-McLachlan Abstract This paper investigates the potentials of Large Language Models (LLMs) as adaptive tutors in the context of second-language learning. In particular, we evaluate whether system prompting can reliably constrain LLMs to generate only text appropriate to the student’s competence level. We simulate full teacher-student dialogues in Spanish using instruction-tuned, open-source LLMs ranging in size from 7B to 12B parameters. Dialogues are generated by having an LLM alternate between tutor and student roles with separate chat histories. The output from the tutor model is then used to evaluate the effectiveness of CEFR-based prompting to control text difficulty across three proficiency levels (A1, B1, C1). Our findings suggest that while system prompting can be used to constrain model outputs, prompting alone is too brittle for sustained, long-term interactional contexts - a phenomenon we term alignment drift . Our results provide insights into the feasibility of LLMs for personalized, proficiency-aligned adaptive tutors and provide a scalable method for low-cost evaluation of model performance without human participants. 1 Introduction The popularization of large language models (LLMs), particularly through the emergence of user-friendly interfaces such as ChatGPT, has led many stakeholders across society to consider how to use such technology effectively and safely to facilitate access to knowledge and education ( Yan et al., 2024 ) . Language education has not been immune to this hype, and with seemingly good cause, since LLMs show potential across a range of areas where they might enhance language learning. One such area is their inherent interactivity . Interactive feedback is widely regarded as an important factor in second-language (L2) learning ( Loewen and Sato, 2018 ) . For L2 learners far removed from their target language community, opportunities for such interaction can be rare. With LLMs, though, learners appear to now have the opportunity to engage with a "speaker" of the target language freely and at their own pace ( Kohnke et al., 2023 ) . Other potential benefits include personalized teaching ( Klimova et al., 2024 ) and reduced L2 anxiety ( Hayashi and Sato, 2024 ) . These ideas build on decades of research on intelligent tutoring systems and computer-assisted learning ( Psotka et al., 1992 ; Slavuj et al., 2015 ) . In contrast to earlier rule-based approaches ( D’Mello and Graesser, 2023 ) , appropriately implemented LLMs may offer a more adaptable and effective solution. However, current use of LLMs in language learning mostly relies on general-purpose tools like ChatGPT, where learners are encouraged to acquire "prompt-engineering" skills to get the most out of their AI language tutor ( Hwang et al., 2024 ) . It remains unclear exactly how effective and appropriate this approach is for creating successful language tutoring technology. This paper takes steps to address this problem by examining whether, and to what extent, the complexity of LLM outputs can be constrained through prompting based on the Common European Framework of Reference for Languages (CEFR). We find that, while prompting may initially constrain LLM outputs in Spanish, these effects diminish over time. We refer to this as alignment drift , arguing that system prompting may prove to be too unstable for sustained, longer interactions. 2 Related Work 2.1 Exploring the Use of LLMs as Language Tutors While a growing body of work considers LLMs as interactive language tutors ( Kohnke et al., 2023 ; Lin, 2024 ; Kostka and Toncelli, 2023 ) , empirical research is limited, and many questions remain unanswered ( Han, 2024 ) . Nevertheless, the few studies that have been conducted so far offer promising results on the benefits of using LLMs as language tutors, particularly in L2 English learning ( Tyen et al., 2022 ; Tyen et al., 2024 ; Zhang and Huang, 2024 ) . Among other findings, Tyen et al. (2024) reported that users enjoyed interacting with LLMs more than plain reading and responded well to adaptive difficulty in interactions. Adaptive cognitive tutors hence have the potential to contribute positively to motivation , a psychological process increasingly viewed as crucial to L2 learning outcomes ( Dornyei and Ryan, 2015 ) . 2.2 Assessing L2 Proficiency with CEFR Defining what it means to be "proficient" in an additional language is not a trivial task, with numerous definitions proposed ( Park et al., 2022 ) . Of these, the CEFR is particularly well known. Since its introduction in 2001, the framework has been highly influential in assessing L2 proficiency. Unlike previous approaches with a strong focus on grammatical competency, the CEFR emphasizes social and communicative competences ( Leclercq and Edmonds, 2014 ) . The CEFR comprises a six-level scale (A1, A2, B1, B2, C1, C2) with A1 as the beginner level and C2 as the most advanced. Several official ways have been developed to represent these proficiency levels, each with language-agnostic descriptions ( Council of Europe, 2025a ) . For instance, the CEFR Global scale offers a concise, three- to four-sentence summary of each level, designed as a holistic overview to facilitate communication with non-specialist users. However, its creators acknowledge that it is "desirable" to present the CEFR levels in "different ways for different purposes." ( Council of Europe, 2025b ) . The Self-assessment grid , which provides separate definitions for skills like speaking and writing at each level, has little to no focus on grammatical content ( Council of Europe, 2025d ) . 2.3 Adapting Text Difficulty with LLMs The potential for LLMs to produce simpler text for improved accessibility has not gone unnoticed ( Freyer et al., 2024 ) . Indeed, the CEFR framework has been used alongside LLMs to simplify learning materials in French ( Jamet et al., 2024 ) ; and for a range of purposes in English, such as general writing ( Uchida, 2025 ) and simplifying or writing stories ( Malik et al., 2024 ; Imperial and Tayyar Madabushi, 2023 ) . Alfter (2024) also attempted to generate CEFR-aligned vocabulary lists using LLMs across five languages, including Spanish and French, but found performance issues outside of English. Common to these studies is the use of prompting. Notably, Malik et al. (2024) demonstrated that GPT-4 made fewer errors generating stories at the desired proficiency level as the detail about CEFR increased in the prompts. In contrast, Alfter (2024) found that using numeric levels from 0 to 4 was more effective than explicitly mentioning the CEFR, although the prompts had no description of the levels. Beyond prompting, other approaches include fine-tuning ( Malik et al., 2024 ) or experimentation with decoding strategies. For example, Tyen et al. (2022) experimented with different decoding strategies for constraining LLM text difficulty to CEFR levels, using a classifier fine-tuned on Cambridge English exam sentences ( Xia et al., 2016 ) , to select the best LLM-generated sentence for the user. A similar approach was used by Glandorf and Meurers (2024) , focusing on grammatical constructs for different CEFR levels in English. We identify some gaps in the literature. Firstly, most studies focus on English, with only a few exceptions ( Jamet et al., 2024 ; Alfter, 2024 ) . Moreover, aside from Tyen et al. (2022) ; Tyen et al. (2024)