Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation

2025

Read Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09984.txt.

Introduction

Bo Li, Zhenghua Xu, and Rui Xie study language drift in multilingual retrieval-augmented generation, where a model answers in the wrong language. Drift appears when retrieved documents and examples come in a different language than the user's question, and it worsens during step-by-step reasoning. Their controlled tests span multiple data sets, languages, and model backbones.

The drift, they find, is not a failure to understand: it is a decoder-level collapse in which frequent English token patterns overwhelm the intended language. English acts as a semantic attractor, both the strongest interference source and the most common fallback. Their remedy, Soft Constrained Decoding, gently penalizes tokens from the wrong language during generation, needing no retraining. It improved language alignment and task performance across three multilingual data sets.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-09984

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation Thanks: Corresponding Author. Abstract Multilingual Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to perform knowledge-intensive tasks in multilingual settings by leveraging retrieved documents as external evidence. However, when the retrieved evidence differs in language from the user query and in-context exemplars, the model often exhibits language drift by generating responses in an unintended language. This phenomenon is especially pronounced during reasoning-intensive decoding, such as Chain-of-Thought (CoT) generation, where intermediate steps introduce further language instability. In this paper, we systematically study output language drift in multilingual RAG across multiple datasets, languages, and LLM backbones. Our controlled experiments reveal that the drift results not from comprehension failure but from decoder-level collapse, where dominant token distributions and high-frequency English patterns dominate the intended generation language. We further observe that English serves as a semantic attractor under cross-lingual conditions, emerging as both the strongest interference source and the most frequent fallback language. To mitigate this, we propose Soft Constrained Decoding (SCD), a lightweight, training-free decoding strategy that gently steers generation toward the target language by penalizing non-target-language tokens. SCD is model-agnostic and can be applied to any generation algorithm without modifying the architecture or requiring additional data. Experiments across three multilingual datasets and multiple typologically diverse languages show that SCD consistently improves language alignment and task performance, providing an effective and generalizable solution in multilingual RAG. Code and Dataset — https://github.com/pkuserc/SCD 1 Introduction Recent advances in Retrieval-Augmented Generation (RAG) have significantly enhanced large language models’ ability to generate factually grounded answers in open-domain question answering ( Xu et al. 2024a ; Luo et al. 2024 ; Fang et al. 2024 ; Shi et al. 2024 ) . However, in multilingual settings, most existing studies have focused on improving cross-lingual retrieval performance and contextual alignment ( Luo et al. 2020 ; Park and Lee 2025 ; Chirkova et al. 2024 ; Wu et al. 2024 ; Ranaldi, Haddow, and Birch 2025 ; Ranaldi et al. 2025 ; Liu et al. 2025 ; Bland’on et al. 2025 ) , while overlooking a critical issue: the mismatch between the input and output languages. Figure 1: Illustration of language drift in multilingual RAG. The user query and in-context examples are provided in the target language (e.g., Chinese), while the retrieved context is written in a non-target language (e.g., English). During reasoning, the model mixes languages and ultimately outputs the final answer in a non-target language. In multilingual RAG settings, the query, instructions, and in-context exemplars are typically written in the target language, aiming to elicit responses in that language. However, due to the predominance of English in open-domain corpora ( Xu et al. 2024b ; Zeng and Yang 2024 ; Resnik and Smith 2003 ) , the retrieved context is often in English, even when the query is in another language. This creates a mixed-lingual input scenario where only the retrieved context differs in language. Nevertheless, models frequently generate responses in the language of the retrieved content rather than in the intended target language ( Park and Lee 2025 ; Liu et al. 2025 ) . We refer to this phenomenon as output language drift , which poses practical challenges for multilingual applications yet remains underexplored. Our empirical study reveals that such cross-lingual conditions negatively affect both task performance and output language consistency. Notably, English serves as the strongest interference source, significantly degrading output quality in non-English settings, while also serving as the most robust target language when subjected to interference. This issue becomes more severe under few-shot prompting and Chain-of-Thought (CoT) reasoning ( Shi et al. 2022 ; Yu et al. 2025 ) . Interestingly, we observe that language drift does not necessarily follow the language of the retrieved context. Instead, models frequently default to English during generation, even when the context passages are in Arabic, Russian, or other non-English languages. This indicates that English plays a dominant role beyond being a common training language: it functions as a semantic attractor in multilingual generation. Our analysis indicates that, under cross-lingual ambiguity, LLMs tend to prefer English over the context language. This fallback tendency further verifies the dominant role of English as the default trajectory in multilingual decoding. To better understand whether this fallback behavior results from misunderstanding or from generative biases, we conduct human evaluation and reference translation. Interestingly, many of the outputs that drift to the non-target language are still semantically faithful, indicating that the model has accurately understood both the task and the retrieved context. By analyzing intermediate reasoning steps (i.e., CoT traces), we find that the language inconsistency often emerges mid-generation, even when earlier steps remain in the target language. This indicates that the failure stems not from semantic comprehension, but from generation biases favoring frequent English tokens. As a result, the model produces outputs that are structurally fluent but linguistically inconsistent, reflecting a form of language collapse driven by token-level priors rather than task misunderstanding. These findings motivate the need for lightweight decoding-time strategies that maintain output language consistency without compromising reasoning performance. To this end, we introduce Soft Constrained Decoding (SCD), a token-level control mechanism that assigns soft penalties to non-target-language tokens, thereby encouraging target-language generation while preserving fluency. In contrast to hard vocabulary filtering, SCD is a flexible, model-agnostic mechanism compatible with standard decoding algorithms. Extensive experiments across diverse datasets, model backbones, target languages, and context languages demonstrate that SCD improves both output language alignment and answer quality, providing a practical solution to a persistent yet underexplored challenge in multilingual RAG. Our main contributions are as follows: • Multilingual Dataset Construction. We construct multilingual versions of HotpotQA, MuSiQue, and DuReader by translating and human-verifying all components (queries, answers, prompts, exemplars, and retrieved context), enabling controlled evaluation across four diverse languages. • Analysis of Language Drift. We conduct controlled experiments that vary only the language of retrieved contexts, revealing overlooked patterns in multilingual RAG such as performance degradation, target-language inconsistency, and a strong fallback tendency to English. Chain-of-Thought traces show that drift typically arises mid-generation due to decoding-time biases. • Training-free Language Control. We introduce SCD, a lightweight decoding-time method that softly penalizes non-target tokens. SCD is model-agnostic, requires no training, and improves both output language consistency and task accuracy across datasets and LLMs. 2 Language Drift in Multilingual RAG In this section, we conduct a comprehensive empirical investigation into the phenomenon of language drift, where model outputs deviate from the intended target language during multilingual RAG generation. To support this study, we construct multilingual variants of several benchmark RAG datasets by translating and aligning all critical components, including queries, answers, prompts, exemplars, and retrieved passages. We then evaluate LLM behavior across a range of controlled conditions. Our findings reveal a set of systematic behaviors that undermine both task accuracy and output language alignment under cross-lingual conditions. 2.1 Multilingual Dataset Construction To systematically evaluate how multilingual retrieved context influences LLM behavior in RAG, we require datasets in which the language of each input component can be independently controlled. This enables us to isolate the impact of cross-lingual retrieved passages on model reasoning and output consistency. However, no existing benchmark satisfies these constraints while remaining compatible with RAG. To address this gap,

Read the complete text