Reason XL : Shifting LLM Reasoning Language Without Sacrificing Performance
Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt, Josef van Genabith, Alexander Löser, Simon Ostermann · 2026
Read Reason XL : Shifting LLM Reasoning Language Without Sacrificing Performance by Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt, Josef van Genabith, Alexander Löser, Simon Ostermann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12378.txt.
Introduction
Daniil Gurgurov and five colleagues address a blind spot in multilingual language models. Such models usually reason in English, even for non-English questions. Their ReasonXL corpus gathers over 2,500,000 aligned reasoning examples. The languages are English, German, French, Italian, and Spanish. Each example holds a prompt, a reasoning trace, and a final answer. A two-stage training recipe then teaches models to reason fully in a target language. It matches or beats English reasoning in two of four tested languages.
Catalog record
| CREATOR | Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt, Josef van Genabith, Alexander Löser, Simon Ostermann |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55685 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12378
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reason XL : Shifting LLM Reasoning Language Without Sacrificing Performance Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt, Josef van Genabith, Alexander Löser, Simon Ostermann Abstract Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in their production of reasoning traces. Even when tasked with non-English problems, these models predominantly reason in English, creating a fundamental mismatch for non-English usage scenarios. We address this disparity directly with three contributions. (i) We introduce Reason XL , the first large-scale parallel corpus of cross-domain reasoning traces spanning five European languages (English, German, French, Italian, and Spanish), with over two million aligned samples per language, each comprising prompts, reasoning traces, and final outputs, enabling direct supervision of language-specific reasoning. (ii) Using Reason XL , we demonstrate that LLMs can be adapted to reason entirely in a desired target language, using a simple two-stage pipeline of supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR). The resulting models match or exceed baseline performance, with minimal loss in general knowledge and broadly preserved cross-lingual transfer. (iii) We conduct an extensive representational analysis of the adaptation and find a clear functional division across model depth: early layers contain an activation bottleneck that causally determines language identity, while upper layers concentrate the weight and activation changes driven by adaptation. We further find that RLVR achieves greater behavioral divergence from the base model with smaller parameter updates than SFT, suggesting a more efficient representational rerouting despite much smaller weight updates. 1 Introduction Recent work shows that training large language models (LLMs) on reasoning data in a small number of languages can generalize to other languages via cross-lingual transfer ( Mistral-AI et al. 2025 ; Ghosh et al. 2025 ; Sutawika et al. 2026 ) . However, despite these apparent multilingual capabilities, models mostly default to English for intermediate reasoning, even when prompted in other languages ( Wang et al. 2025b ) . The absence of non-English reasoning traces creates a barrier for users whose native language is not English, reducing understanding and trust in the LLM ( Aggarwal et al. 2025 ) . Users interacting with LLMs in their native language expect reasoning that is transparent and followable. This English-centric behavior is also accompanied by consistently higher accuracy in English compared to other languages ( Saji et al. 2026 ; Yong et al. 2025 ) . Existing work has primarily documented these disparities without directly optimizing for reasoning in a target language. Closing this gap requires two pieces that are currently missing: (1) sufficiently large non-English reasoning data ( Lai & Nissim 2024 ; E Sobhani et al. 2026 ) , and (2) evidence that language-specific post-training can match English-level performance ( Qi et al. 2025 ) . We address both by introducing, to our best knowledge, the first large-scale parallel multilingual reasoning corpus and using it in a two-stage post-training pipeline to demonstrate that an LLM’s reasoning language can be durably redirected to a target language without sacrificing task performance. Beyond this empirical result, our controlled setup enables a representational analysis of how language identity is encoded and manipulated within LLMs, which we study through weight- and activation-level analyses ( Saphra & Wiegreffe 2024 ) . Specifically, our contributions are: Research Contributions • (i) The release of Reason XL , a large-scale, five-language parallel corpus of reasoning data with over two million reasoning traces per language spanning multiple domains, with rich annotations for every sample. 1 1 1 XL stands for cross-l ingual, and reflects the scale of the corpus. The data and models are available on HuggingFace: DGurgurov/reasonxl . Reason XL is a living corpus under active development; the current release serves as a stable foundation, with ongoing expansion exceeding 20–30B tokens per language. • (ii) An evaluation of language-specific reasoning adaptation through a two-stage pipeline of fine-tuning and reinforcement learning, including target-language reasoning, general knowledge retention, and cross-lingual transfer. • (iii) A mechanistic analysis of how adaptation reshapes model internals in terms of weight changes and activation patterns across layers. 2 Related Work Multilingual Reasoning. Despite rapid progress in multilingual LLMs ( Dang et al. 2024 ; Qwen 2025 ; Mistral-AI et al. 2025 ) , models still default to English for intermediate reasoning even when prompted in other languages ( Wang et al. 2025b ) , achieving higher accuracy in English than in other languages ( Tam et al. 2025 ; Qi et al. 2025 ; Saji et al. 2026 ) . Existing approaches fall into three categories. Translation-based methods use English as a pivot in various ways ( Shi et al. 2023 ; Ahuja et al. 2023 ; Etxaniz et al. 2024 ; Zhang et al. 2024 ; Ko et al. 2025 ; Yoon et al. 2024 ; She et al. 2024 ; Fan et al. 2025b ) . Prompt-based methods attempt to control reasoning language without fine-tuning, but often trade accuracy for compliance ( Huang et al. 2023 ; Yong et al. 2025 ; Qi et al. 2025 ) . Post-training approaches improve language alignment more effectively, even with limited data, though a performance gap remains ( Qi et al. 2025 ; Son et al. 2025 ) . Test-time scaling provides additional but inconsistent gains and tends to benefit English disproportionately ( Son et al. 2025 ) . Progress across all approaches is constrained by the scarcity of large-scale non-English reasoning data ( Lai & Nissim 2024 ; E Sobhani et al. 2026 ; Ghosh et al. 2025 ) . The Case for non-English Reasoning. Beyond accuracy, in-language reasoning improves trust, allows users to inspect reasoning traces directly, and preserves linguistic and cultural nuance ( Aggarwal et al. 2025 ) . Reliance on English as a pivot introduces “lost in translation” errors that can distort problem semantics ( Saji et al. 2026 ) and limits oversight in multilingual deployments ( Qi et al. 2025 ) . These considerations motivate approaches that directly optimize target-language reasoning rather than routing through English. We address both the data gap and the optimization challenge by introducing a large-scale parallel multilingual reasoning corpus (Section 3 ) and using it to study language-specific post-training without relying on English as a pivot (Section 4 ). Multilingual Interpretability. Prior work studies how multilingual LLMs encode language through language-specific neuron analyses ( Tang et al. 2024 ; Kojima et al. 2024 ; Deng et al. 2025 ) , cross-layer vocabulary projections ( Wendler et al. 2024 ; Wu et al. 2025 ) , and cross-lingual alignment and steering ( Ferrando & Costa-jussà 2024 ; Gurgurov et al. 2026 ) . However, most studies focus on pretrained models in isolation, leaving open how post-training adaptation reshapes language representations and where the output language is causally determined. Our controlled setup (base, SFT, and RL checkpoints on parallel data) enables a targeted study of these and other effects; we present an initial analysis in Section 5 3 Methodology 3.1 Dataset Creation We annotate, curate, and translate a novel multilingual cross-domain reasoning corpus, Reason XL , sampled from 10 existing resources, including data from Olmo et al. 2025 , NVIDIA 2025 , Bercovich et al. 2025 , and Wang et al. 2025a . A full overview