Lang : Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance

Yuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu, Jian Yang, Xin Chen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Jingbo Zhu, Tong Xiao · 2026

Read Lang : Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance by Yuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu, Jian Yang, Xin Chen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Jingbo Zhu, Tong Xiao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22567.txt.

Introduction

Fan and others tackle a trade off in multilingual reasoning: forcing input language consistency harms reasoning, while chasing reasoning lets answers drift toward English. Their LANG framework steers non English reasoning with language matched hints, then withdraws that scaffolding on a decay schedule. A language adaptive switch tunes learning horizons to each language's difficulty, blocking dependence on the hints. On hard multilingual maths tests, LANG lifts reasoning while keeping language consistency, and the gains carry beyond maths to broader language alignment.

Catalog record

CREATORYuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu, Jian Yang, Xin Chen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Jingbo Zhu, Tong Xiao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98443
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22567

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lang : Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance Yuchun Fan, Bei Li, Peiguang Li, Yilin Wang, Yongyu Mu, Jian Yang, Xin Chen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Jingbo Zhu, Tong Xiao Abstract Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models (LLMs), yet its benefits have not fully translated to multilingual contexts. Existing methods struggle with a fundamental trade-off: prioritizing input-language consistency severely hampers reasoning quality, while prioritizing reasoning often leads to unintended language drift toward English. We address this challenge with Lang , a novel framework that leverages language-conditioned hints to guide exploration in non-English reasoning tasks. Our method incorporates two key mechanisms to prevent dependency on these hints: a progressive decay schedule that gradually withdraws scaffolding, and a language-adaptive switch that tailors learning horizons to specific language difficulties. Empirical results on challenging multilingual mathematical benchmarks reveal that Lang substantially enhances reasoning performance without compromising language consistency. Moreover, we show that our framework generalizes beyond mathematics, fostering more consistent language alignment across model layers 1 1 1 The project will be available at: https://github.com/fmm170/LANG . 1 Introduction Recent years have witnessed the rise of large reasoning models such as OpenAI-o3 ( 34 ) and DeepSeek-R1 ( 11 ) , which leverage reinforcement learning with verifiable rewards (RLVR) to incentivize multi-step reasoning capabilities. However, these advances remain largely English-centric, leaving a critical performance gap in non-English settings, especially for low-resource languages ( 50 ; 27 ) . Beyond accuracy, when it comes to multilingual scenarios, language-consistent reasoning is equally crucial for native-user interpretability. However, as illustrated in Figure 1 , current large language models (LLMs) face two coupled challenges. First, when we explicitly require models to reason in the input language, strong reasoning capability in English does not reliably transfer across languages, leading to significant performance degradation in non-English settings. Second, when such language constraints are relaxed, models frequently drift toward English, reducing readability for native users. Consequently, it remains non-trivial to balance language consistency and reasoning performance. Language-Constrain Reasoning User query: 196의 양의 정수 약수는 모두 몇 개입니까? 한국어로 생각하고 답변하세요. (Use Korean to think and answer.) <think> 먼저 유클리드 호제법을 사용하여 196=2\times 98 로 나타낸다. 이어서 98을 소인수분해한다[…]. 196 의 약수에 대한 소인수분해의 경우의 수는 3\times 2=6 가지이다.</think> 정답은 \boxed{6} 이다. The reasoning trace is language-consistent , but the answer is incorrect . Language-Drifting Reasoning User query: 196의 양의 정수 약수는 모두 몇 개입니까? <think> First prime factorize 196=2^{2}\cdot 7^{2} . The prime factor decomposition of any divisor […] to distinct integers.</think> The answer is \boxed{9} . The answer is correct , but the reasoning trace is language-inconsistent . Figure 1: The trade-off between answer accuracy and the language of reasoning trace in multilingual reasoning scenarios. The Korean question in English means “How many positive integer divisors are there of 196?” To address these challenges, recent studies primarily follow two directions. One line of work introduces explicit prompt constraints to steer models to reason in the language of the user’s query during reinforcement learning (RL) rollouts or at inference time ( 39 ; 28 ) . However, such prompt-based steering offers limited control and often comes at the cost of reasoning quality. Building on this, another line of work further introduces language-consistency rewards in RL, encouraging models to align the output language with the input ( 61 ) . Nevertheless, when the model’s multilingual reasoning ability is limited, trajectories that satisfy such strict objectives are extremely scarce, exacerbating reward sparsity and hindering effective exploration during multilingual reasoning RL. Inspired by prior studies that leverage hints ( e.g. , partial reasoning steps from expert demonstrations) to alleviate reward sparsity during RL training ( 57 ; 48 ; 26 ) , we propose Lang , a language-adaptive hint-guided RL framework for multilingual reasoning. Lang uses language-conditioned hints as scaffolding to bootstrap exploration in non-English settings. However, we find that keeping multilingual hints throughout training induces strong hint dependence, leading to substantial performance degradation when hints are unavailable at test time. To address this, we draw inspiration from scheduled sampling ( 3 ; 58 ; 40 ) , and introduce a progressive hint-decay schedule that reduces hint exposure over training, shifting learning from hint-conditioned rollouts to autonomous multilingual reasoning. Moreover, considering the difference in learning difficulty across languages, we further employ a language-adaptive switch that sets language-specific horizons for turning off hints. To validate the effectiveness of Lang , we conduct extensive experiments on two representative LLMs across different sizes, covering two challenging multilingual mathematical reasoning benchmarks: MMATH ( 27 ) and PolyMath ( 50 ) . The results show that Lang effectively enhances multilingual reasoning ability while preserving language consistency, improving accuracy over vanilla GRPO by 24.1% on MMATH and 18.7% on PolyMath. Moreover, Lang can also be generalized to non-mathematical multilingual tasks. Further analysis highlights that our method achieves consistently language-consistent reasoning across layers. 2 Related Work 2.1 Multilingual Mathematical Reasoning Recent advances in LRMs have significantly improved their multi-step reasoning capabilities. However, reasoning performance remains highly uneven across languages ( 8 ; 59 ; 64 ) , and models often suffer from language drift ( i.e. , responding in a language different from the user’s query), degrading both performance and user experience in non-English settings ( 50 ; 27 ; 9 ; 49 ; 60 ) . Current approaches to enhancing multilingual reasoning under language consistency conditions mainly fall into two categories. One line of work ( 39 ; 27 ) uses explicit constraints in prompts to guide models to reason in the user-desired language. Another line of research ( 36 ; 54 ; 28 ; 19 ) introduces language-consistency rewards in reinforcement learning to further encourage alignment between input and output languages. Building upon this, our work attempts to leverage multilingual reasoning traces as explicit guidance to steer model exploration within specific language spaces, a perspective that remains underexplored. 2.2 Hint-guided Reinforcement Learning A well-known challenge in RLVR is the reward sparsity issue: when task difficulty exceeds model capability, sampled trajectories can all be incorrect, yielding zero group-wise advantage ( 56 ; 4 ; 13 ) . To mitigate this, prior studies ( 57 ; 48 ; 26 ) have focused on injecting hint-like partial solutions into prompts as in-context guidance to improve exploration. This issue is even more pronounced in multilingual reasoning, where non-English reasoning ability typically lags behind English, making it difficult to sample feasible trajectories. Building on this line of work, we introduce a language-adaptive hint decay strategy that scales guidance by language difficulty, effectively breaking the exploration bottleneck in multilingual reasoning. 3 Preliminary Study 0 100 200 300 400 500 600 0.2 0.4 0.6 Training Steps (a) Policy Entropy 0 100 200 300 400 500 600 10 15 20 25 30 Training Steps (b) Average Accuracy 0 100 200 300 400 500 600 200 300 400 500 600 Training Steps (c) Response Length 0 0.2 0.4 0.6 0.8 Reward score 0 0.2 0.4 0.6 0.8 Repeat Score Vanilla GRPO QuestA solid lines: left y -axis * -marked lines: right y -axis Figure 2: The comparison of Vanilla GRPO and QuestA during RL training. Blue curves denote Vanilla GRPO and orange curves denote QuestA . In

Read the complete text