Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMs

KangChen, Mengdi Zhang, Yixin Cao · 2025

Read Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMs by KangChen, Mengdi Zhang, Yixin Cao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-18341.txt.

Introduction

Making language models reason longer at test time costs data and tokens. The three authors note that reasoning looks different across languages, and their pilot studies suggest those styles can help each other. They propose multilingual unification learning: train on full chains of thought written in different languages, and on chains that mix languages step by step, then intervene at decoding time.

Even small amounts of such data improved reasoning clearly. Multilingual training cut both the data needed and the inference tokens spent, while holding performance level. The method stacks with other data-efficient tricks rather than replacing them. The authors stress diverse data selection and offer the approach as an answer to the twin costs of data collection and test-time compute.

Catalog record

CREATORKangChen, Mengdi Zhang, Yixin Cao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-18341

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMs KangChen, Mengdi Zhang, Yixin Cao Abstract This paper explores the challenges of test-time scaling of large language models (LLMs), regarding both the data and inference efficiency. We highlight the diversity of multi-lingual reasoning based on our pilot studies, and then introduce a novel approach, L 2 L^{2} multi-lingual unification learning with a decoding intervention strategy for further investigation. The basic idea of L 2 L^{2} is that the reasoning process varies across different languages, which may be mutually beneficial to enhance both model performance and efficiency. In specific, there are two types of multi-lingual data: the entire long chain-of-thought annotations in different languages and the step-wise mixture of languages. By further tuning based on them, we show that even small amounts of data can significantly improve reasoning capabilities. Our findings suggest that multilingual learning reduces both the required data and the number of inference tokens while maintaining a comparable performance. Furthermore, L 2 L^{2} is orthogonal to other data efficient methods. Thus, we also emphasize the importance of diverse data selection. The L 2 L^{2} method offers a promising solution to the challenges of data collection and test-time compute efficiency in LLMs. 1 Introduction Scaling up training-time and test-time compute are two complementary strategies for enhancing the performance of large language models (LLMs). Training-time scaling allows the model to learn various knowledge through a massive corpus, but it often leads to unsatisfactory reasoning during inference, sometimes causing absurd mistakes. One explanation for this is that conventional inference primarily relies on pattern recognition from memory. In contrast, test-time scaling (e.g., OpenAI o1) significantly improves reasoning generalization by mirroring human cognitive processes, where problem-solving is not always a direct input-to-output mapping as in supervised fine-tuning, but instead involves iterative reflection and error correction, with a longer thinking process (measured by the number of predicted tokens) guiding the model toward the correct answer. Figure 1: Pilot experimental results of Deepseek-R1-32b on MATH500 dataset using different languages. Much research has explored this idea, revealing two key challenges. The first is the heavy burden of data collection. Some attempts to replicate o1 require up to 747k training samples Guan et al. 2025 , while deepseek R1-32b necessitates 80k samples to achieve o1-level performance DeepSeek-AI et al. 2025 . To reduce the costly long chain-of-thought (CoT) annotations, Sky-T1 distilled 17k samples from QwQ-32b Team 2025 using well-designed data selection strategies. S1 Muennighoff et al. 2025 further reduced the tuning dataset size to 1,000 by carefully selecting only high-quality, difficult, and diverse samples. Competition continues, with the latest work, LIMO Ye et al. 2025 , demonstrating that as few as 817 samples can enable the model to acquire long reasoning capabilities and tackle highly challenging math problems. As the demand for annotations decreases, an interesting question arises: What is the limit of “less" data? Another key challenge is the efficiency of test-time compute. As the reasoning chain expands, solving a problem often requires tens of thousands of tokens, significantly increasing the burden on inference efficiency. For ordinary problems, o1-type models use 1953% more tokens than traditional models to arrive at the same answer Chen et al. 2025 .Higher performance on math competition problems often requires tens of thousands of tokens; thus, reducing inference tokens without sacrificing performance is crucial. In this paper, we simplify the learning of test-time compute with L ess data and L ess inference tokens, namely L^{2} , through multilingual unification learning. Our core idea is that logical thinking varies across different languages, leading to various solutions and inference token lengths given the same query. As shown in figure 1 , our pilot study translates English math questions into other languages, which are prompted to Deepseek-R1-32b to seek solutions in their own languages. We can see the performance and efficiency vary a lot on the AIME24 dataset, ranging from 73.3% accuracy (French) to 40.0% (Hebrew), and from around 7k to 9k inference tokens (Section 2 ). Therefore, we assume that augmenting a small amount of CoT data using multiple languages not only enhance data diversity, but also leverage the more concise thinking patterns in certain languages to help inference efficiency. To test our assumption, we propose a three-step L^{2} multilingual unification learning: (1) collecting high-quality English samples (e.g., 6 from OpenAI o1, 1k from s1), (2) generating multilingual CoT annotations using Deepseek API, and (3) creating multilingual data by translating selected reflection steps and tagging them with language tokens; additionally, we introduce a decoding intervention strategy to guide language-specific inference. We have conducted extensive experiments. Here are our main findings: 1) Through data augmentation in different languages, only six high-quality samples can improve long reasoning performance by 20%. 2) Multilingual enhancement is orthogonal to other learning strategies. By introducing more high-quality samples, the performance of our L^{2} -32B can be continuously improved, reaching comparable 53% with 651 samples. 3) While limited data can evoke extended reasoning, performance eventually plateaus; simply increasing samples or languages yields minimal gains, highlighting the need for more diverse data selection or construction. 4) Multilingual learning enhances performance and notably reduces inference token usage compared to single-language learning. 5) Once trained with multi-lingual data, it is unnecessary to infer with different languages. Our major contributions can be summarized as follows: 1. We highlight the differences in reasoning across languages, which not only helps enhance data diversity but also has the potential to improve reasoning efficiency. 2. We propose the namely L^{2} paradigm, which is orthogonal to other efficient data methods. 3. We constructed several datasets with different languages and scale. Based on them, we trained models to gain valuable insights for future research. 2 Preliminary Observation on Multi-lingual long Reasoning We begin by evaluating multi-lingual long CoT reasoning as pilot studies mentioned in the introduction. Specifically, we translate the AIME, GPQA, and MATH500 datasets into nine languages [2] and investigate how language choice affects accuracy, normal stopping rates, and token usage in each language. We also compare models of varying scales to examine the influence of multilingual factors on extended reasoning chains. 2.1 Setup To assess multilingual long-form CoT reasoning, we adopt a selection of open-source models varying in size and pretraining architecture, chosen for their demonstrated reasoning strength and suitability for local evaluation setups: • Qwen2.5-based Models with parameter sizes of 1.5B, 7B, 14B, and 32B, including the Deepseek R1 Distilled Model, which is primarily trained on Chinese and English. • LLaMA-based Models with parameter sizes of 8B and 70B, representing models pretrained on diverse multilingual corpora. During inference, we record whether the model ends at an appropriate end-of-sequence marker (reporting the proportion of such “normal stops”), and we quantify tokens generated in each language to assess whether reasoning genuinely unfolds in the target language. Due to the space limitation, we only report the results of Deepseek-R1-32B as representative models due to its strong performance. Other results can be found in Appendix. Note that the scores are based on our careful re-implementation, which may be different from the report due to varied prompts or other config. 2.2 Observation As shown in Figure 2 , we can see that: (a) AIME (b) GPQA Figure 2: Results of

Read the complete text