Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation
Ashim Gupta Vivek Srikumar Kahlert School of Computing University of Utah [email protected] · 2025
Read Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation by Ashim Gupta Vivek Srikumar Kahlert School of Computing University of Utah [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21941.txt.
Introduction
Ashim Gupta and Vivek Srikumar test a simple trick to lift text quality in many languages in a 2025 paper. The trick draws many draft replies at use time and keeps the best one, as judged by a scoring tool. It had shown gains on logic tasks, but proof for work in many tongues was scarce.
The team tried the trick on two broad test sets that span many languages. Gains were steady, and passed 35 percent in some cases. A score based on how smooth the text reads worked well for open prompts. For math and code tasks, only a reward-based judge lifted scores. Picking the right judge for the task is the key lesson.
Catalog record
| CREATOR | Ashim Gupta Vivek Srikumar Kahlert School of Computing University of Utah [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21941
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation Ashim Gupta Vivek Srikumar Kahlert School of Computing University of Utah [email protected] Abstract Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach using perplexity- and reward-based verifiers on two multilingual benchmarks: the Aya Evaluation Suite and m-ArenaHard. Our results show consistent quality improvements, with gains exceeding 35% in some cases. While perplexity-based scoring is effective for open-ended prompts, only reward-based verifiers improve performance on tasks requiring reasoning (e.g., math, code). Our results demonstrate the broader utility of repeated sampling for multilingual text generation and underscore the importance of selecting right verifiers for the task. 1 Introduction Scaling the inference-time compute has significantly improved LLM performance on coding, mathematics, and other reasoning tasks ( Madaan et al. 2023 ; Wu et al. 2024 ; Snell et al. 2024 ; Brown et al. 2024 ; Bansal et al. 2024 ; Muennighoff et al. 2025 , e.g.,) . There are two broad strategies for test-time scaling: (1) with lengthy chains of thought using an LLM trained to reason longer ( Guo et al. 2025 ; Muennighoff et al. 2025 ) , or (2) by repeated sampling, where a verifier or a scorer model selects a final answer from generated samples ( Brown et al. 2024 ; Snell et al. 2024 ) . Despite the success of test-time scaling in reasoning-heavy tasks, its effect on multilingual text generation remains largely unexplored. In this work, we investigate whether the repeated sampling strategy can improve multilingual generation, even for benchmarks that do not explicitly focus on reasoning, code, or mathematics. We employ two approaches for selecting the final output: (a) perplexity-based scoring using an auto-regressive language model, and (b) reward scoring with pre-trained reward models. The perplexity-based method considers only the generated responses, selecting the output with the lowest perplexity score as a proxy for fluency. In contrast, the reward-based method evaluates both the prompt and the generated response together, scoring each prompt-response pair to determine the best output. Figure 1 illustrates this procedure. Figure 1: Repeated sampling procedure using a verifier to pick the final answer. Repeated sampling may allow models to explore a wider range of potential outputs. Consequently, in the case of multilingual generation, especially for low-resource languages, the likelihood of generating text that is both high quality and contextually appropriate increases. When model confidence is limited, e.g., for underrepresented languages, initial outputs may not be the best. By selecting from multiple generated candidates using criteria based on fluency (via perplexity) or prompt-response alignment (via reward models), we aim to identify outputs that are more consistent with human judgments or benchmark standards. We show empirically that repeated sampling with verification improves multilingual text generation. Using two perplexity- and reward-based verifiers each, we observe that output quality improves with the number of samples. On the Aya Evaluation Suite ( Singh et al. 2024 ) , a multilingual benchmark for open-ended generation, both verifier types yield substantial gains; a 2B-parameter Gemma-based reward model achieves up to 35% increase in win rates. However, for m-ArenaHard ( Dang et al. 2024 ) , a benchmark focused on programming and mathematics, only reward-based verifiers lead to measurable improvements. Our contributions are twofold. (1) Ours is the first work to demonstrate that multilingual generation benefits from test-time scaling, highlighting the broader applicability of repeated sampling beyond traditional English-based reasoning tasks. (2) We show that verifier choice is critical: both perplexity- and reward-based scoring helps general multilingual generation, but only reward-based verifiers are effective for tasks requiring reasoning. 2 Experimental Setup This section outlines our experimental setup, including details on the datasets, languages, LLMs, verifiers, and the evaluation protocol used. Dataset and Languages. We use two datasets for evaluation. First is the Aya Evaluation Suite Singh et al. 2024 , that contains open-ended conversation-style, culturally grounded prompts for evaluating multilingual LLM generation. We conduct evaluations on nine languages: Arabic, English, Hindi, Punjabi, Portuguese, Russian, Telugu, Turkish, and Chinese. The second dataset is m-ArenaHard Dang et al. 2024 , the multilingual variant of ArenaHard-Auto ( Li et al. 2024 ) . This dataset contains the translated prompts from the ArenaHard-Auto benchmark. With this dataset, we perform evaluation in seven languages: Arabic, Czech, English, Hindi, Japanese, Portuguese, and Vietnamese. Verifiers. We employ two types of verifiers to score and select generations from repeated sampling. For perplexity-based verification, we select sample that has the lowest perplexity and use the pre-trained variants of LLaMA-3.1-8B ( Grattafiori et al. 2024 ) and Gemma-2B models ( Team 2024 ) . For reward-based scoring, we use two fine-tuned reward models that are based on the above pre-trained ones: URM-LLaMA-3.1-8B ( Lou et al. 2024 ) , and GRM-Gemma-2B-rewardmodel-ft ( Yang et al. 2024b ) . Both reward models rank high on the RewardBench benchmark Lambert et al. 2024 and are among the best models in their respective parameter ranges. Note that while the underlying pre-trained models are multilingual, the reward models have only been trained on the English preference data. Section A.2 elaborates on our verifiers. LLMs Evaluated. We evaluate commonly used open-weight multilingual LLMs at both their large and small parameter scales. Specifically, we use Qwen-2.5 Yang et al. 2024a at 14B, 72B parameter scales, Llama-3 at 8B and 70B scales Grattafiori et al. 2024 , and aya-expanse at 8B and 32B Dang et al. 2024 . Evaluation Protocol. We follow prior work and adopt the LLM-as-a-Judge framework for evaluating multilingual generation Dang et al. 2024 , based on MT-Bench Zheng et al. 2023 . The core evaluation metric is the win/loss rate — i.e., how often a model’s response is judged better or worse than a baseline. We report the difference ( delta ) between win and loss rates, averaged across all languages. All evaluations use gemini-2.0-flash as the judge, which we choose because of its cost efficiency and strong multilingual performance. Our baseline consists of single-sample generations without repeated sampling, using temperature-based decoding. Due to its stochastic nature, the baseline output may vary across runs, introducing randomness into the delta scores. Comparing against multiple baseline samples is impractically expensive. In a pilot study ( fig. 5 , appendix), we found that the variance across runs is small ( <2 percentage points), indicating that our findings are robust to baseline selection. Figure 2: Test-time scaling with repeated sampling for Aya Evaluation Suite. The plots show the difference between win and loss rates (delta). We see that all verifiers — both perplexity-based (PPL) and reward-based (RM) — can improve generation quality. 3 Results and Analysis We plot the impact of number of samples ( n ) used for repeated sampling vs the average delta score achieved to show the scaling effect. We perform this evaluation for n=5,10,15,20,25 and then scale it in steps of 25 after that (to n=50,75,100 ). Figure 3: Test-time scaling with repeated sampling for m-ArenaHard. The plots show the difference between win and loss rates (delta). Only reward-model based verifiers improve generation quality. Figure 4: Training-time compute vs Test-time compute. Figure 2 shows the main scaling plots for the Aya Evaluation Suite and fig. 3 for the m-ArenaHard. For