One Sample to Rule Them All: Extreme Data Efficiency in RL Scaling
Yiyuan Li γ Zhen Huang γ Yanan Wu τ Weixun Wang τ Xuefeng Li γ Yijia Luo τ Wenbo Su τ Bo Zheng τ Pengfei Liu σ γ Taobao & Tmall Group of Alibaba τ , Shanghai Jiaotong Univeristy σ , GAIR γ [email protected] Code · 2026
Read One Sample to Rule Them All: Extreme Data Efficiency in RL Scaling by Yiyuan Li γ Zhen Huang γ Yanan Wu τ Weixun Wang τ Xuefeng Li γ Yijia Luo τ Wenbo Su τ Bo Zheng τ Pengfei Liu σ γ Taobao & Tmall Group of Alibaba τ , Shanghai Jiaotong Univeristy σ , GAIR γ [email protected] Code (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03111.txt.
Introduction
Yiyuan Li and eight co-authors show that a single training example, chosen and engineered with care, can teach reasoning models a surprising amount. Reinforcement learning for language models usually leans on large volumes of high-quality samples. Their framework, polymath learning, instead designs one training sample meant to trigger improvement across many disciplines at once.
One strategically selected math reasoning sample produced significant gains not only in math but also in physics, chemistry, and biology benchmarks. By analyzing which mathematical skills such samples exercise, the authors identify what makes a sample effective. An engineered synthetic sample that packs in broader skill coverage did even better than natural examples. Across benchmarks, polymath learning with tiny data beat training on much larger datasets.
Catalog record
| CREATOR | Yiyuan Li γ Zhen Huang γ Yanan Wu τ Weixun Wang τ Xuefeng Li γ Yijia Luo τ Wenbo Su τ Bo Zheng τ Pengfei Liu σ γ Taobao & Tmall Group of Alibaba τ , Shanghai Jiaotong Univeristy σ , GAIR γ [email protected] Code |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03111
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
One Sample to Rule Them All: Extreme Data Efficiency in RL Scaling Yiyuan Li γ Zhen Huang γ Yanan Wu τ Weixun Wang τ Xuefeng Li γ Yijia Luo τ Wenbo Su τ Bo Zheng τ Pengfei Liu σ γ Taobao & Tmall Group of Alibaba τ , Shanghai Jiaotong Univeristy σ , GAIR γ [email protected] Code Abstract The reasoning ability of large language models (LLMs) can be unleashed with reinforcement learning (RL) ( 35 ; 7 ; 72 ) . The success of existing RL attempts in LLMs usually relies on high-quality samples of thousands or beyond. In this paper, we challenge fundamental assumptions about data requirements in RL for LLMs by demonstrating the remarkable effectiveness of one-shot learning. Specifically, we introduce polymath learning , a framework for designing one training sample that elicits multidisciplinary impact. We present three key findings: (1) A single, strategically selected math reasoning sample can produce significant performance improvements across multiple domains, including physics, chemistry, and biology with RL; (2) The math skills salient to reasoning suggest the characteristics of the optimal polymath sample; and (3) An engineered synthetic sample that integrates multidiscipline elements outperforms training with individual samples that naturally occur. Our approach achieves superior performance to training with larger datasets across various reasoning benchmarks, demonstrating that sample quality and design, rather than quantity, may be the key to unlock enhanced reasoning capabilities in language models. Our results suggest a shift, dubbed as sample engineering , toward precision engineering of training samples rather than simply increasing data volume. 1 Introduction Recent advances in Large Language Models (LLMs) have demonstrated the remarkable effectiveness of reinforcement learning (RL) in enhancing complex reasoning capabilities. Models like o1 ( 35 ) , Deepseek R1 ( 7 ) , and Kimi1.5 ( 50 ) have shown that RL training is able to naturally induce sophisticated reasoning behaviors, including self-verification ( 59 ) , reflection ( 47 ) , and extended chains of thought. While these advances typically rely on large-scale training data, recent work has begun to challenge this paradigm. 25 demonstrated with their LIMR approach that a strategically selected subset of just 1,389 samples can outperform the full 8k sample MATH dataset ( 16 ) . More recently, 54 made the surprising observation that even one single sample can produce meaningful improvements in math reasoning through RL, and 57 achieved similar gains by distilling high-quality reasoning paths from strong commercial models. However, this finding remains preliminary and math-specific, and leaves the critical questions of cross-domain generalization with internal abilities of LLMs unanswered: whether reasoning improvements beyond math can be achieved in similar manner? Whether a strategy exists in directing the optimal sample? Whether such sample can be synthesized to enhance the sample quality? In this paper, we build upon these emerging insights to systematically investigate the phenomenon of one-shot reinforcement learning in broad reasoning tasks termed as polymath learning . Our central finding is that a single, carefully selected math reasoning sample is able to produce significant performance gains not only in mathematics but across diverse domains including physics, chemistry, biology, as well as more general reasoning domains. This cross-domain generalization suggests that RL may enhance fundamental reasoning mechanisms rather than merely domain-specific knowledge without saturated domain-specific training. Specifically, our work addresses three research questions: Cross-Domain Generalization: Does a single mathematical reasoning sample yield improvements across diverse knowledge domains through polymath learning? We investigate the transfer mechanisms that allow reasoning patterns to transcend domain boundaries and observe that one single math sample selected on the math categories elicits greater reasoning gains of LLM than comprehensive datasets with thousands of samples, and the reasoning gains even extend to less quantitative subjects and domains that are distant from math. Optimal Sample Selection: What characteristics define the ideal training sample for maximal impact in general reasoning domains? Although the optimal polymath sample varies across domains, we find that their efficacy correlates with the salient math skills critical to reasoning, particularly the algebra and precalculus skills. Synthetic Sample Construction: How can we engineer a hybrid “meta-sample” beyond naturally occurred ones that integrates multiple reasoning skills? We propose a synthesis technique through the lens of salient math skills to construct the sample with comprehensive skill coverage and multidisciplinary context. The results illustrate that the multidisciplinary background strengthens the comprehensiveness of the salient skills, and yields greater cross-domain reasoning gains than the natural samples that mainly possess math skills in limited categories and volume. It highlights the power of individual sample amplified by properly enriching its internal multidisciplinary knowledge. By demonstrating that a single sample can trigger broad and transferrable reasoning improvements, our findings refine the current understanding of data requirements in RL, suggesting that the field may benefit from a shift toward “ sample engineering ”: deliberate selection, and synthesis of training samples to unlock reasoning capabilities more efficiently, rather than simply scaling data volume, which may potentially induce generalization degradation ( 67 ) . 2 Related Work Reinforcement Learning in Language Models Reinforcement learning has been applied to aligning language models with human intents ( 5 ) or instructions ( 38 ) through learning from human feedback. Later, it is extended to strengthen the long-form reasoning ability of models without relying on imitation of high-quality reasoning data, specifically by employing Reinforcement Learning with Verifiable Reward (RLVR) where the model outcomes can be verified and rewarded by verification functions with the advancement in RL algorithms ( 43 ; 23 ; 17 ) . However, training reliable outcome-based reward models ( 6 ) is challenging, and the rule-based reward function demonstrates effectiveness by simplifying the implementation of critic models and mitigating reward hacking ( 46 ) . In this work, we extend the reasoning ability to broader reasoning domains by learning intensively from one high quality sample. Data Efficiency in Reinforcement Learning 65 selects variance-based subset responses for GRPO training. 73 employs the most recent reward information for filtering prompts, which is beneficial to GRPO training 70 . Other than focusing on the response quality in RL training, 25 highlights the significance of prompt quality by demonstrating the effectiveness of carefully selected training subset. Further, 48 demonstrates cross-domain reasoning ability with less than 100 samples but requires a pre-warmup distillation stage, and 54 utilizes only one training sample and achieves a notable improvement in mathematical reasoning. And 75 requires no human-expert data but still relies on an external executor to generate valid answers to synthetic coding problems. However, these studies still focusing on the mathematical reasoning domain where the training data originates and neglect its broader impacts on multiple disciplines where the reasoning ability essences. Transfer Learning and Cross-Domain Generalization 1 demonstrates that small LLMs can catch up with larger counterparts in domain adaptation with few examples. And 4 adapts models to new domain by extracting domain-invariant features in existing domain. For reasoning problems, 75 unleashes an improvement in mathematical reasoning soly based on training on programming data, and 19 demonstrates that RL achieves better generalization from math to other domains than supervised fine-tuning, without a deep dive into data efficiency. 26 investigates the cross-domain impact in math reasoning, but only limits the study within logical-intensive domains like code and puzzle. In polymath