Synthetic Data RL: Task Definition Is All You Need
Yiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen · 2025
Read Synthetic Data RL: Task Definition Is All You Need by Yiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17063.txt.
Description
Yiduo Guo and co-authors fine-tune language models with reinforcement learning on purely synthetic practice data, in a paper from 2025. The method, Synthetic Data RL, needs nothing but a written definition of the task. It invents question and answer pairs from that definition and related documents, then adjusts question difficulty to what the model can almost solve.
Questions the model passes about half the time are picked for training. On a seven-billion-parameter model, scores jumped by 29 points on grade-school math, with solid gains across harder math, science, medicine, law, and finance tests. The method beat supervised fine-tuning on the same budget and nearly matched training on full human data. Adding one hundred human examples barely helped, the authors report.
Catalog record
| CREATOR | Yiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17063