Synthetic Data RL: Task Definition Is All You Need

Yiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen · 2025

Read Synthetic Data RL: Task Definition Is All You Need by Yiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17063.txt.

Description

Yiduo Guo and co-authors fine-tune language models with reinforcement learning on purely synthetic practice data, in a paper from 2025. The method, Synthetic Data RL, needs nothing but a written definition of the task. It invents question and answer pairs from that definition and related documents, then adjusts question difficulty to what the model can almost solve.

Questions the model passes about half the time are picked for training. On a seven-billion-parameter model, scores jumped by 29 points on grade-school math, with solid gains across harder math, science, medicine, law, and finance tests. The method beat supervised fine-tuning on the same budget and nearly matched training on full human data. Adding one hundred human examples barely helped, the authors report.

Catalog record

CREATORYiduo Guo, Zhen Guo, MIT, Chuanwei Huang, Peking University, Zi-Ang Wang, Zekai Zhang, Peking University, Haofei Yu, UIUC, Huishuai Zhang, Yikang Shen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17063

Browse the full library · About the Tower of Babel