Data Recipes for Reasoning Models
Etash Guha*, Ryan Marten*, Sedrick Keh*, Negin Raoof*, Georgios Smyrnis*, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang · 2025
Read Data Recipes for Reasoning Models by Etash Guha*, Ryan Marten*, Sedrick Keh*, Negin Raoof*, Georgios Smyrnis*, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04178.txt.
Introduction
Etash Guha and a large team of co-authors describe open data sets for training reasoning models, in a 2025 paper. Top reasoning models often train on private data, leaving open questions about the best recipes. The OpenThoughts project aims to make such data public. An earlier set from the project trained a model that matched a leading rival on standard tests.
The team then ran more than a thousand controlled tests on each step of their data pipeline. The improved set holds well over a million examples. A model trained on it set new marks for its size on math, code, and science tests. Gains over a key rival ranged from fifteen to twenty points. All data sets and models are public, the authors note, so others can build on them.
Catalog record
| CREATOR | Etash Guha*, Ryan Marten*, Sedrick Keh*, Negin Raoof*, Georgios Smyrnis*, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 188 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04178