DataRater: Meta-Learned Dataset Curation
Dan A. Calian, Gregory Farquhar, Iurii Kemaev, Luisa M. Zintgraf, Jeremy Shar, Junhyuk Oh, András György, Tom Schaul, Jeffrey Dean, Hado van Hasselt, David Silver · 2025
Read DataRater: Meta-Learned Dataset Curation by Dan A. Calian, Gregory Farquhar, Iurii Kemaev, Luisa M. Zintgraf, Jeremy Shar, Junhyuk Oh, András György, Tom Schaul, Jeffrey Dean, Hado van Hasselt, David Silver (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17895.txt.
Description
Dan Calian and co-authors present DataRater, a system that learns which training examples are worth teaching to a language model, in a 2025 paper. Data quality drives model quality, yet curation today relies on hand-tuned mixtures and rough filters. DataRater instead scores each example by meta-learning: it estimates how much training on that example would help performance on held-out data.
The scoring is learned through meta-gradients, so the rater improves at judging value as it sees more training runs. Across model sizes and datasets, filtering training data with DataRater significantly improved compute efficiency, meaning models reached the same quality with less computation. The authors argue learned curation scales better than manual rules.
Catalog record
| CREATOR | Dan A. Calian, Gregory Farquhar, Iurii Kemaev, Luisa M. Zintgraf, Jeremy Shar, Junhyuk Oh, András György, Tom Schaul, Jeffrey Dean, Hado van Hasselt, David Silver |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17895