Why Less is More (Sometimes): A Theory of Data Curation
Elvis Dohmatob, Mohammad Pezeshki, Reyhane Askari-Hemmat · 2025
Read Why Less is More (Sometimes): A Theory of Data Curation by Elvis Dohmatob, Mohammad Pezeshki, Reyhane Askari-Hemmat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-03492.txt.
Description
Elvis Dohmatob, Mohammad Pezeshki, and Reyhane Askari-Hemmat build a theory for when training on less data beats training on more. Recent results shook the old more-is-better rule, with tiny curated sets beating bulk data on hard reasoning tasks. Their framework imagines an imperfect judge who keeps examples by difficulty and correctness.
Exact scaling curves for test error follow under both label-blind and label-aware judging. Under set conditions, dropping data provably improves generalization, and the curves say by how much. The catch lies in judge quality: poor curation can just as easily harm. A principled account of curation helps builders spend labeling effort where it counts instead of hoarding raw volume.
Catalog record
| CREATOR | Elvis Dohmatob, Mohammad Pezeshki, Reyhane Askari-Hemmat |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-03492