TBDFiltering : Sample-Efficient Tree-Based Data Filtering
Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy · 2026
Read TBDFiltering : Sample-Efficient Tree-Based Data Filtering by Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22016.txt.
Description
Robert Istvan Busa-Fekete and co-authors filter training data for large language models with trees instead of bigger models. A model's quality depends heavily on its training data. Choosing high-quality, diverse documents is hard, because cheap and reliable quality metrics are scarce.
Asking existing language models to grade documents is common, but it cannot scale to the billions of documents used in training. The team's 2026 preprint proposes TBDFiltering, a sample-efficient tree-based method for selecting data. A light, fast filter could approach the judgment of a large model grader. Then training sets could be cleaned at a fraction of the cost. Less of one model's taste would be baked into the next.
Catalog record
| CREATOR | Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22016