TBDFiltering : Sample-Efficient Tree-Based Data Filtering

Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy · 2026

Read TBDFiltering : Sample-Efficient Tree-Based Data Filtering by Robert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22016.txt.

Description

Robert Istvan Busa-Fekete and co-authors filter training data for large language models with trees instead of bigger models. A model's quality depends heavily on its training data. Choosing high-quality, diverse documents is hard, because cheap and reliable quality metrics are scarce.

Asking existing language models to grade documents is common, but it cannot scale to the billions of documents used in training. The team's 2026 preprint proposes TBDFiltering, a sample-efficient tree-based method for selecting data. A light, fast filter could approach the judgment of a large model grader. Then training sets could be cleaned at a fraction of the cost. Less of one model's taste would be baked into the next.

Catalog record

CREATORRobert Istvan Busa-Fekete, Julian Zimmert, Anne Xiangyi Zheng, Claudio Gentile, Andràs Gyorgy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22016

Browse the full library · About the Tower of Babel