Data Filtering Methods for Training Language Models
E. Shevchenko and E. Bruches · 2026
Read Data Filtering Methods for Training Language Models by E. Shevchenko and E. Bruches (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29807.txt.
Description
Egor Shevchenko and Elena Bruches compare two ways to catch label errors in Russian training data. Label noise, even in benchmarks, drags down how well models generalize. They test two filters, Confident Learning and Dataset Cartography, across three Russian corpora of varied size, classes, and domain, using a fine-tuned Russian BERT. Filtering helps most on small, noisy sets, where Confident Learning lifts macro F1 markedly, while on large, clean sets it adds little. Targeted removal beats random removal, confirming that both filters find real errors, not smaller data.
Catalog record
| CREATOR | E. Shevchenko and E. Bruches |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38695 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29807