EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening
Yan Song · 2026
Read EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening by Yan Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26337.txt.
Description
Yan Song trims the slowest step in training gradient boosted decision trees. Histogram building eats most of the training clock, around two thirds of it in libraries like LightGBM. The usual shortcut drops random features and hopes for the best. His EMA-FS method keeps score instead. An exponential moving average tracks each feature's split gains across rounds. After a short warmup, only the top-ranked features get histograms built.
Informed screening beats blind dropping. Features that keep earning splits stay; chronic dead weight goes. The method works tree by tree and keeps LightGBM's histogram subtraction trick intact. Five datasets, from fraud detection to click prediction, test the speed. A synthetic five-hundred-feature set trains more than two and a half times faster. Extremely sparse data marks the method's limit, which the paper maps honestly.
Catalog record
| CREATOR | Yan Song |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.26337