EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening

Yan Song · 2026

Read EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening by Yan Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26337.txt.

Description

Yan Song trims the slowest step in training gradient boosted decision trees. Histogram building eats most of the training clock, around two thirds of it in libraries like LightGBM. The usual shortcut drops random features and hopes for the best. His EMA-FS method keeps score instead. An exponential moving average tracks each feature's split gains across rounds. After a short warmup, only the top-ranked features get histograms built.

Informed screening beats blind dropping. Features that keep earning splits stay; chronic dead weight goes. The method works tree by tree and keeps LightGBM's histogram subtraction trick intact. Five datasets, from fraud detection to click prediction, test the speed. A synthetic five-hundred-feature set trains more than two and a half times faster. Extremely sparse data marks the method's limit, which the paper maps honestly.

Catalog record

CREATORYan Song
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26337

Browse the full library · About the Tower of Babel