DYNAMIC DISTILLATION AND GRADIENT CONSISTENCY FOR ROBUST LONG-TAILED INCREMENTAL LEARNING
Taigo Sakai, Kazuhiro Hotta · 2026
Read DYNAMIC DISTILLATION AND GRADIENT CONSISTENCY FOR ROBUST LONG-TAILED INCREMENTAL LEARNING by Taigo Sakai, Kazuhiro Hotta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03364.txt.
Description
Taigo Sakai and Kazuhiro Hotta address long-tailed class incremental learning, where new classes arrive in sequence from skewed data. Forgetting old classes is bad enough alone; imbalance adds under-learning of rare classes and overfitting of common ones.
Their method steadies training with a gradient consistency rule based on moving average gradients, cutting abrupt swings. It also weighs the distillation loss by an imbalance score from normalized entropy, balancing old and new knowledge adaptively. On three long-tailed benchmarks, accuracy rose consistently, with dramatic gains when tasks ran from common to rare classes, at little extra cost.
Catalog record
| CREATOR | Taigo Sakai, Kazuhiro Hotta |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03364