Superposition unifies power-law training dynamics
Zixin Jessie Chen, Hao Chen, Yizhou Liu, Jeff Gore · 2026
Read Superposition unifies power-law training dynamics by Zixin Jessie Chen, Hao Chen, Yizhou Liu, Jeff Gore (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01045.txt.
Description
Zixin Jessie Chen and co-authors test how packed features shape the speed law of training in a teacher-pupil setup. With no packing, their math ties the speed exponent to data shape and channel weight. Once a packing bottleneck bites, the law flips to a plain one-over-time curve near 1, whatever the data or channel mix. That packed regime trains up to ten times faster than step-by-step learning unpacked. Since many big nets pack features, the find may bear on a wide swath of real models.
Catalog record
| CREATOR | Zixin Jessie Chen, Hao Chen, Yizhou Liu, Jeff Gore |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48752 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01045