What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang · 2026
Read What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression by Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01292.txt.
Introduction
Wu, Zhang, Zhang, and Fang seek one theory for knowledge transfer. Transfer spans model compression and weak-to-strong learning. Past work treats these cases in isolation. No unified frame explains why transfer works across them. The team study SGD in high-dimensional linear regression. A spectral view lets them compare regimes side by side. Two distinct mechanisms emerge from that view.
In distillation, the student reaches signals it could not learn alone. They call this spectral horizon expansion. In weak-to-strong cases, the student filters noise instead. They call that spectral denoising. Transfer strength rests on implicit regularization, they find. It also rests on varied learning speeds across the spectrum.
Catalog record
| CREATOR | Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 174 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01292