What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression

Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang · 2026

Read What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression by Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01292.txt.

Introduction

Wu, Zhang, Zhang, and Fang seek one theory for knowledge transfer. Transfer spans model compression and weak-to-strong learning. Past work treats these cases in isolation. No unified frame explains why transfer works across them. The team study SGD in high-dimensional linear regression. A spectral view lets them compare regimes side by side. Two distinct mechanisms emerge from that view.

In distillation, the student reaches signals it could not learn alone. They call this spectral horizon expansion. In weak-to-strong cases, the student filters noise instead. They call that spectral denoising. Transfer strength rests on implicit regularization, they find. It also rests on varied learning speeds across the spectrum.

Catalog record

CREATORAntiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale, Ji Q. Ren & Yevgeny LeNet, Coauthor, Wendao Wu, Fangqing Zhang, Haihan Zhang, Cong Fang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE174 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01292

Browse the full library · About the Tower of Babel