Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration
Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li · 2026
Read Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration by Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06160.txt.
Description
Strong models can collapse into repeating the same wrong logic. Dayu Wang and 6 co-authors call that reasoning collapse and propose a fix in a 2026 paper. On hard math problems, random sampling often produces word-level variations of one flawed approach rather than genuinely different attempts.
Their Spectral Orthogonal Exploration uses a weaker model not for imitation but as a probe. The probe injects diverse reasoning signals into directions outside the strong model's dominant hidden-state subspace. That steers the teacher toward more varied trajectories and better exploration. On mathematical benchmarks, average accuracy and sampling efficiency rose sharply over baselines. Early evidence suggests the geometric intervention also helps on logic and code generation tasks.
Catalog record
| CREATOR | Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-06160