Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration

Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li · 2026

Read Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration by Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06160.txt.

Description

Strong models can collapse into repeating the same wrong logic. Dayu Wang and 6 co-authors call that reasoning collapse and propose a fix in a 2026 paper. On hard math problems, random sampling often produces word-level variations of one flawed approach rather than genuinely different attempts.

Their Spectral Orthogonal Exploration uses a weaker model not for imitation but as a probe. The probe injects diverse reasoning signals into directions outside the strong model's dominant hidden-state subspace. That steers the teacher toward more varied trajectories and better exploration. On mathematical benchmarks, average accuracy and sampling efficiency rose sharply over baselines. Early evidence suggests the geometric intervention also helps on logic and code generation tasks.

Catalog record

CREATORDayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-06160

Browse the full library · About the Tower of Babel