Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent

Chi-Ning Chou, Oscar Uzdelewicz ∗, Neng-Chun Chiu, Yao-Yuan Yang, SueYeon Chung · 2026

Read Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent by Chi-Ning Chou, Oscar Uzdelewicz ∗, Neng-Chun Chiu, Yao-Yuan Yang, SueYeon Chung (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27078.txt.

Description

Chou and four co-authors decompose learning dynamics into representation learning in the encoder and readout calibration in the final classifier. Both processes run throughout training. Shifts in their relative speed produce grokking, where test performance jumps only after a long delay, and epoch-wise double descent, where test error rises and falls while train loss drops. Readout proves train-biased before grokking onset, while representation learning is gradual but present, against the lazy-to-rich account. The decomposition also yields signatures that separate spurious from genuine generalization.

Catalog record

CREATORChi-Ning Chou, Oscar Uzdelewicz ∗, Neng-Chun Chiu, Yao-Yuan Yang, SueYeon Chung
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122423
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27078

Browse the full library · About the Tower of Babel