Is Grokking Worthwhile? Functional Analysis and Transferability of Generalization Circuits in Transformers
Kaiyu He, Mian Zhang, Peilin Wu, Xinya Du, Zhiyu Zoey Chen · 2026
Read Is Grokking Worthwhile? Functional Analysis and Transferability of Generalization Circuits in Transformers by Kaiyu He, Mian Zhang, Peilin Wu, Xinya Du, Zhiyu Zoey Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09049.txt.
Description
Kaiyu He and four co-authors ask if slow, deep learn in a small net is worth its cost. Some nets learn a task fast by rote. A few keep train for long past that point, then snap to a true rule. That snap is called grokking. Past work says grokked nets build a shared circuit for two step thought, a known weak spot for chat models.
The team tests if a grokked net beats a plain net on new tasks. They pull the circuit out and check if it moves to fresh work. A reader who trains nets gets a cost call. Long grok train is dear, so its gain must be real and must move.
Catalog record
| CREATOR | Kaiyu He, Mian Zhang, Peilin Wu, Xinya Du, Zhiyu Zoey Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-09049