Using physics-inspired Singular Learning Theory to understand grokking & other phase transitions in modern neural networks
Anish Lakkapragada · 2025
Read Using physics-inspired Singular Learning Theory to understand grokking & other phase transitions in modern neural networks by Anish Lakkapragada (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00686.txt.
Description
Anish Lakkapragada wrote a 2025 paper on why modern neural networks learn in sudden jumps. A known puzzle is grokking, where a network does poorly for a long time and then, without warning, starts to generalize well. Phase transitions like this are hard to explain with classic learning theory.
Lakkapragada turns to singular learning theory, a framework borrowed from physics and built on algebraic geometry. Classic tools assume each setting of a model is distinct, but neural networks break that rule. Many different settings can act the same. Singular learning theory handles that mismatch, and the paper uses it to explain sudden learning shifts.
Catalog record
| CREATOR | Anish Lakkapragada |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00686