Using physics-inspired Singular Learning Theory to understand grokking & other phase transitions in modern neural networks

Anish Lakkapragada · 2025

Read Using physics-inspired Singular Learning Theory to understand grokking & other phase transitions in modern neural networks by Anish Lakkapragada (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00686.txt.

Description

Anish Lakkapragada wrote a 2025 paper on why modern neural networks learn in sudden jumps. A known puzzle is grokking, where a network does poorly for a long time and then, without warning, starts to generalize well. Phase transitions like this are hard to explain with classic learning theory.

Lakkapragada turns to singular learning theory, a framework borrowed from physics and built on algebraic geometry. Classic tools assume each setting of a model is distinct, but neural networks break that rule. Many different settings can act the same. Singular learning theory handles that mismatch, and the paper uses it to explain sudden learning shifts.

Catalog record

CREATORAnish Lakkapragada
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00686

Browse the full library · About the Tower of Babel