Grokking in Linear Models for Logistic Regression

Nataraj Das, Atreya Vedantam, Chandrasekhar Lakshminarayanan · 2026

Read Grokking in Linear Models for Logistic Regression by Nataraj Das, Atreya Vedantam, Chandrasekhar Lakshminarayanan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08302.txt.

Description

A model can master its training data long before it handles new data well, and that lag is usually blamed on the depth of neural networks. Atreya Vedantam and two co-authors test whether the same delay appears in a much simpler case. They use a straight-line model fitted by gradient descent to data that split cleanly into two groups.

The authors split the motion into three stages. First comes a broad population stage. Then a few key borderline examples dominate and briefly push learning backward. Finally, broad accuracy catches up. They link the delay to the time scale of that middle stage, and they back the account with controlled trials plus a small check on handwritten digits. Readers get a plain account of why slow catch-up need not require a deep network.

Catalog record

CREATORNataraj Das, Atreya Vedantam, Chandrasekhar Lakshminarayanan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08302

Browse the full library · About the Tower of Babel