KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networks
James Hazelden, Laura Driscoll, Eli Shlizerman, Eric Shea-Brown · 2025
Read KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networks by James Hazelden, Laura Driscoll, Eli Shlizerman, Eric Shea-Brown (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06381.txt.
Description
James Hazelden, Laura Driscoll, Eli Shlizerman, and Eric Shea-Brown dissect how gradient descent reshapes the internal activity of recurrent networks, in a paper published in two thousand twenty-five. As these networks train, their dynamics can collapse onto simple patterns and form latent representations that seem to support strong generalization.
Neuroscientists use the shape of such representations to compare artificial and biological learning. Yet rigorous theory for finite, nonlinear models has lagged behind the observations. The authors take an operator view of the gradient flow, the continuous process that tracks how training changes the model. They show this flow splits into a product of two operators, one acting on parameters and one on the network's activity patterns. The split, which they call KPFlow, exposes the mechanism behind dynamic collapse. The work gives a sharper tool for explaining why trained recurrent systems organize themselves the way they do.
Catalog record
| CREATOR | James Hazelden, Laura Driscoll, Eli Shlizerman, Eric Shea-Brown |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 108 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06381