Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective
Joel Wendin, Claudio Altafini · 2025
Read Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective by Joel Wendin, Claudio Altafini (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10362.txt.
Introduction
Joel Wendin and Claudio Altafini survey what is known about training deep linear neural networks, in an article accepted by SIAM Review. These are networks built only from matrix multiplication, with no activation functions, trained on a simple squared error loss. Stripped down this way, training can be studied exactly as a set of differential equations.
The survey describes the loss landscape in detail, with its many global minima and saddle points but no local minima. It explains how the training equations preserve certain quantities and how a quotient space view tidies the picture of critical points. Readers also get results on stability at saddle points and on what a training run actually learns.
Catalog record
| CREATOR | Joel Wendin, Claudio Altafini |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 167 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-10362