On the Infinite Width and Depth Limits of Predictive Coding Networks
Francesco Innocenti, El Mehdi Achour, Rafal Bogacz · 2026
Read On the Infinite Width and Depth Limits of Predictive Coding Networks by Francesco Innocenti, El Mehdi Achour, Rafal Bogacz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07697.txt.
Description
Predictive coding offers a biologically plausible option to backpropagation, minimizing an energy over network activities before updating weights, but its scaling action was unclear. Francesco Innocenti and two co-authors study predictive coding networks in the infinite width and depth limits.
For linear networks they derive stable, non-lazy parameterizations when scaling width and depth, revealing that standard predictive coding outputs explode with width during training. Under stable parameterizations, predictive coding gradients at activity equilibrium converge to backpropagation gradients for networks much wider than deep. Learning theorists get the scaling theory predictive coding lacked, plus tests showing strong gradient match at large width.
Catalog record
| CREATOR | Francesco Innocenti, El Mehdi Achour, Rafal Bogacz |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07697