On the Infinite Width and Depth Limits of Predictive Coding Networks

Francesco Innocenti, El Mehdi Achour, Rafal Bogacz · 2026

Read On the Infinite Width and Depth Limits of Predictive Coding Networks by Francesco Innocenti, El Mehdi Achour, Rafal Bogacz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07697.txt.

Description

Predictive coding offers a biologically plausible option to backpropagation, minimizing an energy over network activities before updating weights, but its scaling action was unclear. Francesco Innocenti and two co-authors study predictive coding networks in the infinite width and depth limits.

For linear networks they derive stable, non-lazy parameterizations when scaling width and depth, revealing that standard predictive coding outputs explode with width during training. Under stable parameterizations, predictive coding gradients at activity equilibrium converge to backpropagation gradients for networks much wider than deep. Learning theorists get the scaling theory predictive coding lacked, plus tests showing strong gradient match at large width.

Catalog record

CREATORFrancesco Innocenti, El Mehdi Achour, Rafal Bogacz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07697

Browse the full library · About the Tower of Babel