Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training
Przemyslaw Spyra · 2025
Read Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training by Przemyslaw Spyra (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-19063.txt.
Introduction
Przemysław Spyra, writing alone, asks whether neural networks can be trained without backpropagation, the standard method that sends errors backward through a network. Backpropagation drives most of the rising energy and computing cost of deep learning. Spyra compares three methods that avoid it: Forward-Forward, Cascaded-Forward, and Mono-Forward. Each was tuned carefully and trained on the kind of network it was designed for, then matched against a backpropagation baseline.
Mono-Forward not only matched the baseline but beat it on accuracy for its native networks. It also used up to about forty percent less energy and trained about a third faster, measured at the hardware level. Its wins come from settling into a better spot in the error landscape, not from global optimization. The study also exposes wasteful spots in the Forward-Forward design, giving a data-driven roadmap for leaner training.
Catalog record
| CREATOR | Przemyslaw Spyra |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 135 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-19063