Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training

Przemyslaw Spyra · 2025

Read Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training by Przemyslaw Spyra (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-19063.txt.

Introduction

Przemysław Spyra, writing alone, asks whether neural networks can be trained without backpropagation, the standard method that sends errors backward through a network. Backpropagation drives most of the rising energy and computing cost of deep learning. Spyra compares three methods that avoid it: Forward-Forward, Cascaded-Forward, and Mono-Forward. Each was tuned carefully and trained on the kind of network it was designed for, then matched against a backpropagation baseline.

Mono-Forward not only matched the baseline but beat it on accuracy for its native networks. It also used up to about forty percent less energy and trained about a third faster, measured at the hardware level. Its wins come from settling into a better spot in the error landscape, not from global optimization. The study also exposes wasteful spots in the Forward-Forward design, giving a data-driven roadmap for leaner training.

Catalog record

CREATORPrzemyslaw Spyra
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-19063

Browse the full library · About the Tower of Babel