Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses

Nazmus Saadat As-Saquib, A N M Nafiz Abeer, Hung-Ta Chien, Byung-Jun Yoon, Suhas Kumar, Su-in Yi · 2025

Read Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses by Nazmus Saadat As-Saquib, A N M Nafiz Abeer, Hung-Ta Chien, Byung-Jun Yoon, Suhas Kumar, Su-in Yi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11030.txt.

Introduction

Nazmus Saquib and co-authors train neural networks with no backward pass at all. Their 2025 paper targets backpropagation's flaws: it needs mirrored weights, moves credit nonlocally, and freezes activity mid-step. Their method, Forward Target Propagation, swaps the backward sweep for a second forward one. Each layer gets a target built from feedforward math alone, so learning stays local and modular.

Across plain, convolutional, and recurrent nets, scores rivaled backpropagation on standard image sets. Sequence tasks with long memory also worked. Under low-precision and odd hardware limits, the method beat its teacher. The authors aim it at frugal on-device learning and brain-like chips.

Catalog record

CREATORNazmus Saadat As-Saquib, A N M Nafiz Abeer, Hung-Ta Chien, Byung-Jun Yoon, Suhas Kumar, Su-in Yi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11030

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses Nazmus Saadat As-Saquib, A N M Nafiz Abeer, Hung-Ta Chien, Byung-Jun Yoon, Suhas Kumar, Su-in Yi Abstract Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that—despite its widespread success—suffers from key limitations in both biological and hardware perspectives. These include backward error propagation by symmetric weights, non-local credit assignment, and frozen activity during backward passes. We propose Forward Target Propagation (FTP), a biologically plausible and computationally efficient alternative that replaces the backward pass with a second forward pass. FTP estimates layer-wise targets using only feedforward computations, eliminating the need for symmetric feedback weights or learnable inverse functions, hence enabling modular and local learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating accuracies competitive with BP on MNIST, CIFAR-10, and CIFAR-100, as well as effective modeling of long-term dependencies in sequential tasks. Moreover, FTP outperforms BP under quantized low-precision and emerging hardware constraints while also demonstrating substantial efficiency gains over other biologically inspired methods such as target propagation variants and forward-only learning algorithms. With its minimal computational overhead, forward-only nature, and hardware compatibility, FTP provides a promising direction for energy-efficient on-device learning and neuromorphic computing. 1 Introduction Backpropagation (BP) has been the foundational algorithm for training neural networks, driving the success of deep learning in tasks such as image recognition, language modeling, and decision-making rumelhart1986backprop ; attention_vashwani ; feweshot_language . Despite its proven effectiveness, BP faces several limitations in terms of both biological plausibility and emerging hardware compatibility. One significant issue is its reliance on symmetric weight transport between the forward and backward passes, a requirement that does not align with how biological neurons function, where synaptic updates are not typically symmetric whittington2019credit . Moreover, during training on edge devices and emerging analog in-memory computing hardware such as resistive random access memory (RRAM) and phase change memory (PCM) crossbars, symmetric weight transport poses additional challenges Yi2023 ; Yi_SONOS ; ISAAC . In each iteration, updated weight matrices must be written and verified in their transposed form for error propagation back through the network. However, achieving perfect symmetry in the analog hardware between the forward and backward matrices during this process is often unfeasible insitu_memristtor , leading to performance degradation. This issue becomes even more severe in edge applications, where low-bit precision constraints further exacerbate the problem lowbit_issues . Additionally, BP propagates a global error signal backward through multiple layers, in contrast to the localized learning signals that characterize biological neural systems Lillicrap2020 . Moreover, the weight updates in BP are computed using information from distant layers, implying that neurons must access far-off signals during learning. This non-locality contradicts the local learning mechanisms observed in biological systems. BP also suffers from the vanishing and exploding gradient problems, where gradients can either become too small to make meaningful updates or too large, destabilizing training. This issue is especially prominent in very deep or recurrent networks bengio1994learning . Lastly, the assumption of layer-wise processing and exact error computation at each layer does not map onto biological learning systems, which operate in a noisier, more heuristic manner rather than relying on precise gradient-based methods crick1989recent ; scellier2023energy_suhas . These challenges have motivated research into biologically plausible alternatives that better match the way the brain might perform credit assignment during learning Lillicrap2020 ; lillicrap2016random ; bengio2014target ; Hinton2022TheFA ; PredictiveCA . This paradigm is worth exploring because the brain achieves remarkable efficiency and adaptability through local learning rules, providing valuable insights for developing energy-efficient learning algorithms. In this work, we introduce Forward Target Propagation (FTP), a biologically inspired, forward-only learning algorithm that addresses several limitations of backpropagation, including the need for symmetric weight transport, non-local error propagation, and vanishing gradients. Motivated by evidence from biological systems, where learning is believed to occur via local computations and forward signaling, FTP eliminates the backward pass entirely by estimating layer-wise targets through a second forward sweep using the projections from the output predictions and target labels. Synaptic updates are then computed by minimizing the discrepancy between each layer’s activation and its corresponding target, thereby enabling local credit assignment without exact global gradient information. By combining biologically inspired principles with a mathematically grounded framework, FTP advances the development of biologically plausible, energy-efficient, and hardware-compatible alternatives to traditional backpropagation. To summarize, our key contributions in this work are: • We introduce Forward Target Propagation (FTP), a biologically plausible, forward-only learning algorithm that enables local credit assignment without relying on backward error propagation or symmetric weight transport. • We evaluate FTP across fully connected, convolutional, and recurrent neural networks on image classification and multivariate time-series forecasting tasks, demonstrating competitive accuracy with backpropagation and improved computational efficiency compared to other biologically inspired methods. • We present both theoretical justification and empirical evidence showing strong alignment between FTP and backpropagation, measured via the angular similarity between their respective gradient directions during training. • We assess FTP’s robustness in low-precision and noisy hardware settings and benchmark its efficiency in TinyML scenarios, thereby showing that FTP achieves near-backpropagation-level efficiency while outperforming other biologically plausible algorithms. This highlights FTP’s suitability for edge and neuromorphic computing. 2 Related Work This section discusses biologically plausible approaches to credit assignment, focusing on different feedback mechanisms and forward-only models that address the limitations of backpropagation. The subsections cover random feedback algorithms, other biologically motivated credit assignment methods, and forward learning models. 2.1 Conventional and Biologically Plausible Methods BP has long served as the backbone of neural network training, utilizing both forward and backward passes to compute error gradients. During the forward pass, input signals are propagated through the network to the output layer, where errors are assessed by comparing the predicted outputs to the target values. The subsequent backward pass propagates these error derivatives back through the network using the same weights that were used during the forward pass. The outer product of the activity vectors from both passes constructs the gradient matrices of the global objective. While BP has proven effective in many applications, it is often criticized for its biological implausibility, raising questions about whether similar mechanisms exist in biological systems rumelhart1986backprop . To address the limitations of backpropagation, several biologically inspired learning algorithms have been proposed. Feedback Alignment (FA) lillicrap2016random replaces symmetric feedback with fixed random weights, showing that learning can still succeed without precise weight mirroring. Direct Feedback Alignment (DFA) extends this idea by projecting output errors directly to hidden units via random matrices nokland2016direct . A simplified variant, Direct Random Target Projection (DRTP), further removes dependence on global error signals by using target labels to update hidden layers frenkel2021learning . Although these methods mitigate issues such as weight transport and update locking, they remain limited in biological plausibility due to their reliance on global error signals and frozen network dynamics. Moreover, DRTP has demonstrated lower accuracy and limited scalability compared to more recent approaches, as evidenced by their results. (a) BP (b) DTP (c) PEPITA (d) FTP Figure 1 : Configuration of various learning methods: a) BP, b) DTP, c) PEPITA, and d) FTP. Black arrows represent forward paths, while blue arrows indicate error/target paths. Each {\bm{W}} denotes the learnable weight matrices of hidden layers,

Read the complete text