Hebbian Learning with Global Direction

Wenjia Hua, Kejie Zhao, Luziwei Leng, Ran Cheng, Yuxin Ma, Qinghai Guo · 2026

Read Hebbian Learning with Global Direction by Wenjia Hua, Kejie Zhao, Luziwei Leng, Ran Cheng, Yuxin Ma, Qinghai Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21367.txt.

Introduction

Wenjia Hua and five co-authors revisit Hebbian learning, a training rule inspired by how brain cells wire together. The classic backpropagation rule trains most deep networks, but it is not how brains learn and it costs a lot to run. Hebbian rules are local and cheaper, yet they have lagged in performance.

The authors add a shared global direction to guide the local updates. That signal helps the local rules pull together instead of drifting. The goal is training that stays brain-like but works at modern scale. Success would open cheaper, more brain-like ways to build capable networks.

Catalog record

CREATORWenjia Hua, Kejie Zhao, Luziwei Leng, Ran Cheng, Yuxin Ma, Qinghai Guo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21367

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hebbian Learning with Global Direction Wenjia Hua, Kejie Zhao, Luziwei Leng, Ran Cheng, Yuxin Ma, Qinghai Guo Abstract Backpropagation algorithm has driven the remarkable success of deep neural networks, but its lack of biological plausibility and high computational costs have motivated the ongoing search for alternative training methods. Hebbian learning has attracted considerable interest as a biologically plausible alternative to backpropagation. Nevertheless, its exclusive reliance on local information, without consideration of global task objectives, fundamentally limits its scalability. Inspired by the biological synergy between neuromodulators and local plasticity, we introduce a novel model-agnostic Global-guided Hebbian Learning (GHL) framework, which seamlessly integrates local and global information to scale up across diverse networks and tasks. In specific, the local component employs Oja’s rule with competitive learning to ensure stable and effective local updates. Meanwhile, the global component introduces a sign-based signal that guides the direction of local Hebbian plasticity updates. Extensive experiments demonstrate that our method consistently outperforms existing Hebbian approaches. Notably, on large-scale network and complex datasets like ImageNet, our framework achieves the competitive results and significantly narrows the gap with standard backpropagation. † † address: 1 Department of CSE, Southern University of Science and Technology , Shenzhen, China 2 ACS Lab, Huawei Technologies Co., Ltd. , Shenzhen, China 3 Department of Data Science and Artificial Intelligence, Department of Computing, Hong Kong Polytechnic University , Hong Kong, China 1 Introduction Figure 1: Comparison of different training methods. Left: Standard BP training process, using global error signals to compute precise gradients for weight updates. Middle: Two-factor Hebbian learning approach, which relies only on local information from pre and post synaptic neurons to update weights, without considering global task objectives. Right: Our method, Global-guided Hebbian Learning (GHL), integrates local Hebbian plasticity with a global sign-based modulation. Deep Neural Networks (DNNs) have achieved revolutionary success in recent years, with Backpropagation (BP) playing an important role. Despite its remarkable achievements, BP still has some limitations such as the weight transport problem [ 20 ] . Also, BP’s reliance on the precise, global propagation of error signals lacks local plasticity, making it biologically implausible and contrasting sharply with the mechanisms of neural computation in the brain [ 32 , 22 ] . To address these limitations, the research community has explored more efficient and biologically inspired training alternatives, such as Feedback Alignment (FA) [ 21 ] , Predictive Coding (PC) [ 28 ] , and Equilibrium Propagation (EP) [ 29 ] . Among these approaches, Hebbian learning stands out due to its strong biological plausibility and simple local plasticity rules. Early studies have successfully integrated Hebbian learning with deep models like Convolutional Neural Networks (CNNs) [ 1 , 17 ] . However, these attempts resulted in poor performance. Also, in these early frameworks, the role of Hebbian learning was often restricted, either functioning as an update rule for specific network modules or being used exclusively during a pre-training phase before standard end-to-end training with backpropagation. To enhance the performance of Hebbian approaches in end-to-end training, researchers have introduced mechanisms such as competitive learning [ 31 ] and anti-Hebbian plasticity [ 15 ] , leading to representative methods including SoftHebb [ 23 , 13 ] , HWTA-BCM [ 24 ] , and FastHebb [ 19 , 18 ] . However, like other alternatives to backpropagation, these methods are mostly tested on simple tasks such as MNIST or CIFAR, and with shallow models like MLPs or small CNNs [ 10 , 12 ] . When facing more complex tasks such as ImageNet or deeper networks, their performance is much worse than backpropagation, showing a critical scalability bottleneck [ 13 , 27 ] . Furthermore, these methods often require specialized network architectures or modules to achieve desirable performance. For instance, DeepSoftHebb [ 13 ] , which achieves state-of-the-art performance in Hebbian learning, relies heavily on rapidly expanding convolutional channels and specific activation functions. Any modifications to its architecture can lead to a drastic performance drop, sometimes even worse than a single-layer network. This strong dependency on specific network structures severely restricts the practical application and generalization of bio-inspired learning algorithms. The root of this limitation in existing Hebbian methods lies in their over reliance on local information, which takes the form of two-factor learning [ 25 ] , where weight updates are determined solely by the activity of pre and post synaptic neurons. While this pure locality largely circumvents BP’s issues like weight transport and update locking, it suffers from a lack of global information. From a broader perspective, this is also inconsistent with biological reality. While the brain lacks precise error backpropagation, it does possess feedback mechanisms, such as neuromodulators, that convey global, task-relevant signals [ 32 , 22 ] . Such a learning rule, which is modulated by a global signal, is also known as the three-factor learning rule. In this rule, synaptic changes depend not only on local activity but also on a third modulatory factor that provides global information. In biological systems, various neuromodulators serve as global signals, controlled by complex feedback mechanisms [ 16 ] . However, in the machine learning community, there is currently no unified implementation for effectively incorporating such global guidance signals into existing artificial neural networks. To fuse local plasticity with global objectives, we propose a novel and model-agnostic framework called G lobal-guided H ebbian L earning ( GHL ), designed to bridge the gap between theoretical research and practical machine learning implementation. In our framework, the local update signal is provided by Hebbian learning, using competitive learning and Oja’s rule. Inspired by the biological phenomenon where neuromodulators can ‘flip’ the polarity of synaptic plasticity [ 33 , 5 , 4 ] , we instantiate the global guidance signal as a sign signal derived from the global task gradient. This global sign signal provides only directional information (i.e., potentiation or depression) without requiring the precise gradient magnitude. We extensively benchmarked our proposed framework against state-of-the-art Hebbian methods, demonstrating superior performance across multiple network architectures and achieving accuracy comparable to end-to-end BP. More importantly, our framework demonstrates strong scalability and generalization, and its performance is less affected by changes in network structure. We successfully applied high-performance Hebbian learning to the large and complex ImageNet task, reducing the performance gap with standard backpropagation to within 4% on ResNet-50. In addition, we are the first to successfully scale a bio-inspired learning algorithm to an extremely deep ResNet-1202 architecture. These results push the boundaries of bio-inspired learning. Our main contributions are as follows: • We propose a novel, model-agnostic Global-guided Hebbian Learning (GHL) framework, which guides local synaptic plasticity using the sign of the global gradient. This approach naturally combines global optimization objectives with local update rules. • Through extensive experiments, we demonstrate that our framework not only outperforms existing Hebbian learning algorithms but is also scalable to challenging datasets like ImageNet, greatly improving the practical use of bio-inspired learning. • Our work bridges the gap between neuroscience-inspired learning mechanisms and those used in machine learning, offering new possibilities for interpreting learning processes in biological systems. 2 Method 2.1 Overview of GHL framework The three-factor learning rule provides a model for synaptic plasticity. It posits that the change in a synaptic weight \Delta w_{ik}

Read the complete text