HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm
Jie-En Yao, Hong-En Chen, C.-C. Jay Kuo · 2026
Read HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm by Jie-En Yao, Hong-En Chen, C.-C. Jay Kuo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24797.txt.
Introduction
Jie-En Yao and co-authors improve the Forward-Forward algorithm with hierarchical and contrastive learning. Backpropagation-trained networks perform well in vision but remain biologically implausible, costly, and hard to interpret. Forward-Forward trains each layer alone through local goodness objectives, yet lacks cross-layer coordination. Their HCL-FF framework adds a coarse-to-fine hierarchy guiding features from basic cues to rich meaning, plus a contrastive objective for class alignment. On CIFAR-10, CIFAR-100, and Tiny-ImageNet it sets new Forward-Forward records, up 5.46 percent on CIFAR-10.
Catalog record
| CREATOR | Jie-En Yao, Hong-En Chen, C.-C. Jay Kuo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60395 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24797
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm Jie-En Yao, Hong-En Chen, C.-C. Jay Kuo Abstract Deep neural networks trained with backpropagation have achieved outstanding performance in vision tasks but remain biologically implausible, computationally demanding, and difficult to interpret. The Forward-Forward (FF) algorithm offers a promising alternative by training each layer independently through local goodness objectives. However, its purely local optimization lacks hierarchical coordination across layers, and the decoupling of goodness from features leaves the representations unconstrained and semantically ambiguous. We propose a Hierarchical and Contrastive Learning FF framework (HCL-FF) to address these limitations. HCL-FF introduces (1) a coarse-to-fine hierarchical learning strategy that guides representations from low-level cues to high-level semantics, and (2) a supervised contrastive objective that enforces class-discriminative alignment after goodness decoupling. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that HCL-FF achieves new state-of-the-art performance among FF-based methods, with notable accuracy gains of + 5.46 % +5.46\% , + 17.00 % +17.00\% , and + 12.51 % +12.51\% , respectively. 1 Introduction Backpropagation (BP) 48 has long been the foundation of modern deep learning. Despite its empirical success, BP remains biologically implausible 16 ; 9 ; 20 ; 31 , as there is no evidence that the brain propagates error signals backward or stores neural activations for future updates. It is also computationally demanding, requiring gradients and activations to be stored and propagated across all layers. Moreover, it often yields opaque internal representations, limiting interpretability and transparency. These limitations have motivated growing interest in non-backpropagation learning paradigms 5 ; 52 ; 13 ; 40 ; 20 ; 22 . Among them, the Forward-Forward (FF) algorithm 18 has emerged as a promising bottom-up alternative that trains each layer independently to maximize or minimize its goodness , defined as the magnitude of the layer’s activations. The layer-wise training scheme enables biologically plausible learning without gradient propagation across layers. It also improves both parallelism 19 ; 53 and memory efficiency 46 , as layers update without storing activations or waiting for global gradients, making it attractive for lightweight and edge settings 4 . Finally, FF offers enhanced interpretability through direct visualization of layer-wise goodness responses 45 . Despite these appealing properties, the FF algorithm faces two key limitations. First, its greedy layer-wise learning lacks hierarchical coordination. While CNNs trained with backpropagation naturally learn a progression from low-level cues to high-level semantics, FF forces shallow layers to infer high-level semantic concepts directly, often leading to suboptimal early-layer representations. Second, multi-layer learning remains ineffective due to the goodness decoupling problem. Specifically, FF decouples the goodness information from the layer output via vector-length normalization 18 or layer normalization 53 , which removes magnitude information and preserves only relative activation patterns, preventing subsequent layers from trivially inheriting the previous goodness signal. However, the local goodness objective optimizes only the goodness score, i.e., the activation magnitude, leaving the goodness-decoupled features unconstrained and semantically ambiguous once the goodness is removed. Recent studies attempt to mitigate the resulting information loss by relaxing strict decoupling, using batch normalization 45 ; 12 or triangle activation function 7 . Although these modifications accelerate learning, they leak goodness information across layers, allowing deeper layers to exploit it rather than learn new patterns. As a result, these methods tend to overfit and are limited to relatively shallow networks. This reveals a fundamental decoupling dilemma in FF learning: Strict goodness decoupling prevents deeper layers from overfitting on previous goodness signals, yet it inevitably discards valuable semantic information encoded in activation magnitudes, the only part of the representation directly optimized by the goodness objective. To address these limitations and resolve the decoupling dilemma, we propose a Hierarchical and Contrastive Learning FF framework (HCL-FF) that preserves the layer-wise nature of FF while introducing two key innovations. First, we introduce a coarse-to-fine hierarchical learning strategy: shallow layers are supervised using coarse super-class targets, and deeper layers are progressively assigned finer-grained labels. This curriculum reduces early-layer complexity, improves gradient-free coordination across depth, and encourages representations to evolve from broad semantic structure to fine-grained discrimination. Second, we incorporate supervised contrastive learning 21 on the goodness-decoupled features. This explicitly constrains their relational geometry, aligning samples from the same class and separating different classes, thereby restoring the semantic structure that is lost when magnitude (goodness) is removed. Extensive experiments show that HCL-FF sets a new state-of-the-art among FF-based methods, delivering higher accuracy, stronger feature separability, and stable semantics after goodness decoupling. Notably, HCL-FF achieves improvements of +5.46\% on CIFAR-10, +17.00\% on CIFAR-100, and +12.51\% on Tiny-ImageNet, demonstrating the effectiveness of hierarchical coordination and contrastive grounding in overcoming long-standing challenges of FF learning. 2 Related Work 2.1 Learning Without Backpropagation Recent studies 42 have explored alternatives to backpropagation, pursuing biologically plausible learning mechanisms. Feedback alignment and its variants 30 ; 39 ; 3 ; 14 ; 25 replace the transposed forward weights used in backpropagation with random or learned feedback matrices to relax the weight-symmetry/transport constraint. Target Propagation methods 6 ; 28 ; 13 ; 5 approximate local targets through learned inverse mappings, allowing updates without backpropagated gradients. Hebbian-based methods 24 ; 35 ; 38 ; 20 define weight updates from correlations between pre- and post-synaptic activity, often with competitive learning dynamics 49 ; 16 ; 51 . Local representation alignment methods 41 ; 40 compute the local alignment targets with error units and error weights. The forward-only framework PEPITA 10 ; 52 replaces the backward pass with a second forward pass that perturbs inputs using error information, yielding Hebbian-like local updates. Signal propagation 22 treats the target as part of the forward input, propagating both input and target jointly through the network to achieve single-pass learning. 2.2 Forward-Forward Algorithm The Forward-Forward (FF) algorithm 18 replaces backpropagation with a layer-wise local goodness objective. Formally, given an input x , each layer computes \displaystyle y^{(1)}=\phi(W^{(1)}x), (1) \displaystyle y^{(\ell)}=\phi(W^{(\ell)}z^{{(\ell-1)}}),\quad\text{for}\quad 2\leq\ell\leq L, (2) \displaystyle g^{(\ell)}=\sum_{i=1}^{N^{(\ell)}}(y_{i}^{(\ell)})^{2},\quad z^{(\ell)}=\frac{y^{(\ell)}}{\sqrt{\frac{1}{N^{(\ell)}}g^{(\ell)}+\epsilon}}, (3) where \phi denotes the activation function, W^{(\ell)} is the weight matrix of layer \ell , z^{{(\ell-1)}} is the feature from the previous layer, and N^{(\ell)} is the number of neurons. The term g^{(\ell)} represents the goodness, defined as the squared sum of activations. To prevent deeper layers from trivially inheriting the goodness signal, each layer normalizes its activations to produce a unit-length feature vector z^{(\ell)} , thereby decoupling goodness from the representation. During training, FF maximizes g^{(\ell)} for positive samples (e.g., correct image–label pairs) and minimizes it for negative samples, encouraging layers to produce stronger activations for correct labels and suppress responses for incorrect ones. Since its introduction, the FF algorithm has inspired many extensions 15 ; 43 ; 50 ; 47 ; 29 ; 58 ; 44 . However, existing variants largely remain limited by two core issues: (1) the lack of hierarchical coordination and (2) the decoupling dilemma. CwComp 45 leverages the CIFAR-100 super-classes but uses them merely as a heuristic to handle larger label spaces. To address coordination, Trifecta 12 introduces block-wise backpropagation, but this violates FF’s layer-wise independence. Collaborative FF 33 adds a global goodness objective but sacrifices FF’s inherent parallelism, as layers must wait for a global signal before updating. To address the decoupling issue, CwComp 45 and Trifecta 12 replace strict goodness decoupling with batch normalization, while SCFF 7 employs a triangle