Neural Architecture Search with Mixed Bio-inspired Learning Rules
Imane Hamzaoui, Riyadh Baghdadi · 2025
Read Neural Architecture Search with Mixed Bio-inspired Learning Rules by Imane Hamzaoui, Riyadh Baghdadi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13485.txt.
Introduction
Imane Hamzaoui and Riyadh Baghdadi mix bio inspired learning rules through architecture search in a 2025 paper. Networks inspired by the brain are robust and energy frugal, and they match cortical biology better than standard models. Yet they lag in accuracy and scale behind backpropagation based networks.
Their search method chooses both network structure and a learning rule for each layer. Networks with mixed rules beat single rule networks. The best model set records for bio inspired systems on several image benchmarks, and in some settings even passed comparable standard networks while keeping robustness advantages.
Catalog record
| CREATOR | Imane Hamzaoui, Riyadh Baghdadi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-13485
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neural Architecture Search with Mixed Bio-inspired Learning Rules Imane Hamzaoui, Riyadh Baghdadi Abstract Bio-inspired neural networks are attractive for their adversarial robustness, energy frugality, and closer alignment with cortical physiology, yet they often lag behind back-propagation (BP) based models in accuracy and ability to scale. We show that allowing the use of different bio-inspired learning rules in different layers, discovered automatically by a tailored neural-architecture-search (NAS) procedure, bridges this gap. Starting from standard NAS baselines, we enlarge the search space to include bio-inspired learning rules and use NAS to find the best architecture and learning rule to use in each layer. We show that neural networks that use different bio-inspired learning rules for different layers have better accuracy than those that use a single rule across all the layers. The resulting NN that uses a mix of bio-inspired learning rules sets new records for bio-inspired models: 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet. In some regimes, they even surpass comparable BP-based networks while retaining their robustness advantages. Our results suggest that layer-wise diversity in learning rules allows better scalability and accuracy, and motivates further research on mixing multiple bio-inspired learning rules in the same network. † † paperid: 123 1 Introduction Bio-inspired neural networks have gained significant traction recently. They are more biologically plausible, more robust to adversarial attacks [ Sanfiz and Akrout(2021b) , Moraitis et al.(2022a)Moraitis, Toichkin, Journé, Chua, and Guo ] , and have higher energy efficiency [ Malcolm and Casco-Rodriguez(2023) ] . Despite their significant success, these methods are still in development and fall short in terms of accuracy and in scaling effectively for complex tasks. In this paper, we aim to use neural architecture search to improve state-of-the-art bio-inspired neural networks. Neural Architecture Search (NAS) is a method that automatically searches for neural network architectures and effectively finds high-quality architectures. This is usually done by exploring different types of architectures and picking one that best satisfies an objective (e.g., minimizing the loss). Early work based on Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) was able to find interesting architectures, however such methods take significant amount of time. AmoebaNet-B [ Real et al.(2019)Real, Aggarwal, Huang, and Le ] , for example, takes around 3150 GPU days. Several efficient NAS methods have been developed since then, offering a trade-off between search time and accuracy. Differentiable Architecture Search (DARTS) [ Liu et al.(2018)Liu, Simonyan, and Yang ] is a notable example. It has shown success in designing differentiable neural architecture search. EG-NAS [ Cai et al.(2024)Cai, Chen, Liu, Ling, and Lai ] , a follow-up work, improved on top of DARTS using genetic algorithms. It has shown an ability to find an accuracy competitive with DARTS in 0.1 GPU days compared to 0.4 GPU day for DARTS on CIFAR10. In this paper, we build on top of DARTS and EG-NAS to propose BioNAS, a framework for neural architecture search that explores different bio-inspired neural network architectures and learning rules. To the best of our knowledge, BioNAS is the first NAS framework that explores the use of different bio-inspired learning rules in the same neural network. This choice is motivated by recent research that shows that the brain might be using different learning rules instead of a single rule [ Marblestone et al.(2016)Marblestone, Wayne, and Kording ] . We show that interesting architectures can be discovered within this framework, surpassing state-of-the-art bio-inspired neural networks, and achieving an accuracy of 95.16% on CIFAR10 (compared to 91.8% for ResNet56 trained with Uniform Sign-concordant Feedbacks (uSF)), 76.48% on CIFAR-100 (compared to 73.04% on SNASNet), 43.42 % on ImageNet16-120 and 60,51% on ImageNet. In addition to the improved accuracy, the use of bio-inspired learning rules also leads to better robustness against adversarial attacks. We show that a part of this improvement comes from the use of different learning rules instead of using a single learning rule for all the layers. The contributions of this paper are as follows 1. We propose BioNAS, a framework for neural architecture search designed for bio-inspired neural networks. To the best of our knowledge, BioNAS is the first framework that incorporates learning rules as part of the neural architecture search. 2. We show that using different bio-inspired learning rules improves the accuracy of bio-inspired models as well as their adversarial robustness. 3. We release BioNAS to the community. and make the code available via this link ( https://anonymous.4open.science/r/LR-NAS-DFE1 ) . 2 Background & Related Works 2.1 Biologically Inspired Neural Networks Biologically inspired neural networks are neural networks that draw inspiration from principles observed in natural nervous systems. Broadly, these networks integrate biological insights in two ways: (i) Architectures. This involves designing network structures that closely resemble biological neural circuits. Examples include using spiking neurons (which mimic biological neurons that communicate via discrete spikes), dendritic branching (reflecting the complex input-processing capabilities of real neurons), local recurrent connections (small-scale loops found abundantly in brain circuits), or sparsely connected layers similar to the olfactory circuits observed in fruit flies. Such architectures can still leverage conventional neural network optimization methods for training (e.g., back-propagation). (ii) Learning rules. Alternatively, networks may retain standard deep learning architectures, such as convolutional neural networks (CNNs) or transformers, but adopt biologically plausible learning rules. These rules typically rely only on local information flow (each neuron updates based only on its immediate neighbors or local environment) or utilize random feedback connections. Such approaches avoid the strict symmetry and global knowledge required by traditional back-propagation algorithms. Our framework specifically explores this second approach, extending Neural Architecture Search (NAS) to automatically select biologically plausible learning rules for each layer during network construction. 2.2 Bio-inspired Learning Rules Back-propagation (BP) [ Rumelhart et al.(1986)Rumelhart, Hinton, and Williams ] is the most widely used algorithm for training neural networks. It operates by first performing a forward pass, where the network processes inputs to produce predictions, and then executing a backward pass, computing gradients of the loss function to update network weights. However, BP relies on symmetric weight matrices—meaning the weights used for feedback (gradient calculation) are the transposed version of those used for the forward pass—a requirement considered biologically implausible. To address this, several alternatives have been proposed. Feedback Alignment (FA) [ Lillicrap et al.(2016)Lillicrap, Cownden, Tweed, and Akerman ] simplifies the feedback process by sending random error signals from the network’s output layer back to the hidden layers. In other words, it replaces the transposed weight matrix with a fixed random feedback matrix, relaxing the need for symmetry. Direct Feedback Alignment (DFA) [ Nøkland(2016) ] goes further by sending random feedback from the output layer directly to each hidden layer, bypassing intermediates. Further research indicates that using sign-concordant feedback matrices—such as Uniform Sign Feedback (uSF) and Batchwise Random-Magnitude Sign Feedback (brSF) [ Liao et al.(2016)Liao, Leibo, and Poggio ] —can more closely mimic BP performance by preserving the signs of feedback signals. Figure 1 summarizes these credit-assignment schemes. Other forward-only paradigms include PEPITA [ Dellaferrera and Kreiman(2023) ] , which uses a second forward pass modulated by error signals, and Forward-Forward (FF) [ Hinton(2022) ] , which relies on a layer-wise “goodness” metric computed for positive and negative data. Finally, Hebbian learning , inspired by synaptic