A flexible framework for structural plasticity in GPU-accelerated sparse spiking neural networks Publication type: Paper
James C. Knight 1 \orcid 0000-0003-0577-0074, Johanna Senk 1,2,∗ \orcid 0000-0002-6304-062X and Thomas Nowotny 1 \orcid 0000-0002-4451-915X · 2025
Read A flexible framework for structural plasticity in GPU-accelerated sparse spiking neural networks Publication type: Paper by James C. Knight 1 \orcid 0000-0003-0577-0074, Johanna Senk 1,2,∗ \orcid 0000-0002-6304-062X and Thomas Nowotny 1 \orcid 0000-0002-4451-915X (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19764.txt.
Introduction
James C. Knight, Johanna Senk, and Thomas Nowotny add structural plasticity to the GeNN simulator for spiking neural networks. Brains learn by strengthening links, but also by growing new ones and removing others. Most machine learning frameworks assume fixed, dense wiring. They gain nothing from pruning during training. GeNN already simulates sparse spiking networks fast on graphics processors.
The new framework lets users plug in their own rewiring rules and run them on the GPU. Demos include training sparse classifiers with the e-prop rule and the DEEP R rewiring method. Those cut training time by up to ten times while matching dense models. A further demo forms topographic maps without supervision, faster than real time. The tool opens sparsity research for neuromorphic computing.
Catalog record
| CREATOR | James C. Knight 1 \orcid 0000-0003-0577-0074, Johanna Senk 1,2,∗ \orcid 0000-0002-6304-062X and Thomas Nowotny 1 \orcid 0000-0002-4451-915X |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19764
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A flexible framework for structural plasticity in GPU-accelerated sparse spiking neural networks Publication type: Paper James C. Knight 1 \orcid 0000-0003-0577-0074, Johanna Senk 1,2,∗ \orcid 0000-0002-6304-062X and Thomas Nowotny 1 \orcid 0000-0002-4451-915X Abstract The majority of research in both training Artificial Neural Networks (ANNs) and modeling learning in biological brains focuses on synaptic plasticity, where learning equates to changing the strength of existing connections. However, in biological brains, structural plasticity – where new connections are created and others removed – is also vital, not only for effective learning but also for recovery from damage and optimal resource usage. Inspired by structural plasticity, pruning is often used in machine learning to remove weak connections from trained models to reduce the computational requirements of inference. However, the machine learning frameworks typically used for backpropagation-based training of both ANNs and Spiking Neural Networks (SNNs) are optimized for dense connectivity, meaning that pruning does not help reduce the training costs of ever-larger models. The GeNN simulator already supports efficient GPU-accelerated simulation of sparse SNNs for computational neuroscience and machine learning. Here, we present a new flexible framework for implementing GPU-accelerated structural plasticity rules and demonstrate this first using the e-prop supervised learning rule and DEEP R to train efficient, sparse SNN classifiers and then, in an unsupervised learning context, to learn topographic maps. Compared to baseline dense models, our sparse classifiers reduce training time by up to 10 × 10\times while the DEEP R rewiring enables them to perform as well as the original models. We demonstrate topographic map formation in faster-than-realtime simulations, provide insights into the connectivity evolution, and measure simulation speed versus network size. The proposed framework will enable further research into achieving and maintaining sparsity in network structure and neural communication, as well as exploring the computational benefits of sparsity in a range of neuromorphic applications. keywords spiking neural network, structural plasticity, topographic map, learning rules \RS@ifundefined subsecref name = \RSsectxt \RS@ifundefined thmref name = theorem \RS@ifundefined lemref name = lemma 1 Introduction A simulation of a Spiking Neural Network (SNN) is usually a two-step process: First, data structures for neurons and synapses are instantiated on the machine to define the network, including its connectivity; afterwards, the network state is calculated over time starting from some initial conditions according to the neuron and synapse dynamics. This approach assumes that the connectivity is fixed once the network is constructed (typically using population-level connection rules [ 1 ] ) and at most the strength of connections may be subject to change during the simulation [ 2 ] . However, biological evidence suggests that the connectivity itself is continuously changing, not only during development but also in the adult brain [ 3 ] . These changes are associated with learning, adaptation, memory, recovery from lesions, drug addiction, and occur following enriched experience, after sensory deprivation or upon stimulation. Besides the biological evidence, it has been found that algorithms abstracted from structural plasticity can benefit machine-learning (ML) models – in particular in the form of pruning [ 4 ] . To shed light on candidate mechanisms of structural plasticity and their function for learning and memory, computational models have been developed to study, for example, whether changes are induced by activity, homeostasis, stochasticity, or other factors and simulator developers have implemented functionality for modifying networks during runtime (i.e., forming or eliminating synapses) to simulate these models, e.g. [ 5 , 6 , 7 , 8 , 9 , 10 , 11 ] . However, structural plasticity remains underexplored because simulating networks with it is computationally expensive: First, structural changes in biology are slow (hours, days) and therefore need long simulation durations. Second, there are complex inter-dependencies with other biological mechanisms such as synaptic plasticity and some processes may need global instead of only local information, hence limiting implementations. Third, continuously applying changes to the connectivity clashes with the traditional approach of keeping the network structure untouched in computer memory during state propagation which requires to rethink how data structures are handled. Different simulation platforms including neuromorphic systems therefore face different difficulties. Structural plasticity algorithms have previously largely been implemented in CPU-based SNN simulators [ 5 , 12 ] and neuromorphic systems such as SpiNNaker, where the main computational elements are scalar CPU cores [ 7 , 8 ] and thus the standard routines used for modifying connectivity can be employed. [ 11 ] presented a structural plasticity implementation for the BrainScaleS–2 [ 13 ] Plasticity Processing Unit (PPU) – a Single Instruction Multiple Thread (SIMD) processor that is tightly integrated with the BrainScaleS–2 neuromorphic system. Here, we propose what we believe is the first structural plasticity framework for implementation on general purpose parallel hardware such as GPUs. The framework aims to combine flexibility (i.e., the possibility to freely explore different structural plasticity mechanisms) with efficiency (i.e., fast simulations). We identify the following constraints: 1. Adding and removing connections cannot require the connectivity data structures to be re-allocated. 2. The amount of data that needs to be moved around when adding or removing a connection must be minimal. 3. The connectivity must still be indexed and read efficiently (i.e., via a coalesced memory read on a GPU). The GeNN simulator [ 14 , 15 ] was originally developed for Computational Neuroscience research but its longstanding focus on flexibility and its efficient implementation of sparse data structures on GPU accelerators has enabled it to become a powerful tool for many areas of SNN research. We have previously added support for various machine-learning-specific functionalities including parallel batch simulation of models – allowing multiple copies of the model to be run simultaneously in order to maximize GPU occupancy – and user-defined ‘custom update’ operations [ 16 , 17 ] . Here, we present a new extension to GeNN which allows users to easily define hybrid CPU/GPU structural plasticity algorithms using the same code-generation based approach already used to define neuron models and learning rules as well as initialization algorithms. Using this framework we demonstrate two applications. Firstly, building on our previous work [ 17 , 18 ] using the e-prop learning rule [ 19 ] to efficiently train recurrent SNN classifiers, we combine the GeNN implementation of e-prop with the DEEP R structural plasticity rule [ 20 ] – implemented in our new framework – to train very sparse networks on the DVS gesture classification task [ 21 ] with no loss of accuracy while using 90\times fewer parameters than comparable recurrent dense SNN models. The time overhead of applying structural plasticity is less than 1\,\mathrm{\%} and the sparse models used are an order of magnitude faster to train than their dense counterparts. Secondly, we implement a model of topographic map formation [ 22 ] in GeNN, motivated by a recent example on the SpiNNaker platform [ 7 ] . The spatially embedded network employs an activity-independent formation rule: new connections are drawn according to a Gaussian dependence on the lateral distance between neurons. However, synapse elimination is activity-dependent: Spike-Timing-Dependent Plasticity (STDP) [ 2 ] varies synaptic strengths and weak synapses are more likely to be removed. The interplay of different plasticity mechanisms in the model results in a refinement of receptive fields and an embedding of learnt input