Working Memory in a Recurrent Spiking Neural Networks With Heterogeneous Synaptic Delays
Laurent U Perrinet · 2026
Read Working Memory in a Recurrent Spiking Neural Networks With Heterogeneous Synaptic Delays by Laurent U Perrinet (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14096.txt.
Introduction
Perrinet builds working memory into a recurrent spiking network by giving each synapse 41 heterogeneous delays. Target spike patterns become chains of overlapping spiking motifs, windows that predict the next spike. On a synthetic benchmark of 16 patterns over 512 neurons and 1000 steps, training reaches a perfect F1 score, with recall spreading forward from the clamped starting window. Heterogeneous delays thus offer an efficient substrate for neuromorphic edge memory.
Catalog record
| CREATOR | Laurent U Perrinet |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29489 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14096
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Working Memory in a Recurrent Spiking Neural Networks With Heterogeneous Synaptic Delays Laurent U Perrinet Abstract Working memory — the ability to store and recall precise temporal patterns of neural activity — remains an open challenge for spiking neural networks (SNNs). We propose a recurrent SNN of N N neurons in which each synapse is equipped with D = 41 D=41 delays, modelled as a weight tensor 𝐖 ∈ ℝ N × N × D \mathbf{W}\in\mathbb{R}^{N\times N\times D} and trained end-to-end with surrogate-gradient backpropagation through time. The network stores M M arbitrary target spike patterns by representing each as a sequential chain of overlapping Spiking Motifs: contiguous windows of length D D that uniquely predict spikes at the next time step. On a synthetic benchmark of M = 16 M=16 patterns ( N = 512 N=512 neurons, T = 1000 T=1000 steps), training achieves a mean F1 score of 1.0 1.0 , with recall emerging first near the clamped initialisation window and propagating forward in time. This result demonstrates that heterogeneous delays provide an efficient substrate for working memory in SNNs, enabling energy-efficient neuromorphic edge deployment. 1 Introduction Figure 1: Working memory as sequential spike prediction via heterogeneous delays. Left: A recurrent HD-SNN of five neurons numbered 1 – 5 , connected by two subsets of recurrent connections that define two Spiking Motifs from the presynaptic side (neurons a ) to the postsynaptic side (neurons b ): dark-red connections from a_{2} , a_{3} , a_{4} , and a_{5} target b_{1} with delays 4 , 9 , 6 , 3 ms; green connections from a_{1} , a_{2} , a_{3} , a_{4} target b_{5} with delays 1 , 6 , 3 , 8 ms. Right: Raster plot illustrating how the network predicts successive spikes from a past context window. Step 1 — first prediction (green box): the orange input spikes emitted at staggered times by a_{2} – a_{5} within the first context window (dashed green box) with precise interspike intervals. These are used to define a heterogeneous Spiking Motif such that the (green) delays of all four spikes converge synchronously at b_{5} , crossing the firing threshold and producing the green output spike. Step 2 — second prediction (dark-red box): the green spike of a_{5} re-enters the network as a new input and, together with the spikes now visible in the shifted context window (dashed dark-red box), is routed through the dark-red delays in order to converge synchronously on b_{1} , generating its output spike. Because output spikes re-enter the same population, detecting one Spiking Motif provides the context for predicting the next spike in the sequence, implementing working memory as a chain of temporal predictions grounded in the preceding context window. A century ago, 2 established that neural communication relies on all-or-none action potentials, spikes . Whether information is encoded in precise spike timing or in the coarser measure of instantaneous firing rate remains debated ( 21 ; 4 ; 10 ) — a question that becomes especially sharp for working memory, where the brain must maintain and manipulate information across delays of seconds. In traditional artificial neural networks, long-range temporal dependencies are addressed by gated recurrent unit (GRU) architectures or Transformers ( 23 ) , which decouple temporal binding range from the timescale of individual units. The analogous problem for spiking neural networks (SNNs) — binding activity patterns separated by intervals far exceeding a single neuron’s time constant — lacks a general solution and remains a key challenge for biologically plausible sequence processing ( 25 ) . One principled approach relies on the precise relative timing of spikes across a population. 1 was a precursor in proposing that cortical neurons act as coincidence detectors; 3 formalised this into synfire chains , feedforward pools in which synchronous volleys propagate reliably. Experimental support came from 5 , who showed that thalamocortical inputs are individually weak but effective through precise synchrony; theoretical work further showed such synchrony can be learned in an unsupervised manner ( 18 ) . 13 then directly observed recurring spatiotemporal spike sequences in neocortex at both sub- and supra-second timescales, and complementary hippocampal evidence shows that such sequences self-organise spontaneously as internal population templates ( 24 ) . 14 extended the synfire framework by introducing heterogeneous axonal delays with spike-timing-dependent plasticity (STDP). The resulting polychronous neuronal groups (PNGs) — time-locked, non-synchronous firing patterns at millisecond precision — vastly outnumber the neurons in the network, yielding a large short-term memory capacity exploited for visual feature binding ( 6 ) and sequence classification 17 . 22 showed that short-term plasticity can cue a PNG’s reactivation and sustain a working memory. However, storing spike patterns of arbitrary length is inherently difficult: the sequential structure of the task means each context window is not an independent query but is itself the output of the network at the previous step, so small errors at early time steps propagate and amplify towards the end of the sequence — a compounding difficulty absent from static associative memory. The limitation of the original model is that delays are fixed rather than learned. Recent work addresses this: 8 demonstrated fast, energy-efficient inference on neuromorphic hardware using first-spike-time coding, while 12 introduced gradient-based delay learning via dilated convolutions with learnable spacings (DCLS), achieving sparse solutions in which only \sim\!20\% of delay channels remain active. Applied to event-based vision, heterogeneous learned delays in a single spiking layer suffice for fast motion detection ( 11 ) , scalable to always-on object recognition ( 9 ) . Multiple learned delays per synapse give rise to Spiking Motifs (SMs) ( 19 ) : precisely timed spike patterns that generalise PNGs to a trainable, hardware-compatible framework, here referred to as Heterogeneous-Delay SNNs (HD-SNNs). 15 recently extended this to motifs of arbitrary length by chaining sub-motif detectors with bounded synaptic delays, achieving robust encoding of complete Spiking Heidelberg Digits patterns. The present work generalises this chain-of-motifs framework to enable working memory in a recurrent SNN. Rather than relying on fixed delays and unsupervised STDP rules as in the original polychronisation model, we extend the spike-timing theory of ( 15 ) to a fully recurrent topology: once primed by a short clamping window, the network autonomously recalls arbitrary-length spike sequences. The model is trained end-to-end with surrogate gradients ( 16 ) , bridging the computational neuroscience framework of polychronisation with gradient-based machine learning and targeting the energy-efficiency and sparsity objectives central to neuromorphic edge deployment. 2 Methods 2.1 Representation of spiking activity In neurobiological recordings or the output of event-based cameras, any generic raster plot consists of a stream of spikes , defined as a list of address–timestamp tuples \mathcal{E}=\bigl\{(a_{r},\,t_{r})\bigr\}_{r\in[1,\,N_{\mathrm{ev}}]}, where N_{\mathrm{ev}}\in\mathbb{N} is the total number of events and r indexes each event in temporal order ( 9 ) . Each event has a time of occurrence t_{r} and an address a_{r} in the neural population space \mathcal{A} . In a Single Unit Activity (SUA) recording for instance, \mathcal{A} is the identified set of neurons. Neurons are characterised by their membrane potential dynamics and by the integration of synaptic potentials along their dendritic tree. Crucially, neurons can receive inputs from multiple presynaptic neurons with heterogeneous delays \delta_{s} , representing the travel time of the presynaptic spike