Equivalence of approximation by networks of single- and multi-spike neurons
Dominik Dold, Philipp Petersen · 2026
Read Equivalence of approximation by networks of single- and multi-spike neurons by Dominik Dold, Philipp Petersen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13478.txt.
Introduction
Dominik Dold and Philipp Christian Petersen ask whether one spike per neuron is enough in spiking neural networks. Approximation bounds to date hold only for neurons that spike at most once, seen as a strong limit. They prove the opposite holds for a large class of models, including leaky integrate-and-fire with subtractive reset. Any bound for multi-spike networks has an equivalent single-spike version with only linearly more or fewer neurons. The reverse holds too, so the two classes are equivalent for approximation in general tasks.
Catalog record
| CREATOR | Dominik Dold, Philipp Petersen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28726 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13478
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Equivalence of approximation by networks of single- and multi-spike neurons Dominik Dold, Philipp Petersen Abstract In a spiking neural network, is it enough for each neuron to spike at most once? In recent work, approximation bounds for spiking neural networks have been derived, quantifying how well they can fit target functions. However, these results are only valid for neurons that spike at most once, which is commonly thought to be a strong limitation. Here, we show that the opposite is true for a large class of spiking neuron models, including the commonly used leaky integrate-and-fire model with subtractive reset: for every approximation bound that is valid for a set of multi-spike neural networks, there is an equivalent set of single-spike neural networks with only linearly more neurons (in the maximum number of spikes) for which the bound holds. The same is true for the reverse direction too, showing that regarding their approximation capabilities in general machine learning tasks, single-spike and multi-spike neural networks are equivalent. Consequently, many approximation results in the literature for single-spike neural networks also hold for the multi-spike case. 1 Introduction Is it sufficient for spiking neurons to spike at most once, or is there any benefit of spiking multiple times? How to best encode information in the temporal domain of spikes is still an open question. The possibilities range from encoding via exact spike times, such as in the latency of a neuron’s spike response given a stimuli onset [ 27 , 8 ] , to averaged quantities such as spike time rates [ 7 ] . Although using only single spikes might seem restrictive at first, it comes with several benefits. First, for exact spike times, analytical gradient-based learning rules are available for specific neuron models [ 13 , 17 , 2 , 9 , 14 , 24 ] , as well as learning of synaptic delays [ 10 ] in addition to synaptic weights. Second, decoding outputs using lowest latency leads to a low time-to-solution, while the inherent sparsity of these spiking neural networks (SNNs) promises high energy efficiency [ 9 , 25 ] . This makes single-spike SNNs quite attractive for edge applications, as recently proposed in the context of artificial intelligence (AI) onboard spacecraft [ 15 , 12 ] . For most practical purposes though, SNNs consisting of leaky integrate-and-fire (LIF) neurons (or variations thereof) with an unconstrained number of spikes per neuron are used nowadays. These SNNs are then commonly trained using the surrogate gradient method [ 19 , 5 , 21 ] . In fact, a common critique is that neurons which spike at most once restrict the expressivity of SNNs [ 26 , 28 , 5 ] and do not reflect biology well. This raises the question whether theoretical results derived for single-spike neurons apply to the more significant and relevant case of multi-spike neurons. This work clarifies this open question, and we show a simple but extremely consequential result: both single-spike and multi-spike SNNs are equivalent when it comes to their approximation capabilities. Our result is inspired by recent work on the equivalence in approximation of fully-connected and convolutional neural networks (NNs) [ 22 ] . A formal statement of the result is given in Theorem 1 , which is roughly summarised as follows: 1. All upper and lower bounds concerning approximation rates of multi-spike SNNs with n\in\mathbb{N} neurons that spike at most N_{\text{s}}{}\in\mathbb{N} times translate to the same bounds concerning approximation rates of single-spike SNNs with at most N_{\text{s}}{}\cdot n neurons. 2. All upper and lower bounds concerning approximation rates of single-spike SNNs translate to the same bounds concerning approximation rates of multi-spike SNNs with at most \alpha N_{\text{s}}{}\cdot n neurons that spike at most N_{\text{s}}{} times. The prefactor satisfies \alpha\leq\text{min}\left(1,\frac{6}{\pi^{2}}+1/\sqrt{N_{\text{s}}{}}\right) for N_{\text{s}}{}\geq 1 and \alpha<\frac{6}{\pi^{2}}+1/(2\sqrt{N_{\text{s}}{}}) for N_{\text{s}}{}\geq 8 . This transference principle holds as long as the single-spike neurons behave dynamically like multi-spike neurons that stop spiking after the first spike. The first part can be seen with the following direct argument: first, we replace each multi-spike neuron by N_{\text{s}}{} single-spike neurons, one per spike. Second, to ensure that the single-spike neurons spike at the correct time, their threshold is adjusted accordingly ( Fig. 1 A,B), and we are done. The central idea for showing the second statement is to combine the output of a small population of multi-spike neurons in such a way that all except one spike cancel out, allowing the population to mimic single-spike neurons ( Fig. 1 C,D). This result holds for a large family of spiking neuron models, including the widely used LIF neuron model with and without synaptic delays. However, both the single-spike and multi-spike NNs have to be based on the same neuron model for equivalence to hold. The result has far-reaching consequences, as every achievable approximation rate derived for single-spike SNNs automatically applies to multi-spike SNNs, and vice versa. Thus, our result directly strengthens recently reported approximation bounds for SNNs, such as covering-number based upper learning bounds derived for a simple spike-response model [ 20 ] , and lower bounds of the approximation error for non-leaky integrate-and-fire neurons (nLIF) derived using a piecewise decomposition of SNNs [ 4 ] . Moreover, in [ 24 , 25 ] , it was shown that for a simple spike-response model with linearly rising postsynaptic potential, an exact mapping between ReLU NNs and SNNs exists, i.e., where both NNs produce the exact same output for every input sample. Applying our result to this work, it follows that for each ReLU NN, there exists a multi-spike SNN (with the aforementioned neuron model) that satisfies the same approximation bounds. Generalisation error bounds for ReLU NNs are well known and can, for instance, be derived using covering numbers or Rademacher complexity, while interpolation error bounds can be derived via the number of affine pieces [ 23 ] . In the following, we first provide an overview of all assumptions for deriving Theorem 1 . We then continue to stating our main theorem, proving it, and briefly discussing the main limitations of our result for completeness. Figure 1: Illustration of how to construct equivalent single-and multi-spike SNNs, shown here for multi-spike neurons that spike at most six times in a given finite time interval. A. A multi-spike neuron can be replaced by a population of single-spike neurons that produce an equivalent spike output. B. The k^{\text{th}} single-spike neuron spikes at the k^{\text{th}} spike time of the multi-spike neuron. This is achieved by setting the threshold of each neuron accordingly (alternatively, the reset potential could be adjusted). C. A single-spike neuron can also be replaced by a population of multi-spike neurons. D. Given any input, the first neuron produces a spike train. We add additional neurons to the population that gain identical input, but have increasing thresholds. For instance, the second neuron only spikes at every second spike time of the first neuron. By choosing synaptic weights accordingly, all but the first spike of the first neuron are cancelled out, producing an effective spike train with only one spike. 2 Background 2.1 Learning task First, we introduce the set of all spike trains consisting of at most N\in\mathbb{N} spikes: S_{N}\coloneqq\bigcup_{k=1}^{N}\left\{(t_{1},\dots,t_{k})\in[0,T]^{k}:t_{1}\leq\dots\leq t_{k}\right\}\cup\{()\}\ \ \ \ \text{with}\ \ \