Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks

Todd Morrill, Christian Pehle, Anthony Zador · 2026

Read Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks by Todd Morrill, Christian Pehle, Anthony Zador (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13283.txt.

Introduction

Todd Morrill, Christian Pehle, and Anthony Zador present Bullet Trains for training spiking neural networks. Event-native networks treat spike timing and order as the representation itself. Exact charge-fire-reset dynamics force sequential processing of input spikes, which slows training. Precise spike times must also be found without time bins. Parallel associative scans consume many spikes at once, with speedups up to 44x over sequential simulation. Differentiable solvers then compute spike times to machine precision, and tests on four event-based datasets ran on GPUs.

Catalog record

CREATORTodd Morrill, Christian Pehle, Anthony Zador
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82924
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13283

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks Todd Morrill, Christian Pehle, Anthony Zador Abstract Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge–fire–reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 44x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike-time solvers that compute spike times to machine precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based datasets on GPUs. 1 Introduction A commonly cited advantage of spiking neural networks (SNNs) is their ability to operate in an event-based manner, processing information only when spikes occur ( Merolla et al. 2014 ; Davies et al. 2018 ) . On appropriate neuromorphic hardware, this implies energy efficiency, as computation only occurs when spikes are present ( Roy et al. 2019 ; Yao et al. 2024 ) . At the same time, much SNN research still happens on readily available hardware such as graphical processing units (GPUs) so accelerating this workload (e.g., via parallelization) and mirroring properties of biology and event-based neuromorphic hardware (e.g., precise spike times) ( Carr & Konishi 1990 ; Thorpe & Gautrais 1998 ; Amir et al. 2017 ; Richter et al. 2024 ) is an important research direction. In this work, we present a strictly event-based SNN system that processes multiple spike events in parallel, achieving significant speedups over sequential processing. Second, we present differentiable spike-time solvers that enable machine-precision spike timing. These two contributions allow us to take a step toward event-native spiking computation, where preserving continuous spike times is the objective—both for biological fidelity and for compatibility with neuromorphic sensing and hardware—and where efficiency follows from operating directly on events. Figure 1 : Our parallel method achieves up to 44x speedups over serial processing of spike events on several event-based datasets while retaining exact hard-reset dynamics. The plot shows results for the Spiking Heidelberg Digits (SHD) dataset. Figure 2 : Our contributions visualized—state trajectory of a LIF neuron consuming input spikes in parallel and producing an output spike with our Newton-Raphson solver. A shows the neuron jumping from the initial state \mathbf{s}_{0} at time 0 (composed of membrane potential and synaptic current) directly to all future states \mathbf{s}_{1}=f_{1}(\mathbf{s}_{0}),\mathbf{s}_{2}=f_{2}(\mathbf{s}_{0}),\ldots in parallel using associative scans. We use lightweight analytical checks to determine if an output spike will occur in the interval between any consecutive pair of input spikes by computing—in parallel— t_{V_{\max}}(\mathbf{s}) , the time of maximum voltage starting at the interval’s left endpoint, and V(t_{V_{\max}}) the voltage at that time, namely the maximum voltage value. The voltage state is hard reset at the spike time. B shows our iterative Newton-Raphson solver finding the output spike time t^{\star} , computed to machine precision. SNNs are challenging to parallelize along the time dimension due to the sequential nature of “charge–fire–reset” dynamics. After consuming each and every input spike, a neuron must determine whether it will produce a spike itself before the next input spike arrives. Performed naively, SNNs must process one spike at a time, leading to significant computational bottlenecks on modern parallel hardware such as graphical processing units (GPU) ( Shrestha & Orchard 2018 ; Engelken 2023 ; Fang et al. 2023 ; Li et al. 2024 ; Zhong et al. 2024 ; Feng et al. 2025 ) . In the past few years, many methods have been proposed to parallelize spiking dynamics, but they all modify the neuron model in some way, such as removing resets entirely, which can reduce the non-linear expressivity of the neuron and/or deviate from the way biological neurons operate. In this work, we propose a method for parallel training of exact hard-reset spiking dynamics using parallel associative scans. The key idea is to process input spike trains in chunks, where each chunk is processed in parallel using associative scans. Crucially, this is spike-aware chunking: lightweight analytical checks quickly determine whether a neuron spikes within a chunk and, if so, locate the first such spike, allowing the simulation to jump directly to that time. The next chunk resumes processing input spikes immediately after the output spike. In practice, even if a portion of work is discarded, the overall speedup relative to sequential processing is significant because of the parallelism afforded by modern hardware. The result is a parallel system that achieves up to 44x speedups over sequential processing on several event-based datasets while retaining exact hard-reset dynamics. A second challenge that arises in strictly event-based SNNs is determining spike times precisely without the use of discrete-time bins. Nearly all SNN implementations to date operate on discrete-time grids ( Bauer et al. 2023 ; Hammouamri et al. 2024 ; Nowotny et al. 2025 ) , which means that neurons are performing computation at every time step, regardless of whether spikes are present. The result of this modeling choice is computation—and often memory usage—that scales with the number of time steps rather than the number of spikes, severely limiting the number of time steps that can be simulated. Temporal resolution is limited by the time-step size and spike ordering within a time bin cannot be determined ( Bauer et al. 2023 ) (see Figure 3 ). In contrast, several systems have proposed analytical solutions for continuous spike times, but these methods often impose restrictions on the neuron model, such as the relationship between synaptic and membrane time constants in leaky-integrate-and-fire (LIF) neurons ( Mostafa 2018 ; Göltz et al. 2021 ; Engelken 2023 ) . Here, we apply several methods for differentiable spike-time solvers—namely Newton-Raphson and Bisection numerical root solvers—which resolve spike times to machine precision without the need for discrete-time approximations. This removes a constraint on SNN systems (e.g., to support heterogeneous time constants ( Perez-Nieves et al. 2021 ) ), facilitates the exploration of neuro-inspired theories of time-based computation ( Yang & Zador 2012 ; Xie et al. 2024 ) , matches the temporal-resolution of event-based sensors and neuromorphic hardware ( Müller et al. 2024 ) , and enables compute and memory to scale in proportion to the number of spikes. 2 Related Work Parallelizing Spiking Dynamics. Biological neurons go through a “charge–fire–reset” cycle and in the “reset” stage, the neuron’s membrane potential is set to a baseline value ( Hodgkin & Huxley 1952 ) . It has been argued that this hard reset is essential for implementing precise feature detection ( Berry & Meister 1998 ) . The sequential dependency of the “charge–fire–reset” cycle in simulated Leaky Integrate-and-Fire (LIF) neurons presents a fundamental barrier to parallel training on modern hardware. While sub-threshold dynamics are linear and can be parallelized using associative scans—similar to state-space models ( Schöne et al. 2025 ) —the

Read the complete text