Hardware-Aware Fine-Tuning of Spiking Q-Networks on the SpiNNaker2 Neuromorphic Platform
Sirine Arfa 1 2 , Bernhard Vogginger 1 , Christian Mayr 1 6 4 · 2025
Read Hardware-Aware Fine-Tuning of Spiking Q-Networks on the SpiNNaker2 Neuromorphic Platform by Sirine Arfa 1 2 , Bernhard Vogginger 1 , Christian Mayr 1 6 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23562.txt.
Introduction
Sirine Arfa, Bernhard Vogginger, and Christian Mayr showed in 2025 how to fine-tune spiking neural networks directly for a brain-like chip. Spiking networks promise far lower power use on special hardware, a draw for robots. The team trained a learning agent with a standard method, then shrank the network to eight-bit numbers and tuned it for the SpiNNaker2 chip.
On two classic control tasks, the chip matched a graphics card for speed while using up to thirty-two times less energy per answer. The results make a strong case for brain-like chips in small machines that must think on a tight power budget.
Catalog record
| CREATOR | Sirine Arfa 1 2 , Bernhard Vogginger 1 , Christian Mayr 1 6 4 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-23562
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hardware-Aware Fine-Tuning of Spiking Q-Networks on the SpiNNaker2 Neuromorphic Platform Sirine Arfa 1 2 , Bernhard Vogginger 1 , Christian Mayr 1 6 4 Abstract Spiking Neural Networks (SNNs) promise orders-of-magnitude lower power consumption and low-latency inference on neuromorphic hardware for a wide range of robotic tasks. In this work, we present an energy-efficient implementation of a reinforcement learning (RL) algorithm using quantized SNNs to solve two classical control tasks. The network is trained using the Q-learning algorithm, then fine-tuned and quantized to low-bit (8-bit) precision for embedded deployment on the SpiNNaker2 neuromorphic chip. To evaluate the comparative advantage of SpiNNaker2 over conventional computing platforms, we analyze inference latency, dynamic power consumption, and energy cost per inference for our SNN models, comparing performance against a GTX 1650 GPU baseline. Our results demonstrate SpiNNaker2’s strong potential for scalable, low-energy neuromorphic computing, achieving up to 32 × \times reduction in energy consumption. Inference latency remains on par with GPU-based execution, with improvements observed in certain task settings, reinforcing SpiNNaker2’s viability for real-time neuromorphic control, making the neuromorphic approach a compelling direction for efficient deep Q-learning. I Introduction Spiking neural networks (SNNs) have emerged as a promising computational model that combines biological inspiration with practical engineering advantages most notably, sparse activation, event-driven processing, and ultra-low power consumption [ 1 ] . Unlike conventional artificial neural networks (ANNs), which rely on continuous-valued activations and synchronous computation, SNNs process information through discrete spikes, enabling asynchronous updates [ 2 ] and significantly reducing computational load. These properties make SNNs an attractive candidate for energy-constrained and latency-sensitive applications [ 3 ] , especially when paired with specialized neuromorphic hardware platforms designed to exploit their unique characteristics [ 4 , 5 ] . As the demand for on-chip intelligence continues to grow, neuromorphic systems such as SpiNNaker2 [ 6 ] have been developed to support scalable and power-efficient execution of SNNs. By embracing asynchronous, parallel architectures that align with the sparse, temporal nature of spiking computation, these platforms offer a powerful path toward embedded aritificial intelligence (AI) capable of real-time inference without relying on large, power-hungry accelerators [ 7 , 8 ] . While supervised learning with SNNs has seen significant progress thanks in large part to surrogate gradient training, applications in RL remain relatively underexplored. RL is a natural match for SNNs [ 9 ] : both are inherently temporal processes that unfold over time [ 10 ] , and the internal state carried by spiking neurons (e.g., membrane potential) offers a lightweight, biologically inspired form of short-term memory. Despite this theoretical alignment, most prior approaches either focus on shallow, biologically motivated algorithms such as reward-modulated spike-timing dependent plasticity (STDP) [ 11 ] , or rely on ANN-to-SNN conversion techniques [ 12 ] that neglect the dynamic structure of spike-based learning during training. While RL with SNNs has shown promise in recent studies [ 13 , 14 ] , practical deployment on embedded neuromorphic platforms remains a complex challenge due to the tight coupling between algorithmic design and hardware constraints. Recent experimental results have demonstrated that neuromorphic reinforcement learning can deliver substantial efficiency gains across a range of application domains. In robotic control, spiking policies deployed on Intel Loihi achieved real-time performance in force-sensitive manipulation tasks with up to an order-of-magnitude reduction in energy consumption compared to conventional platforms [ 15 ] . Similarly, population-coded actor-critic architectures running on neuromorphic chips have been shown to match the performance of standard deep RL methods while consuming over 100× less energy in continuous control settings [ 16 ] . These advantages stem from the close alignment between neuromorphic hardware and the sparse, temporally structured nature of spike-based computation, enabling event-driven updates and efficient state encoding. As a result, RL on neuromorphic platforms holds particular promise for power-constrained, latency-critical applications such as autonomous robotics, edge AI, and closed-loop control in real-world environments. In this work, we build upon the model architecture proposed in [ 17 ] , used SnnTorch [ 18 ] for training the rate-coded SNN models, fine-tuned them for energy-efficient deployment on the SpiNNaker2 neuromorphic platform. Our pipeline involves training spiking agents using the Q-learning algorithm [ 19 ] in simulation, followed by quantization to 8-bit precision to meet the memory and compute constraints of embedded neuromorphic execution. This approach enables a direct evaluation of the real-world viability of deploying deep spiking RL agents on hardware optimized for low-power operation, while preserving task performance. This workflow addresses both algorithmic and system-level challenges: enabling deep RL in a spike-based network while ensuring compatibility with energy-efficient hardware execution. The overall system architecture used in our experiments is illustrated in Figure 1 . We validate our approach on two classical control benchmarks from the OpenAI Gym [ 20 ] suite, focusing not only on cumulative reward but also on practical deployment metrics including dynamic power, latency, and energy per inference. Compared to a conventional GPU baseline (GTX 1650), our system achieves up to a 96% reduction in energy usage and a 32% reduction in inference latency, while maintaining competitive task performance. These results demonstrate the viability of neuromorphic hardware as a platform for efficient deep RL and offer a concrete step toward deploying intelligent, learning-enabled agents in constrained environments such as mobile robots, IoT devices, and autonomous vehicles [ 21 , 16 ] , [ 22 ] . Fig. 1 : Pipeline Overview: Closed-loop SNN-based RL with SpiNNaker2. Observations from the OpenAI Gym environments are processed by a quantized spiking neural network deployed on the SpiNNaker2 neuromorphic platform. The system runs in closed-loop, with selected actions sent back to the environment. The SNN architecture includes two fully connected hidden layers; additional details are provided in Table II . II Background II-A Deep Q-Learning Q-learning is a model-free, off-policy RL algorithm [ 23 ] that estimates the optimal action-value function Q^{*}(s,a) , representing the expected return from taking action a in state s and following the optimal policy thereafter: Q^{*}(s,a)=\max_{\pi}\mathbb{E}\left[\sum_{t=0}^{\infty}\gamma^{t}r_{t}\,\bigg|\,s_{0}=s,\,a_{0}=a,\,\pi\right], (1) where \gamma\in[0,1) is the discount factor determining the importance of future rewards, and r_{t} is the reward at time step t . Q-values are updated based on observed transitions (s,a,r,s^{\prime}) using the Bellman equation: Q(s,a)\leftarrow Q(s,a)+\alpha\left[r+\gamma\max_{a^{\prime}}Q(s^{\prime},a^{\prime})-Q(s,a)\right], (2) where \alpha is the learning rate, s^{\prime} denotes the next state resulting from action a , and a^{\prime} denotes the action selected in s^{\prime} . In high-dimensional environments, Deep Q-Learning approximates Q(s,a) with a neural network Q(s,a;\theta) , where \theta are the parameters of the network. To improve training stability, a separate target network with parameters \theta^{-} is maintained; these parameters are periodically copied from \theta and held fixed between updates. Learning proceeds by storing transitions (s,a,r,s^{\prime}) in a replay memory buffer \mathcal{D} . During optimization, minibatches are sampled uniformly at random from \mathcal{D} , and the parameters \theta are updated to minimize the temporal difference loss: \mathcal{L}(\theta)=\left(r+\gamma\max_{a^{\prime}}Q(s^{\prime},a^{\prime};\theta^{-})-Q(s,a;\theta)\right)^{2}. (3) This approach enables the agent to learn effective policies directly from high-dimensional inputs and generalize across diverse tasks. II-B The CartPole and Acrobot Tasks II-B 1 CartPole The CartPole environment [ 24 ] involves a pole hinged to a cart that moves along a horizontal track. The system state is given by a 4-dimensional vector s\in\mathbb{R}^{4} , comprising the cart’s position and velocity, and the