Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning

Irene Ambrosini 1,2 , Ingo Blakowski 1,3 , Dmitrii Zendrikov 1 , Cristiano Capone 4 , Luna Gava 2 , Giacomo Indiveri 1 , Chiara De Luca 1,5,∗ , Chiara Bartolozzi 2,∗ · 2026

Read Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning by Irene Ambrosini 1,2 , Ingo Blakowski 1,3 , Dmitrii Zendrikov 1 , Cristiano Capone 4 , Luna Gava 2 , Giacomo Indiveri 1 , Chiara De Luca 1,5,∗ , Chiara Bartolozzi 2,∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21548.txt.

Introduction

Irene Ambrosini and seven co-authors train silicon neurons to play air hockey, a game that punishes slow reactions. The puck moves fast, so choices must land in a split second. Their system uses a compact network of spiking neurons on a mixed analog-digital chip.

Hardware and learning rules were designed together from the start. Training by trial and error reaches solid puck play in very few trials. Few-trial learning matters because real robots wear out and cannot practice forever. The demo shows brain-like chips handling a task where speed, not size, is the whole challenge.

Catalog record

CREATORIrene Ambrosini 1,2 , Ingo Blakowski 1,3 , Dmitrii Zendrikov 1 , Cristiano Capone 4 , Luna Gava 2 , Giacomo Indiveri 1 , Chiara De Luca 1,5,∗ , Chiara Bartolozzi 2,∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21548

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning Irene Ambrosini 1,2 , Ingo Blakowski 1,3 , Dmitrii Zendrikov 1 , Cristiano Capone 4 , Luna Gava 2 , Giacomo Indiveri 1 , Chiara De Luca 1,5,∗ , Chiara Bartolozzi 2,∗ Abstract Air hockey demands split-second decisions at high puck velocities, a challenge we address with a compact network of spiking neurons running on a mixed-signal analog/digital neuromorphic processor. By co-designing hardware and learning algorithms, we train the system to achieve successful puck interactions through reinforcement learning in a remarkably small number of trials. The network leverages fixed random connectivity to capture the task’s temporal structure and adopts a local e-prop learning rule in the readout layer to exploit event-driven activity for fast and efficient learning. The result is real-time learning with a setup comprising a computer and the neuromorphic chip in-the-loop, enabling practical training of spiking neural networks for robotic autonomous systems. This work bridges neuroscience-inspired hardware with real-world robotic control, showing that brain-inspired approaches can tackle fast-paced interaction tasks while supporting always-on learning in intelligent machines. 1 Institute of Neuroinformatics, UZH and ETH Zurich, Switzerland 2 Istituto Italiano di Tecnologia, Genoa, Italy 3 Technical University of Munich, Munich, Germany 4 Natl. Center for Radiation Protection and Computational Physics, Istituto Superiore di Sanità, 00161 Rome, Italy 5 Digital Society Initiative, University of Zurich, Zurich, Switzerland ∗ These authors contributed equally I Introduction Fig. 1: Control pipeline and environment. Top-left: High-level flow from MuJoCo (puck [x_{p},y_{p},v_{x},v_{y}] and end-effector [x_{ee},y_{ee}] ) through the decision module to the robot controller. The CPU encodes sensory data into spike trains, processed by DYNAP-SE’ silicon neurons, then decoded into discrete motion primitives and translated to joint commands [q_{1},q_{2},q_{3}] . Bottom-left: Encoding module, reservoir, and spike-based readout with learning. Right: Table dimensions ( 1.038\text{\,}\mathrm{m} \times 1.948\text{\,}\mathrm{m} ), puck/mallet radii, puck generation region, and two arrival points for Action 0/1 in the robot frame. The AI revolution has achieved unprecedented success by scaling neural networks to billions of parameters. Yet this progress comes at an unsustainable cost: training large models can consume megawatt-hours of energy [ 23 ] , while even inference on edge devices drains batteries in hours [ 8 ] . This power-hungry paradigm conflicts with the demands of autonomous robotics, where mobile platforms must learn and adapt continuously under severe energy constraints, often with budgets measured in milliwatts, not watts [ 11 ] .Nature offers a compelling alternative. The human brain performs remarkable motor learning with roughly 20 W of power and surprisingly few training examples [ 25 ] . This efficiency gap has motivated both algorithmic innovations in sample-efficient learning and hardware advances in neuromorphic computing. On the algorithmic front, model-based reinforcement learning has demonstrated the ability to learn complex behaviors from limited data by building predictive world models [ 14 ] . For instance, MuZero achieved superhuman performance in Atari, Go, Chess, and Shogi by planning with learned models [ 26 ] , while EfficientZero mastered Atari using only two hours of gameplay— 100\times less data than standard approaches [ 27 ] . World models have also shown robustness to model inaccuracies, while universal successor features [ 6 ] enable generalization to novel tasks without retraining. On the hardware front, brain-inspired neuromorphic processors process information through asynchronous spikes rather than clock cycles, achieving 1000\times energy reductions for specific tasks [ 22 ] . Recent chips such as Intel’s Loihi [ 10 ] , IBM’s TrueNorth [ 2 ] , and DYNAP-SE [ 20 ] have reached maturity that enables deployment in proof-of-concept robotic setups. A key step has been the introduction of always-on learning in closed-loop setups, enabling reinforcement learning directly in the spiking domain [ 16 ] . Capone et al. [ 9 ] demonstrated biologically plausible “awake” and “dreaming” RL phases on Atari Pong, achieving competitive performance. This was later extended to real-time hardware on DYNAP-SE [ 5 ] . At the core of these advances are spiking neural networks (SNNs), which model neurons as leaky integrate-and-fire units that communicate through discrete spikes—a key mechanism for energy efficiency and temporal coding [ 15 ] . Deep RL algorithms like DQN and TD3 have been implemented in SNNs [ 24 , 1 ] , but existing training methods often rely on non-local learning rules, making them computationally expensive and biologically implausible. Recent advances in local plasticity [ 17 , 13 ] enable online learning in recurrent SNNs suitable for neuromorphic hardware. However, neuromorphic RL demonstrations remain confined to simplified benchmarks—mainly 2D games with low-dimensional observations, discrete actions, and forgiving timing. Whether such systems can scale to higher-dimensional states, continuous-time control, and tight temporal constraints typical of real-world robotics remains an open question. This paper provides the first affirmative answer by implementing neuromorphic RL for robotic air hockey—a canonical fast-paced manipulation benchmark requiring predictive control under tight temporal constraints. Air hockey presents key challenges absent from game benchmarks: (1) Higher dimensionality : 6D continuous state space including puck position, velocity, and striker coordinates across a 1.038\times 1.948 m workspace; (2) Physical constraints : precise end-effector control with mechanical latencies; (3) Temporal dynamics : puck trajectories at 1.0–1.5 m/s demand predictive planning with 50 Hz control and 1 ms physics integration.Our work complements efforts in neuromorphic robotics addressing other layers of autonomy. Romero et al. [ 21 ] demonstrated event-based vision and spiking CNNs on SpiNNaker for a 5-bar planar manipulator on a mini air-hockey table. While their approach targets fast sensorimotor integration for tracking, ours tackles the complementary problem of learning control policies through interaction . Instead of relying on mirroring strategies, we employ neuromorphic RL to adapt behavior online. Our platform uses an anthropomorphic arm on a standard air-hockey table (Figure 1 ), introducing larger workspaces and higher kinematic complexity. Together, these contributions point toward neuromorphic control architectures coupling efficient event-driven perception with adaptive decision-making. I-A Our approach and contribution We extend the neuromorphic RL framework of Blakowski et al. [ 5 ] , originally demonstrated on Atari Pong, to the far more challenging domain of physical robot manipulation. Our contributions address three key scalability challenges: • Beyond toy RL : We move from simple grid-based games [ 7 ] to real-world air hockey with adaptive-precision continuous state encoding, bridging the gap between toy problems and embodied control. • From discrete to continuous control : We demonstrate neuromorphic RL for continuous motion primitives executing ballistic trajectories at 50 Hz, requiring predictive rather than frame-level decisions. • Generalization under uncertainty : By randomizing puck positions and velocities (1.0–1.5 m/s), we achieve 96–98% success over 2000 episodes, showing robust learning and adaptation. Our system achieves robust gameplay, demonstrating that neuromorphic RL can scale from gaming benchmarks to physical robotic platforms with meter-scale workspaces and millisecond-precision constraints. II Methods Our approach integrates hardware-in-the-loop (HIL) learning with a simulated robotic task. Specifically, we utilize the DYNAP-SE analog-digital mixed-signal neuromorphic chip [ 19 ] for closed-loop, low-latency inference at every control time step, enabling real-time interaction with the environment. This methodology allows for the rigorous testing of the neuromorphic hardware’s capabilities in a dynamic, closed-loop robotic control application. II-A Experimental Setup Experiments are conducted in the MuJoCo implementation of the Air Hockey

Read the complete text