Elastic Spiking Transformers for Efficient Gesture Understanding
Alberto Ancilotto, Gianluca Amprimo, Stefano Di Carlo, and Elisabetta Farella Fondazione Bruno Kessler Politecnico di Torino · 2026
Read Elastic Spiking Transformers for Efficient Gesture Understanding by Alberto Ancilotto, Gianluca Amprimo, Stefano Di Carlo, and Elisabetta Farella Fondazione Bruno Kessler Politecnico di Torino (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13869.txt.
Introduction
Alberto Ancilotto and three colleagues introduce the Elastic Spiking shifter, a runtime-flexing design bringing stretch into the spiking field. Current designs are rigid static networks with fixed setting counts and compute graphs. Spurred by doll-style coding learning, it embeds nested stretch in the Feature puller, Spiking Self-focus and Feed-Forward blocks. One Elastic Spiking shifter spans a broad range of tangle-hits swaps on CIFAR10/100, CIFAR10-DVS and the EHWGesture clinic gesture data.
Catalog record
| CREATOR | Alberto Ancilotto, Gianluca Amprimo, Stefano Di Carlo, and Elisabetta Farella Fondazione Bruno Kessler Politecnico di Torino |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34855 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13869
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Elastic Spiking Transformers for Efficient Gesture Understanding Alberto Ancilotto, Gianluca Amprimo, Stefano Di Carlo, and Elisabetta Farella Fondazione Bruno Kessler Politecnico di Torino Abstract Spiking Neural Networks (SNNs), and particularly Spiking Transformers, have recently emerged as a promising solution for energy-efficient processing of event-based sensor data in healthcare applications. However, existing architectures are inherently rigid, being trained and deployed as static networks with fixed parameter counts and computational graphs. This rigidity presents a critical bottleneck for deployment on neuromorphic hardware (e.g., Loihi, Spinnaker), where on-chip hardware limitations often force the use of smaller, lower-accuracy models that sacrifice representational capacity and accuracy. To bridge this gap, we introduce the Elastic Spiking Transformer, a novel architecture that embeds runtime adaptability directly into the spiking paradigm. Drawing inspiration from Matryoshka-style representation learning, our framework introduces nested elasticity within the Feature Extractor, Spiking Self-Attention, and Feed-Forward blocks. By training a single “universal” model with granularity-aware weight sharing, we enable dynamic slicing of network width and attention heads during inference without retraining. This elasticity offers a unique dual advantage in the SNN domain. First, it solves the memory-constrained problem by allowing the model to instantly scale its parameter footprint to fit diverse hardware specifications without retraining. Second, and crucially, we show that elasticity in SNNs not only lowers synaptic weight counts but also directly lowers the total spike firing rate. By dynamically reducing the number of active neurons, our approach achieves a proportional reduction in synaptic operations, unavailable to standard artificial neural networks. We validate our approach on standard benchmarks (CIFAR10/100, CIFAR10-DVS) and the novel EHWGesture clinical gesture understanding dataset. Our results demonstrate that a single Elastic Spiking Transformer spans a wide range of complexity trade-offs, matching or surpassing the accuracy of independently trained baselines while enabling adaptive, real-time gesture recognition on resource-constrained edge devices. I INTRODUCTION Gesture recognition and understanding are critical tasks for next-generation human–computer interaction, offering a wide range of applications, from clinical rehabilitation monitoring [ 20 ] and surgical assistance [ 27 ] to sign language interpretation [ 25 ] and hands-free device control [ 30 ] . Deployment on always-on, low-power edge devices allows continuous gesture monitoring, enabling real-time interaction while preserving user privacy and reducing communication latency [ 1 , 19 ] . However, deployment on such devices faces a fundamental challenge, as their strict power budgets and capabilities limit the computational complexity required for accurate gesture understanding. Fig. 1: Proposed NESTformer modules. Event cameras offer a promising sensing modality for gesture recognition on such low-resource platforms. Unlike traditional frame-based cameras, these sensors only encode motion, producing a sparse representation that naturally aligns with Spiking Neural Networks (SNNs) based processing. SNNs enable computation only when spikes arrive, reducing costly multiply-accumulate (MAC) operations to simple accumulations. On neuromorphic hardware platforms such as Intel Loihi [ 5 ] and IBM TrueNorth [ 22 , 21 ] , this paradigm achieves energy consumption as low as 0.9-26 pJ per synaptic operation, enabling always-on gesture recognition on severely power-constrained devices. Recent advances in SNNs have brought attention-based architectures to the neuromorphic domain [ 35 , 31 , 34 ] , achieving strong performance on gesture benchmarks. However, despite recent progress, current spiking transformers still have architectural limitations that limit their effective use on edge devices. First, existing models are structurally rigid; these architectures have fixed sizes, meaning moving from one neuromorphic chip to another requires scaling and training a completely new version. Secondly, these architectures still often rely on CPU processing. For example, many of these architectures rely on matrix multiplication, which is generally incompatible with the spike-driven processing of true neuromorphic chips. As these chips do not handle such global computations natively, the workload must be offloaded to a higher-power CPU, limiting the actual energy savings in real life. To address these two limitations, we introduce the Neuromorphic Elastic Spiking Transformer (NESTformer) architecture. Inspired by recent advancements in elastic transformer architectures [ 7 ] , NESTformer was developed to allow elasticity to spiking transformer architectures. Our framework is based on three novel architectural blocks: 1. Elastic Spiking MLP inspired by Matryoshka-style representation learning [ 15 ] , uses a variable hidden dimension, which allows post-train tuning of the model operations and parameter count. 2. Elastic Spiking Attention: A novel approach to spiking self-attention [ 35 ] that allows variable attention head count while also ensuring compatibility when mapping to neuromorphic hardware. 3. Elastic Feature Extraction: A novel low-resource architecture allows for post-train tuning of the final network energy usage. This elasticity offers a dual advantage for gesture understanding, unique in the SNN domain. First, it solves the deployment flexibility problem by allowing the model to instantly scale its parameter footprint to fit devices with significantly different hardware resources and power levels, without retraining. Second, we demonstrate that elasticity in SNNs also corresponds to proportional reductions in compute costs: reducing network size not only lowers synaptic weight counts but also directly reduces the total spike firing rate, achieving significantly lower energy consumption than current state-of-the-art approaches at comparable or better accuracies. The main contributions of this work are: • Elastic spiking transformer architecture: We propose NESTformer, the first SNN transformer model, to the best of our knowledge, capable of runtime adaptability without retraining. By introducing elasticity across all three building blocks (feature extraction, self-attention, and MLP), we enable a single universal model to dynamically scale its memory footprint without retraining. • Neuromorphic-native attention mechanism: We introduce a novel row-wise spiking attention computation scheme that replaces global matrix multiplications with sequential synaptic operations, for better compatibility with neuromorphic-native hardware. • Energy scaling: We analyze existing spiking transformers and demonstrate how scaling model size often yields sub-linear energy savings due to high spike generation in the feature extraction stage. In contrast, we show that NESTformer’s elasticity directly reduces the total spike firing rate, resulting in a linear reduction in energy. • State-of-the-Art performance: We validate our approach on the EHWGesture clinical dataset [ 2 ] and standard event-based benchmarks (DVS128 [ 1 ] , CIFAR10-DVS [ 17 ] ). Our results show that NESTformer matches or outperforms independently trained baselines (75.98% on EHWGesture, 98.96% on DVS128) while consuming up to 60% less energy than current state-of-the-art architectures. II RELATED WORKS Spiking Neural Networks Early SNN research focused on ANN-to-SNN conversion methods [ 8 , 26 ] or on directly training Convolutional SNNs using surrogate gradient techniques [ 23 , 28 ] . Recent advances in Spiking Transformers have brought attention-based architectures to the neuromorphic domain, achieving strong performance on gesture benchmarks. Spikformer [ 35 ] introduced Spiking Self-Attention (SSA), replacing the softmax operation with spike-based query-key-value interactions. Spikingformer [ 16 ] refined residual connections with “Dual-Spike” connections to improve gradient flow, enabling deeper networks with improved accuracy. QKFormer [ 34 ] , introduced a hierarchical two-stage architecture with Q-K Attention, requiring linear complexity with respect to token length. However, like its predecessors, QKFormer still relies on operations, such as Batched Matrix Multiply (BMM), that are not natively supported on current neuromorphic hardware and instead run on the CPU. Spike-driven Transformer [ 31 ] instead focused on strict adherence to the spike-driven philosophy, introducing Spike-Driven Self-Attention (SDSA) that eliminates all multiplications, relying solely on mask-and-add operations.