FLAMES: A Hybrid Spiking-State Space Model for Adaptive Memory Retention in Event-Based Learning
Biswadeep Chakraborty, Saibal Mukhopadhyay · 2025
Read FLAMES: A Hybrid Spiking-State Space Model for Adaptive Memory Retention in Event-Based Learning by Biswadeep Chakraborty, Saibal Mukhopadhyay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01257.txt.
Introduction
Chakraborty and Mukhopadhyay build a neural network that remembers events the way brains do, in spikes. Their framework, FLAMES, marries structured state-space models, which track long sequences efficiently, with event-driven spiking computation. Its core mechanism stretches or shrinks memory according to the gaps between incoming spikes, so both quick bursts and long-range links survive.
A normal-plus-low-rank factorization keeps the arithmetic cheap, cutting the cost from quadratic to near-linear in the state size. The model tops the Long Range Arena benchmark and performs strongly on two event-camera data sets. The authors present it as a bridge between neuromorphic hardware ideas and modern sequence modeling, aimed at learning from sparse, timed events.
Catalog record
| CREATOR | Biswadeep Chakraborty, Saibal Mukhopadhyay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 112 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-01257
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FLAMES: A Hybrid Spiking-State Space Model for Adaptive Memory Retention in Event-Based Learning Biswadeep Chakraborty, Saibal Mukhopadhyay Abstract We propose FLAMES (Fast Long-range Adaptive Memory for Event-based Systems) , a novel hybrid framework integrating structured state-space dynamics with event-driven computation. At its core, the Spike-Aware HiPPO (SA-HiPPO) mechanism dynamically adjusts memory retention based on inter-spike intervals, preserving both short- and long-range dependencies. To maintain computational efficiency, we introduce a normal-plus-low-rank (NPLR) decomposition, reducing complexity from 𝒪 ( N 2 ) \mathcal{O}(N^{2}) to 𝒪 ( N r ) \mathcal{O}(Nr) . FLAMES achieves state-of-the-art results on the Long Range Arena benchmark and event datasets like HAR-DVS and Celex-HAR. By bridging neuromorphic computing and structured sequence modeling, FLAMES enables scalable long-range reasoning in event-driven systems. † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : Block diagram of the proposed model architecture. Input spikes are processed event-by-event by the Dendrite Attention Layer, which extracts spatio-temporal features from local dendritic branches. The Spatial Pooling Layer aggregates spikes, followed by SA-HiPPO Convolution and LayerNorm layers, which are repeated N times to enable hierarchical feature extraction and long-range temporal dependency modeling. Spiking Neural Networks (SNNs) offer a biologically inspired and computationally efficient framework for processing temporal data through sparse, event-driven computations. Unlike conventional artificial neural networks, which rely on dense, synchronous activations, SNNs operate on discrete spike events, making them particularly effective for neuromorphic vision and real-time sensor processing ( Roy et al. 2019 ; Davies et al. 2018 ; Rathi et al. 2023 ) . This event-driven paradigm enables significant energy savings by performing computation only when information is available. However, a fundamental limitation of SNNs is their difficulty in retaining long-range temporal dependencies, restricting their applicability to complex sequence modeling tasks. The core of this limitation stems from the temporal processing characteristics of standard Leaky Integrate-and-Fire (LIF) neurons, where information is stored in exponentially decaying membrane potentials ( Bengio et al. 1994 ; Hochreiter & Schmidhuber 1997 ) . While this mechanism allows for efficient local integration of spike events, it also leads to rapid memory loss, severely limiting an SNN’s ability to capture dependencies over long timescales ( Neftci et al. 2019 ; Bellec et al. 2018 ) . Approaches that extend SNN memory through recurrent architectures ( Wu et al. 2018 ; Shrestha & Orchard 2018 ; Zheng et al. 2024b ) introduce dense state representations that compromise the inherent sparsity of spike-based computation. This presents a trade-off: increasing memory retention in SNNs typically comes at the cost of increased computational complexity. State Space Models (SSMs) have recently emerged as an effective alternative for long-range sequence modeling ( Gu et al. 2020 ; Gu et al. 2022a ) . A foundational component of these models is the HiPPO (High-Order Polynomial Projection Operator) framework ( Gu et al. 2020 ) , which optimally compresses and updates memory states over time. HiPPO constructs a set of orthogonal basis functions to continuously project and retain historical information, enabling efficient long-range memory retention in a structured manner. However, HiPPO and its extensions (e.g., S4 ( Gu et al. 2022a ) ) are designed for dense, continuous inputs, making them fundamentally incompatible with the sparse and asynchronous nature of SNNs. To bridge this gap, we introduce FLAMES (Fast Long-range Adaptive Memory for Event-based Systems) , a framework that integrates event-driven computation in SNNs with SSM-inspired structured memory to enable efficient long-range temporal modeling. The key innovation in FLAMES is the Spike-Aware HiPPO (SA-HiPPO) mechanism , which dynamically scales memory retention based on inter-spike intervals. Unlike the baseline HiPPO, which assumes continuous input and updates memory uniformly over time, SA-HiPPO adapts the state-space updates to the event-driven nature of SNNs. This adaptation ensures that memory is retained selectively—short-term dependencies are reinforced during frequent spiking, while long-term dependencies are preserved over sparse spiking intervals. By directly addressing the event-driven constraints of SNNs, FLAMES achieves efficient memory retention without sacrificing the sparsity and computational efficiency inherent to spike-based processing. FLAMES builds on three core theoretical innovations. First, we develop a spike-aware state transition mechanism that dynamically modulates memory retention based on inter-spike intervals ( Gu et al. 2022b ; Lotfi Rezaabad & Vishwanath 2020 ) , ensuring stability while maintaining computational efficiency. Second, we introduce a normal-plus-low-rank (NPLR) decomposition specifically optimized for sparse updates ( Gu et al. 2022a ) , reducing computational complexity from \mathcal{O}(N^{2}) to \mathcal{O}(Nr) while preserving expressivity. Third, we establish theoretical bounds on the trade-off between memory capacity and computational cost ( Wei et al. 2024 ; Chakraborty & Mukhopadhyay 2022 ) , providing a rigorous framework for optimizing event-driven memory representations. Our key contributions are: • Spike-Aware HiPPO (SA-HiPPO): A dynamic memory mechanism that adjusts state retention based on inter-spike intervals, preserving both short- and long-range dependencies. • Computationally Efficient State Updates: A normal-plus-low-rank (NPLR) decomposition reduces update complexity from \mathcal{O}(N^{2}) to \mathcal{O}(Nr) , enabling scalable state-space modeling in event-driven systems. • Hybrid Spiking-State Space Framework: Combines the sparsity of SNNs with structured memory updates, enabling efficient processing of long-range dependencies. • Dendritic Attention Layer: Extends LIF neurons with multi-timescale dendritic processing, enhancing feature extraction from sparse event-driven data. • Comprehensive Empirical Validation: FLAMES achieves state-of-the-art results across diverse benchmarks, including long-range dependency modeling (LRA), gesture recognition (DVS Gesture), and human activity recognition (HAR-DVS, Celex-HAR) demonstrating efficiency and scalability in event-driven tasks. Table 1 : Comparison of FLAMES with prior methods, highlighting features like memory retention, event-driven processing, scalability, efficiency, asynchronous updates, and adaptability. Model Type Dynamic Memory Retention Event-Driven Processing Scalable Long-Range Modeling Low Computational Overhead Fully Asynchronous Updates Adaptability to Sparse Data SpikingLMU ( Liu et al. 2024b ) SSM ✓ ✗ ✗ ✗ ✗ ✓ BinaryS4D ( Stan & Rhodes 2024 ) SSM ✗ ✗ ✓ ✗ ✗ ✗ HiPPO ( Gu et al. 2020 ) SSM ✗ ✗ ✓ ✗ ✗ ✗ DH-LIF ( Zheng et al. 2024b ) SNN ✗ ✓ ✗ ✗ ✓ ✗ EventMamba ( Ren et al. 2024 ) Hybrid CNN-SSM ✗ ✗ ✓ ✗ ✗ ✗ EventNet ( Turrero et al. 2024 ) Transformer ✗ ✗ ✓ ✗ ✓ ✗ SpikeRWKV ( Yao et al. 2024 ) Transformer ✗ ✓ ✓ ✗ ✓ ✓ S4 ( Gu et al. 2022a ) SSM ✗ ✗ ✓ ✓ ✗ ✗ FLAMES (Ours) SSM-SNN ✓ ✓ ✓ ✓ ✓ ✓ 2 Related Works State Space Models for Temporal Processing: State Space Models (SSMs) have emerged as a powerful framework for modeling long-range dependencies in temporal data ( Gu et al. 2020 ; Gu et al. 2022a ) . One foundational approach is the HiPPO (High-Order Polynomial Projection Operator) framework ( Gu et al. 2020 ) , which provides a theoretically optimal way to compress and propagate memory in continuous-time recurrent models. HiPPO constructs a set of basis functions that maintain information over time using structured state-space dynamics, offering efficient sequence modeling. However, a key limitation of HiPPO and its extensions (e.g., S4 ( Gu et al. 2022a ) , S5 ( Smith et al. 2022 ) ) is that they assume dense, synchronous inputs, making them inherently incompatible with the sparse, event-driven nature of Spiking Neural Networks (SNNs) ( Gu et al. 2021 ; Hasani et