BSO: Binary Spiking Online Optimization Algorithm
Yu Liang, Yu Yang, Wenjie Wei, Ammar Belatreche, Shuai Wang, Malu Zhang, Yang Yang · 2025
Read BSO: Binary Spiking Online Optimization Algorithm by Yu Liang, Yu Yang, Wenjie Wei, Ammar Belatreche, Shuai Wang, Malu Zhang, Yang Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12502.txt.
Introduction
Yu Liang and six co-authors propose an online training algorithm for binary spiking neural networks, which promise efficiency on resource-limited hardware. Training such networks usually stores latent weights and processes many time steps, so memory cost is high. That burden blocks training on the small devices where the networks are meant to run.
Their algorithm, BSO, updates weights directly through flip signals raised when the product of momentum and weight passes a threshold, with no latent weights kept. A temporal variant uses gradient information across time steps to adapt that threshold. Theory gives convergence guarantees and regret bounds for both versions, and experiments show stronger optimization than existing training methods.
Catalog record
| CREATOR | Yu Liang, Yu Yang, Wenjie Wei, Ammar Belatreche, Shuai Wang, Malu Zhang, Yang Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-12502
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BSO: Binary Spiking Online Optimization Algorithm Yu Liang, Yu Yang, Wenjie Wei, Ammar Belatreche, Shuai Wang, Malu Zhang, Yang Yang Abstract Binary Spiking Neural Networks (BSNNs) offer promising efficiency advantages for resource-constrained computing. However, their training algorithms often require substantial memory overhead due to latent weights storage and temporal processing requirements. To address this issue, we propose Binary Spiking Online (BSO) optimization algorithm, a novel online training algorithm that significantly reduces training memory. BSO directly updates weights through flip signals under the online training framework. These signals are triggered when the product of gradient momentum and weights exceeds a threshold, eliminating the need for latent weights during training. To enhance performance, we propose T-BSO, a temporal-aware variant that leverages the inherent temporal dynamics of BSNNs by capturing gradient information across time steps for adaptive threshold adjustment. Theoretical analysis establishes convergence guarantees for both BSO and T-BSO, with formal regret bounds characterizing their convergence rates. Extensive experiments demonstrate that both BSO and T-BSO achieve superior optimization performance compared to existing training methods for BSNNs. The codes are available at https://github.com/hamings1/BSO . 1 Introduction Spiking Neural Networks (SNNs), as the third generation of neural network paradigms ( 20 ; 28 ) , have garnered significant attention in machine learning attributed to their bio-inspired mechanisms and energy efficiency advantages. Building upon this foundation, Binary SNNs (BSNNs) further enhance computational efficiency by incorporating binary weights representations, offering a promising solution for resource-constrained edge computing scenarios ( 30 ; 19 ; 14 ) . Despite these notable advantages, BSNNs predominantly utilize training algorithms inherited from full-precision SNNs. Current BSNN training methods fall into two main categories: ANN-SNN conversion and direct training. ANN-SNN conversion obtains BSNN weights from pre-trained Artificial Neural Networks (ANNs) ( 29 ; 27 ; 33 ; 41 ) . However, these approaches typically require long simulation time steps to match ANN performance, inevitably increasing energy consumption. Moreover, it cannot handle sequential data. To address these limitations, researchers have developed direct training algorithms for BSNNs, with surrogate gradient-based backpropagation through time (BPTT) emerging as a predominant training paradigm ( 7 ; 22 ; 37 ) . Unfortunately, this methodology requires maintaining both computational graphs and latent weights for BSNNs, resulting in significant memory and computation overhead. Consequently, this resource-intensive training process inherently contradicts the efficiency advantages of BSNNs. This motivates us to raise an intriguing question: Can we leverage the substantial efficiency advantages of BSNNs not only during the forward pass but also throughout the learning process? Recently, online training algorithms for SNNs have gained prominence for their temporal independence, which substantially reduces training costs in memory ( 39 ; 21 ; 11 ) . However, existing SNN online training algorithms are predominantly designed for general situations. Given the efficiency advantages of BSNNs in resource-constrained environments, the integration of online training presents a promising direction in efficient neuromorphic computing. However, this integration faces a fundamental challenge: the storage overhead of latent weights in existing BSNN training methods conflicts with online training’s memory efficiency objectives. This contradiction underscores the urgent need to develop specialized online training techniques tailored for BSNNs. To address these challenges, we propose a Binary Spiking Online (BSO) optimization algorithm that leverages BSNN’s efficiency advantages in both forward and backward propagation. BSO offers two key advantages: first, its memory requirements are orthogonal to time steps, significantly reducing memory and computational overhead; second, it updates weights through sign flips, eliminating the need for latent weights. Furthermore, to enhance performance by leveraging the inherent temporal dynamics of SNNs, we introduce a temporal-aware variant called T-BSO, which effectively integrates gradient information across each time step. Contributions. In summary, our key contributions are summarized as follows: • We introduce BSO, the first online training optimization algorithm tailored for BSNNs. BSO significantly reduces memory overhead through two advantages: (a) ensuring memory requirements orthogonal to time steps and (b) eliminating latent weights storage via gradient-dominated flipping. • We propose T-BSO, an enhanced version of BSO that significantly improves performance by effectively exploiting SNNs’ temporal dynamics, capturing the second-order gradient moment across time steps for adaptive threshold optimization. • We rigorously prove the convergence of both BSO and T-BSO algorithms by deriving their theoretical regret bounds. Our comprehensive analysis demonstrates that the parameter updates in BSO and T-BSO maintain substantial directional consistency with convergence. • Our BSO and T-BSO demonstrate superior performance on both large-scale static and neuromorphic datasets across CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS. Experimental results verify time-independent training memory requirements, substantially reducing training overhead. 2 Related Work 2.1 Learning algorithm for BSNNs Training algorithms for BSNNs fall into two main categories: ANN-SNN conversion methods and direct training methods. The ANN-SNN conversion method trains an ANN first, then directly transfers the weights to a BSNN. ( 27 ) first analyze the combination of different binary neurons and binarized weight methods to train an ANN, and then obtain a deep SNN with binary stochastic activations through a conversion approach. Similarly, ( 19 ) utilize standard training techniques to train a binary convolutional model, then generate a binary SNN via a conversion process. Furthermore, ( 33 ) examine the relationship between weights and spiking neuron thresholds, proposing a weight-threshold balancing conversion method to reduce errors during conversion. However, these converted SNNs suffer from performance degradation and long latency. To address these limitations, researchers have developed direct training methods for BSNNs. ( 23 ) employ a surrogate gradient method to train a hardware-friendly BSNN for processing temporal neuromorphic data. ( 14 ) introduce BS4NN, a temporal-encoded BSNN where each neuron fires at most one spike, providing an energy-efficient event-driven learning approach. To further reduce the quantization error and improve performance, some studies have introduced different strategies to enhance performance, such as alternating direction methods ( 7 ) , accuracy loss estimator ( 22 ) , improved activation function ( 10 ) , weight-spike regulation ( 36 ; 32 ) , and adaptive gradient modulation ( 17 ) . Recently, researchers have also designed quantization strategies for spiking transformers, achieving competitive results on vision tasks ( 25 ; 4 ) . As described above, existing learning algorithms for BSNNs are adapted from full-precision SNNs. While these methods leverage BSNN’s energy efficiency in forward propagation, they fail to exploit its binary weight characteristics during the training process. This motivates us to design a dedicated training algorithm for BSNN that leverages the weight sign flipping property for parameter updates and eliminates latent weight reliance, enabling energy efficiency in both forward and backward propagation. 2.2 Online Training Method in SNNs In recent years, significant progress has been made in online training algorithms for SNNs to achieve memory-efficient and online training ( 42 ; 1 ; 3 ) . 13 proposes local training methods that disregard temporal dependencies while 40 adapts the approach from ( 12 ) using gated neuron models. Notably, OTTT ( 39 ) extends the applicability of online training methods to address large-scale computational tasks. SLTT ( 21 ) establishes that temporal domain gradients contribute minimally to SNN training and proposes their elimination for improved efficiency. NDOT ( 11 ) introduces novel gradient estimation techniques based on neuronal dynamic algorithm. These approaches collectively offer diverse solutions for reducing memory overhead during SNN