Neural Dynamics Self-Attention for Spiking Transformers
Dehao Zhang, Fukai Guo, Shuai Wang, Jingya Wang, Jieyuan Zhang, Yimeng Shan, [0.3em] Malu Zhang, Yang Yang, Haizhou Li, [0.3em] University of Electronic Science, Technology of China, [0.3em] Shenzhen Loop Area Institute, The Chinese University of Hong Kong (Shenzhen) · 2026
Read Neural Dynamics Self-Attention for Spiking Transformers by Dehao Zhang, Fukai Guo, Shuai Wang, Jingya Wang, Jieyuan Zhang, Yimeng Shan, [0.3em] Malu Zhang, Yang Yang, Haizhou Li, [0.3em] University of Electronic Science, Technology of China, [0.3em] Shenzhen Loop Area Institute, The Chinese University of Hong Kong (Shenzhen) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19290.txt.
Description
Zhang and co-authors shrink the memory cost of attention in spiking transformers for vision tasks. These models lag classic networks and store large attention maps at test time. The team traces both flaws to the spiking attention step itself. Their LRF-Dyn unit gives more weight to nearby image regions, like real visual neurons do. It then runs the attention math through charge, fire, and reset steps. No explicit attention map needs to be stored. Tests show lower memory use and clear gains in accuracy.
Catalog record
| CREATOR | Dehao Zhang, Fukai Guo, Shuai Wang, Jingya Wang, Jieyuan Zhang, Yimeng Shan, [0.3em] Malu Zhang, Yang Yang, Haizhou Li, [0.3em] University of Electronic Science, Technology of China, [0.3em] Shenzhen Loop Area Institute, The Chinese University of Hong Kong (Shenzhen) |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34416 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19290