SPKLIP: Aligning Spike Video Streams with Natural Language
Yongchang Gao, Meiling Jin, Zhaofei Yu, Tiejun Huang, Guozhang Chen · 2025
Read SPKLIP: Aligning Spike Video Streams with Natural Language by Yongchang Gao, Meiling Jin, Zhaofei Yu, Tiejun Huang, Guozhang Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12656.txt.
Description
Spike cameras report only sudden brightness changes, as a sparse rain of events, which makes their footage hard to pair with words. Yongchang Gao and co-authors introduce SPKLIP, the first system built to align spike video with natural language. A layered extractor reads the event stream at several time scales at once, and contrastive training pulls matching video and text together in a shared space.
The alignment supports learning from few examples, handy where labelled spike data is scarce. A fully spiking version of the visual encoder runs leaner still, suiting brain-inspired hardware. Tests set state-of-the-art marks on existing spike benchmarks and generalised well to a new real-world dataset the team contributed. Code and data are public.
Catalog record
| CREATOR | Yongchang Gao, Meiling Jin, Zhaofei Yu, Tiejun Huang, Guozhang Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-12656