Learnable Pulse Accumulation for On-Device Speech Recognition: How Much Attention Do You Need?

Yakov Pyotr Shkolnikov · 2026

Read Learnable Pulse Accumulation for On-Device Speech Recognition: How Much Attention Do You Need? by Yakov Pyotr Shkolnikov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16922.txt.

Description

Shkolnikov introduces the Learnable Pulse Accumulator, a linear time swap for self attention in speech models. Self attention grows quadratically with sequence length, which hurts edge devices. The new module replaces key query dot products with learned gates: rectangular pulses, periodic windows, and position based functions. A diagnostic sweep ranks layers by replacement difficulty. Swapping 8 of 12 layers in wav2vec2-base gave a 10.61 percent word error rate on LibriSpeech test-clean. That run was 3.27 times faster at 120 second audio on an Apple M4 Pro.

Catalog record

CREATORYakov Pyotr Shkolnikov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44963
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16922

Browse the full library · About the Tower of Babel