LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel
Zhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang · 2026
Read LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel by Zhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20368.txt.
Description
Feng and seven colleagues present LaplacianFormer, a Transformer swapping softmax attention for a Laplacian kernel to tame quadratic scaling. Softmax attention grows quadratically, blocking high resolution vision work, while Gaussian linear variants lack grounding and mute mid range token links. A provably injective feature map keeps fine grained token detail under low rank approximations, and Nyström plus Newton Schulz iteration dodges matrix inversion and SVD. Custom CUDA code drives fast forward and backward passes for edge use, and ImageNet tests show strong trade offs with sharper attention.
Catalog record
| CREATOR | Zhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43854 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20368