LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel

Zhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang · 2026

Read LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel by Zhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20368.txt.

Description

Feng and seven colleagues present LaplacianFormer, a Transformer swapping softmax attention for a Laplacian kernel to tame quadratic scaling. Softmax attention grows quadratically, blocking high resolution vision work, while Gaussian linear variants lack grounding and mute mid range token links. A provably injective feature map keeps fine grained token detail under low rank approximations, and Nyström plus Newton Schulz iteration dodges matrix inversion and SVD. Custom CUDA code drives fast forward and backward passes for edge use, and ImageNet tests show strong trade offs with sharper attention.

Catalog record

CREATORZhe Feng, Sen Lian, Changwei Wang Muyang Zhang Tianlong Tan Rongtao Xu, Weiliang Meng Xiaopeng Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43854
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20368

Browse the full library · About the Tower of Babel