Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis

Giansalvo Cirrincione · 2026

Read Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis by Giansalvo Cirrincione (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08829.txt.

Description

Cirrincione presents the Layered Kernel model, a multi scale focus design handling series at L detail levels through trainable causal shrinking. It mixes level score tables with learned convex weights at roughly one and a third times the cost of standard focus. Four theory results cover a valid kernel rule, an even plus odd score split, an estimate error split with steady decay, and strict coverage of single head focus and causal filtering. Tests over three seeds show steady gains on ListOps, ordered CIFAR-10, and IMDB feeling.

Catalog record

CREATORGiansalvo Cirrincione
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67945
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.08829

Browse the full library · About the Tower of Babel