Cascade Token Selection for Transformer Attention Acceleration
Stephen J. Thomas · 2026
Read Cascade Token Selection for Transformer Attention Acceleration by Stephen J. Thomas (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03110.txt.
Description
Stephen J. Thomas speeds up representative token selection in transformer attention by reusing choices across layers. One attention method picks a small set of key tokens per layer, but building the full similarity matrix each time is costly.
The cascade inherits the prior layer's set, checks it with a smaller cross-matrix, and adds or drops a few tokens as needed. That cuts the selection cost sharply. On three model families, tests saved a large share of Gram work, and the key token sets overlapped strongly between layers. The result suggests informative tokens persist through network depth.
Catalog record
| CREATOR | Stephen J. Thomas |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03110