SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding

Yikang Yue, Yuqi Xue, Jian Huang · 2026

Read SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding by Yikang Yue, Yuqi Xue, Jian Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07223.txt.

Description

Long-context language models spend much of their inference budget on a growing key-value cache. Yikang Yue and two co-authors present Vegas, a self-speculative decoding method that drafts tokens from a subset of that cache and checks them against the full cache in parallel.

The trick is that checking itself reveals which cache entries matter, so Vegas reuses that information to choose the drafting subset, rather than running a separate choice step. That raises the rate at which drafted tokens are accepted and speeds up generation without changing its output. Builders serving long-context models get a lossless acceleration built from work the system already does.

Catalog record

CREATORYikang Yue, Yuqi Xue, Jian Huang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07223

Browse the full library · About the Tower of Babel