SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding
Yikang Yue, Yuqi Xue, Jian Huang · 2026
Read SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding by Yikang Yue, Yuqi Xue, Jian Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07223.txt.
Description
Long-context language models spend much of their inference budget on a growing key-value cache. Yikang Yue and two co-authors present Vegas, a self-speculative decoding method that drafts tokens from a subset of that cache and checks them against the full cache in parallel.
The trick is that checking itself reveals which cache entries matter, so Vegas reuses that information to choose the drafting subset, rather than running a separate choice step. That raises the rate at which drafted tokens are accepted and speeds up generation without changing its output. Builders serving long-context models get a lossless acceleration built from work the system already does.
Catalog record
| CREATOR | Yikang Yue, Yuqi Xue, Jian Huang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07223