DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity
Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu · 2026
Read DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity by Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08005.txt.
Description
Long-context inference strains both memory, as the key-value cache grows linearly, and compute, as attention grows quadratically. Jitai Hao and four co-authors propose ResidualKV, which splits the cache into a sparse set of globally retrieved reference states plus compact, quantized residual codes for the remaining tokens.
Nothing is permanently evicted: with sparse attention, only selected states are reconstructed on demand, and dynamic-stride scheduling slows reference growth from linear to roughly logarithmic at ultra-long contexts. Across Llama, Qwen, and vision-language backbones, ResidualKV keeps near-full-cache performance using only 13 to 16 percent of KV storage. Builders of long-context, multi-turn systems gain a compression scheme that preserves information instead of throwing it away.
Catalog record
| CREATOR | Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-08005