DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity

Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu · 2026

Read DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity by Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08005.txt.

Description

Long-context inference strains both memory, as the key-value cache grows linearly, and compute, as attention grows quadratically. Jitai Hao and four co-authors propose ResidualKV, which splits the cache into a sparse set of globally retrieved reference states plus compact, quantized residual codes for the remaining tokens.

Nothing is permanently evicted: with sparse attention, only selected states are reconstructed on demand, and dynamic-stride scheduling slows reference growth from linear to roughly logarithmic at ultra-long contexts. Across Llama, Qwen, and vision-language backbones, ResidualKV keeps near-full-cache performance using only 13 to 16 percent of KV storage. Builders of long-context, multi-turn systems gain a compression scheme that preserves information instead of throwing it away.

Catalog record

CREATORJitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08005

Browse the full library · About the Tower of Babel