A Simple Plug-in for Improving Eviction-Based KV Cache Compression

Yuping Lin, Jiayuan Ding, Yue Xing, Pengfei He, Jiliang Tang, Subhabrata Mukherjee · 2026

Read A Simple Plug-in for Improving Eviction-Based KV Cache Compression by Yuping Lin, Jiayuan Ding, Yue Xing, Pengfei He, Jiliang Tang, Subhabrata Mukherjee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23258.txt.

Description

Lin and five coauthors present VECTOR, a plug-and-play add-on for KV cache pipelines. Existing methods rely on binary dropping or approximation, which may underuse tokens not critical but still rebuildable. VECTOR offers three-way token routing: retention, approximation, and dropping. By using rebuildability, VECTOR recovers useful value information. On LongBench, RULER up to 128K tokens, and SCBench, VECTOR gains over base dropping methods under high shrinking. At 90 percent shrinking, SnapKV augmented with VECTOR beats six competing methods, including D2O and RocketKV.

Catalog record

CREATORYuping Lin, Jiayuan Ding, Yue Xing, Pengfei He, Jiliang Tang, Subhabrata Mukherjee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59169
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23258

Browse the full library · About the Tower of Babel