VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
Yixuan Wang 1,2,3 , Qingyu Shi 4 , Jiayu Zhou 1 , Dianbo Liu 5 , Ziwei He 2 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Innovation Institute 3 Shanghai Artificial Intelligence Laboratory 4 University of Cambridge 5 National University of Singapore · 2026
Read VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization by Yixuan Wang 1,2,3 , Qingyu Shi 4 , Jiayu Zhou 1 , Dianbo Liu 5 , Ziwei He 2 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Innovation Institute 3 Shanghai Artificial Intelligence Laboratory 4 University of Cambridge 5 National University of Singapore (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16435.txt.
Description
Wang and co-authors shrink the key-value cache of large language models. Long contexts swell the cache and strain small machines. Older training-free tricks use low-rank fits or scalar quantization. Those fail to give high compression and high fidelity at once. Their VQKV method applies vector quantization instead. Thousands of values become a few integer indices. On LLaMA3.1-8B it compresses the cache by 82.8% while keeping 98.6% of baseline scores, and allows 4.3 times longer output on the same memory.
Catalog record
| CREATOR | Yixuan Wang 1,2,3 , Qingyu Shi 4 , Jiayu Zhou 1 , Dianbo Liu 5 , Ziwei He 2 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Innovation Institute 3 Shanghai Artificial Intelligence Laboratory 4 University of Cambridge 5 National University of Singapore |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34860 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16435