xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah · 2025
Read xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction by Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18893.txt.
Description
Chi-Chih Chang and co-authors shrink the memory that large language models need when they read long texts. Their 2025 paper targets the key-value cache, the stored record a model keeps of earlier words. That cache grows with text length and eats memory. Past attempts to share the cache across layers needed costly retraining or gave weak results.
The team found that the main patterns in the cache line up well across layers. Their method, xKV, squeezes grouped layers into one shared low-rank form, cutting cache size up to eightfold while keeping accuracy on long-text tasks. A selective rebuild step at output time adds speed, reaching over four times the baseline pace. The method needs no retraining, so it can plug into existing models.
Catalog record
| CREATOR | Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18893