xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction

Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah · 2025

Read xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction by Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18893.txt.

Description

Chi-Chih Chang and co-authors shrink the memory that large language models need when they read long texts. Their 2025 paper targets the key-value cache, the stored record a model keeps of earlier words. That cache grows with text length and eats memory. Past attempts to share the cache across layers needed costly retraining or gave weak results.

The team found that the main patterns in the cache line up well across layers. Their method, xKV, squeezes grouped layers into one shared low-rank form, cutting cache size up to eightfold while keeping accuracy on long-text tasks. A selective rebuild step at output time adds speed, reaching over four times the baseline pace. The method needs no retraining, so it can plug into existing models.

Catalog record

CREATORChi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Yucheng Li, Huiqiang Jiang, Xilai Dai, Luis Ceze, Kai-Chiang Wu, Mohamed Abdelfattah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-18893

Browse the full library · About the Tower of Babel