Conf-KV : Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM Inference
Yubo Li Yidi Miao · 2026
Read Conf-KV : Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM Inference by Yubo Li Yidi Miao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24786.txt.
Description
Yubo Li and Yidi Miao present CONF-KV, a cache manager that sets the key-value budget at each step from model uncertainty. Long chats make the cache the main memory cost, yet most eviction rules use fixed windows or past attention. CONF-KV turns the next-token distribution into a confidence score. It keeps more context when unsure and prunes hard when sure. Paired with mixed-precision storage, it reaches 91 percent retrieval accuracy on a needle-in-a-haystack test.
Catalog record
| CREATOR | Yubo Li Yidi Miao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52474 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24786