CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

Ning Yang, Chengzhi Wang, Yibo Liu, Baoliang Tian, Haijun Zhang · 2026

Read CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation by Ning Yang, Chengzhi Wang, Yibo Liu, Baoliang Tian, Haijun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08686.txt.

Description

Prefill-only KV compression picks tokens once and never revisits the choice, yet it estimates its correction signals from a single noisy prompt. Ning Yang and four co-authors argue that is the wrong statistical unit: reliability and sensitivity patterns are far steadier across prompts than within one. Their arXiv paper introduces CompilerKV, which compiles correction tables offline from a calibration corpus.

Online correction then shrinks to constant-time lookups plus a budget clamp. Compiled tables transfer across disjoint corpora and even across models at small cost. At a 512-token budget the method leads compressed baselines on all four backbones tested, and it stays strongest under extreme cache pressure. Readers get a portable prior for KV retention that costs almost nothing at serving time.

Catalog record

CREATORNing Yang, Chengzhi Wang, Yibo Liu, Baoliang Tian, Haijun Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08686

Browse the full library · About the Tower of Babel