You Need an Encoder for Native Position-Independent Caching
Shiju Zhao, Junhao Hu, Jiaqi Zheng, Guihai Chen · 2026
Read You Need an Encoder for Native Position-Independent Caching by Shiju Zhao, Junhao Hu, Jiaqi Zheng, Guihai Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01519.txt.
Description
Shiju Zhao and co-authors propose native position-independent caching for language models, needing an encoder. Prefix-based Key-Value caches handle arbitrary-order retrieved contexts poorly. Position-independent caching promises KV reuse without positional constraints, yet methods degrade accuracy badly. The team reintroduces the encoder into decoder-only models, training it for position-independent caching. Their COMB system integrates with existing inference frameworks, cutting time-to-first-token by 51-94% and tripling throughput with similar accuracy. DeepSeek-V2-Lite-Chat gains show COMB applies elsewhere.
Catalog record
| CREATOR | Shiju Zhao, Junhao Hu, Jiaqi Zheng, Guihai Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39136 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01519