Leyline: KV Cache Directives for Agentic Inference
Bole Ma, Jan Eitzinger, Harald Köstler · 2026
Read Leyline: KV Cache Directives for Agentic Inference by Bole Ma, Jan Eitzinger, Harald Köstler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01065.txt.
Description
Ma, Eitzinger, and Koestler give agent runtimes a way to edit their own memory cache in this 2026 paper. KV caches store past computation so a model need not redo it. Cache tools assume the chatbot pattern: prompts arrive once and the cache only grows. Agent work breaks that. Failed tool calls retry, stale outputs drop, and paths pivot mid task.
Two cache problems follow. Content that moves position breaks exact prefix reuse, a flaw others have begun to fix. The deeper one is theirs: a policy may need a span of cache cut or swapped, then work goes on without re-filling all that followed. No primitive offered that. Harnesses pay full re-fill costs on each edit. Their primitive, Leyline, takes a declarative directive. A four part tuple states what to edit and how to keep positions sound.
Catalog record
| CREATOR | Bole Ma, Jan Eitzinger, Harald Köstler |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 131 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01065