Leyline: KV Cache Directives for Agentic Inference

Bole Ma, Jan Eitzinger, Harald Köstler · 2026

Read Leyline: KV Cache Directives for Agentic Inference by Bole Ma, Jan Eitzinger, Harald Köstler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01065.txt.

Description

Ma, Eitzinger, and Koestler give agent runtimes a way to edit their own memory cache in this 2026 paper. KV caches store past computation so a model need not redo it. Cache tools assume the chatbot pattern: prompts arrive once and the cache only grows. Agent work breaks that. Failed tool calls retry, stale outputs drop, and paths pivot mid task.

Two cache problems follow. Content that moves position breaks exact prefix reuse, a flaw others have begun to fix. The deeper one is theirs: a policy may need a span of cache cut or swapped, then work goes on without re-filling all that followed. No primitive offered that. Harnesses pay full re-fill costs on each edit. Their primitive, Leyline, takes a declarative directive. A four part tuple states what to edit and how to keep positions sound.

Catalog record

CREATORBole Ma, Jan Eitzinger, Harald Köstler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE131 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01065

Browse the full library · About the Tower of Babel