Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse
Bole Ma, Jan Eitzinger, Harald Köstler & Gerhard Wellein · 2026
Read Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse by Bole Ma, Jan Eitzinger, Harald Köstler & Gerhard Wellein (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-23581.txt.
Introduction
Ma, Eitzinger, Köstler, and Wellein make cached attention memory reusable anywhere in a sequence, in a 2026 machine learning systems paper. Multimodal agents keep re-examining the same video frames and screenshots as their context window slides. Standard caches store key-value pairs, the attention memory of a language model, tied to fixed positions. So every fresh look triggers a full re-encoding, wasting most of the compute budget on repeats.
The authors pinpoint what naive reuse loses: conditioning that a chunk of content absorbs from its neighbors. A small low-rank patch, a compact correction, stored beside each chunk restores exactly that missing part. Reuse then needs only a rotation of position codes plus the patch, with no retraining. A patch recovers full accuracy on benchmarks that bind information across chunks, where blind reuse halves multi-hop accuracy. The method also reconstructs production cache data to within rounding error.
Catalog record
| CREATOR | Bole Ma, Jan Eitzinger, Harald Köstler & Gerhard Wellein |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.23581
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse Bole Ma, Jan Eitzinger, Harald Köstler & Gerhard Wellein Abstract Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-encodes from scratch, because prefix caches serve reuse only at a fixed leading position. We show this recompute is avoidable, and identify exactly what naive KV reuse loses: the cross-chunk conditioning a chunk absorbs from its neighbours. This loss is asymmetric. The direct readout of a cached chunk is recovered exactly and for free by the standard state-merge. What remains is a diffuse, low-rank residue concentrated in deep layers, invisible to single-hop retrieval but precisely what multi-hop reasoning binds on. Blind reuse therefore leaves single-hop recall intact while halving multi-hop accuracy; this is the failure mode prior position-independent caches, designed for single-context or single-image reuse, do not address. We repair it with a small, training-free low-rank conditioning patch stored alongside each position-free chunk. Reuse reduces to one operator across MLA, GQA, and MHA: exact RoPE re-rotation to any target position, plus the patch that restores cross-chunk binding. This makes three window operations cheap: reorder (one patch serves every ordering of a cached set), sliding-window survival (surviving chunks relocate via rotation only, zero re-encode), and recall (an evicted chunk is rehydrated by its patch, never re-encoded). A rank- m m patch recovers full task accuracy on cross-chunk-binding benchmarks, MM-NIAH across two attention families and two-page doc-QA, at a fraction of the KV footprint, and reconstructs re-prefill KV to within bf16 rounding in a production SGLang kernel across six backbones. The conditioning signal is strongest in redundant vision and video streams, making our solution most impactful where multimodal agents spend their recompute budget. 1 Introduction A multimodal agent’s context routinely outgrows its attention window. A web agent slides a three-screenshot window over a monotonically growing transcript ( 13 ) ; a long-video agent re-examines the same clip across many reasoning steps under changing prompts ( 12 ; 47 ) ; a document agent traverses pages coarse-to-fine and re-accesses earlier ones after semantic filtering ( 51 ) . In every case the same visual content is encoded, dropped, and seen again at a new position, behind a changed prefix, or after eviction. Memory, not the nominal window, is the operative bound: even as context windows reach 1 M tokens, a locally served model holds only as much KV cache as device memory allows, so the sequence length used in practice is set by KV capacity, and the agent must continually slide, evict, and re-admit content. Managing context beyond the window is, operationally, managing this churn of reuse. Reuse is enormously cheap when it works: encoding a 1024 -token video segment costs \approx\!230 ms of vision-tower compute, while replaying its stored KV costs \approx\!5 ms ( 50 ; 18 ) . But production caches express only one shape of reuse, because they treat the KV store as a position-indexed structure. A prefix cache is an array : a contiguous span addressed by absolute position, so evicting the oldest token shifts every position behind it, an O(n) re-prefill. A radix cache adds a tree of prefixes shared across requests ( 50 ) . Both reuse a chunk only while it sits at a fixed leading position behind a byte-identical prefix; the moment the window slides, the prefix changes, or the chunk is recalled at a new offset, the cache misses and the engine re-encodes and re-prefills from scratch (Fig. 1 , top). The recurring multimodal patterns above (sliding windows, reorderings, look-backs) are radix misses by construction. Making a chunk position-free changes the structure available: the same store can act as a deque , evicting and admitting at either end of the window in O(1) , and, keyed by content rather than offset, points toward a content-addressed hash table of reusable chunks. Why the miss is not fundamental. Re-prefilling a chunk at a new position recomputes two things that need not be recomputed. First, position : a chunk’s keys differ across offsets only by a RoPE phase rotation, which composes exactly ( R(\delta)R(p){=}R(p{+}\delta) ), so relocation is an algebraic re-rotation, not a forward pass. Second, conditioning : prefilling a chunk B (the content we cache and reuse) after an antecedent A (whatever context precedes it in the window) lets B ’s tokens absorb A (coreferences resolved, entities bound). Concatenating independently cached chunks loses this cross-chunk conditioning, and only this, because the other cross-attention (readout, what a query reads out of a chunk) is recovered exactly by the log-sum-exp state-merge that FlashAttention and ring/star attention already perform ( 7 ) . Reuse is thus lossless for single-hop questions and silently breaks multi-hop ones: on a two-page document task, single-hop accuracy is unchanged under reuse ( 0.57 ) while multi-hop accuracy falls 0.41\!\to\!0.28 (MLA) and 0.28\!\to\!0.15 (GQA). The model still answers fluently; it just stops resolving “the object shown earlier.” What we restore, and how. We name the lost term, \Delta=\mathrm{KV}(B\!\mid\!A)-\mathrm{KV}(B\!\mid\!\varnothing) ( B ’s key/value with A in front of it minus B cached alone, \varnothing marking the absent antecedent; § 2 ), measure its shape, and attack it with the operator its shape dictates. The deficit is diffuse across tokens (no small “important-token” set; an oracle token selector needs \approx\!50\% of tokens), yet low-rank in features ( \approx\!90\% of its output-relevant energy in \approx\!32 directions) and deep (negligible in shallow layers). So the prevailing fix, recomputing a few important tokens (CacheBlend ( 44 ) , VLCache ( 25 ) , EPIC ( 14 ) , MPIC ( 15 ) ), corrects the wrong axis. These methods were validated on single-context compression or single-image recurrence under prompt staleness, where the reused KV is nearly valid and a few token recomputes suffice; none target the cross-chunk binding a windowed agent breaks, and on a real multi-hop video task they recover only a fraction of the answer flips (mean {\approx}20\% , \leq\!36\% at any token budget; against the patch’s 97\% ; § 6 ). We instead store each chunk as a position-free canonical \mathrm{KV}(B\!\mid\!\varnothing) plus a rank- m conditioning patch , and reuse it with \widehat{\mathrm{KV}}(B\!\mid\!A)\;=\;\underbrace{R(\delta)\cdot\mathrm{KV}(B\!\mid\!\varnothing)}_{\text{relocate (exact)}}\;+\;\underbrace{U_{m}V_{m}^{\!\top}}_{\text{rank-}m\text{ patch (conditioning)}} (1) where R(\delta) re-rotates the stored keys’ RoPE phase to the new position and U_{m}V_{m}^{\!\top} is the top- m SVD of \Delta , supervised by a single conditioned forward at compile time. The same operator covers MLA ( 8 ) , GQA ( 2 ) , and MHA once each layout is read through a single content \mid rope split (§ 3 ). It is training-free and runs recompute-free inside a production engine. The window operations this buys. Separating position from conditioning, and storing the canonical apart from the patch, turns three window operations from re-prefills into millisecond cache edits (Fig. 1 , bottom; § 5 ). (i) Reorder (reranked RAG, reshuffled frames, multi-image VQA): one stored orbit patch serves every ordering of a cached set (verified exhaustively at K{=}3 ). (ii) Sliding-window survival : when the window slides and an older chunk leaves, the chunks that stay need only the exact re-rotation, no patch, and stay near-lossless. (iii) Recall (reversible eviction): an evicted chunk’s conditioned KV can