Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Models
2025
Read Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Models (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17098.txt.
Introduction
Elias Hossain and three co-authors probe a weak spot in large language models in this 2025 paper. While a model writes, it stores past keys and values in a fast cache. That cache is rarely checked, even when prompts and weights are locked down. The authors test what harm small cache changes can do.
Their attack adds noise, zeros, or twists to stored keys at chosen spots. Math links the size of the harm to the size of the change. Tests on two model families show clear shifts in next-word picks. The harm also spreads to search-aided and agent tasks. The work marks cache safety as a key need for safe use.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17098
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Models Abstract Even when prompts and parameters are secured, transformer language models remain vulnerable because their key–value (KV) cache during inference constitutes an overlooked attack surface. To address this, we present MTI V.1, a modular framework that formalizes and implements the Malicious Token Injection (MTI) attack. Within this framework, cached key vectors are perturbed at selected layers and timesteps through controllable magnitude and frequency, using mechanisms such as additive Gaussian noise, zeroing, and orthogonal rotations. A theoretical analysis characterizes how these perturbations propagate through the attention mechanism, deriving bounds that link (i) logit deviations to the Frobenius norm of the corruption and query scaling, and (ii) shifts in attention and output distributions to these logit changes via softmax Lipschitz properties. Empirical evaluations demonstrate that MTI V.1 reliably alters next-token distributions, semantics, and downstream task performance on GPT-2 and LLaMA-2/7B across both synthetic prompts and standard NLP benchmarks. Extending to retrieval-augmented and agentic pipelines, cache corruption in post-retrieval layers is shown to destabilize grounding, reasoning, and action selection. Collectively, these findings identify cache integrity as a critical yet under-examined vulnerability in current LLM deployments and establish controlled KV corruption as a reproducible threat model for future robustness research. Keywords : Large Language Models, KV Cache, Transformer Security, Cache-Side Attacks, Model Robustness, Adversarial Perturbation, Cache Integrity, Attention Mechanism, RAG Systems, Agentic Reasoning. Elias Hossain, 1,* Swayamjit Saha, 2 Somshubhra Roy, 3 and Ravi Prasad 4 1 College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, United States 2,4 Department of Computer Science and Engineering, Mississippi State University, Mississippi State, MS 39762, United States 3 Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 27695-7911, United States * Corresponding author: [email protected] All authors contributed equally to this work. 1 Introduction Large language models (LLMs) have rapidly become the foundation of modern artificial intelligence, powering applications such as translation, summarization, retrieval-augmented generation (RAG), and autonomous agents [ 1 , 2 , 3 ] . Their effectiveness depends not only on model scale but also on computational mechanisms that make inference efficient in real-world deployment. Among these mechanisms, the key–value (KV) cache plays a crucial role by storing intermediate attention states to accelerate decoding over long contexts. Despite its ubiquity, the KV cache has received limited scrutiny as a potential attack surface compared with prompt injection, adversarial perturbations, or data poisoning attacks [ 4 , 5 ] . This oversight carries serious implications. Manipulations within cached representations can silently propagate through the inference pipeline, altering token predictions without modifying either the input or the model parameters. While previous research has concentrated on adversarial inputs [ 6 , 7 ] and model-level robustness [ 8 ] , vulnerabilities that arise during cache usage remain largely unexamined. Addressing these hidden weaknesses is increasingly critical for safety-sensitive applications, where reliability and verifiability are indispensable. In this paper, we introduce the MTI V.1, a cache-side attack framework that perturbs stored key–value representations during inference. We formally characterize how even small perturbations can bias attention distributions, distort next-token probabilities, and degrade model calibration. Through extensive experiments on GPT-2, LLaMA-2, and Gemma models, we demonstrate that MTI V.1 consistently reduces performance by 15–30% across classification, question answering, and summarization benchmarks. Beyond single-turn tasks, we show that cache corruption propagates through complex pipelines: in RAG settings, it amplifies hallucination rates, and in multi-step agentic reasoning, it disrupts task planning and decision consistency. To explore possible defenses, we assess lightweight strategies such as cache resetting, dropout-mask randomization, and attention smoothing. These countermeasures preserve clean-task accuracy and add minimal runtime cost, yet none entirely neutralize the effects of MTI V.1. Together, these findings reveal that the KV cache constitutes a critical but overlooked dimension of model robustness and security. The contributions of this study are summarized as follows: • We present MTI V.1, a cache-level attack framework that perturbs stored attention states and derive theoretical bounds on the resulting distributional shifts. • We empirically demonstrate the impact of cache corruption across multiple architectures and downstream pipelines, identifying significant vulnerabilities in RAG and agentic systems. • We evaluate practical defense mechanisms that achieve partial mitigation with minimal performance loss, while underscoring the need for verifiable cache integrity. By reframing the KV cache as an active component of model security rather than a passive optimization, this study situates cache corruption within broader discussions of calibration [ 8 ] , Bayesian uncertainty estimation [ 9 ] , and certified robustness [ 10 ] . We argue that maintaining cache integrity is essential for building reliable, transparent, and trustworthy language model systems. 2 Related Work The literature on transformer robustness can be broadly categorized into three domains, as illustrated in Figure 1 . The first involves architectural innovations that improve efficiency while preserving performance, typified by Multi-Head Latent Attention (MLA) and decoder-only caching. The second encompasses attacks targeting model integrity, such as weight-space backdoors or bit-flip corruption. The third and emerging domain examines the security of transient cache states during inference, including leakage and active manipulation. Our work advances this final axis by systematically analyzing the vulnerability of the KV-cache as an adversarial surface. Transformer Robustness & Security Landscape Architectural Efficiency Multi-Head Latent Attention (MLA) Decoder-only KV Caching Model & Weight Integrity Backdoors & Weight Poisoning Bit-Flip & Hardware Faults Cache-State Security Cache Leakage & Side Channels Cache Perturbation (This Work) Fig. 1 : Taxonomy of prior research on transformer robustness and security. Existing studies cluster into (i) architectural efficiency (e.g., MLA and KV caching), (ii) model and weight integrity (e.g., backdoors and bit-flip faults), and (iii) cache-state security (e.g., leakage and perturbation). Our work advances the third category by analyzing cache corruption as an active adversarial threat. 2.1 Multi-Head Latent Attention (MLA) Architectures Recent model families replace or augment standard self-attention with Multi-Head Latent Attention (MLA), which introduces a compact latent bottleneck to reduce memory bandwidth and improve throughput while maintaining long-context performance. DeepSeek-V2 first systematized MLA in large-scale pretraining and instruction tuning, reporting improved efficiency and quality trade-offs over traditional attention mechanisms [ 11 ] . Subsequent iterations such as DeepSeek-V3 refined routing and optimization, demonstrating further scaling and inference gains [ 12 ] . DeepSeek-R1 extended this paradigm to reasoning-focused objectives while retaining the MLA backbone for efficient multi-step inference [ 13 ] . Beyond production systems, TransMLA formalized the architecture for research reproducibility and interpretability [ 14 ] . Together, these works establish MLA as a viable alternative to dense attention, motivating deeper investigation into robustness and security within latent-space computations. 2.2 KV-cache in Decoder-Only LLM Serving Decoder-only Transformers universally employ key–value (KV) caching to accelerate autoregressive decoding by avoiding redundant attention computation. Representative examples include GPT-3, which popularized few-shot inference with cached states for latency reduction [ 1 ] ; LLaMA-2, whose open weights standardized KV-cached serving across research and industry [ 2 ] ; Mistral-7B, which incorporates cache-efficient decoding by default [ 15 ] ; and Mixtral 8×7B, a sparse MoE model caching active expert states for reuse [ 16 ] . Similarly, the Qwen series integrates group-query attention and cache optimizations to improve throughput in production [ 17 ] . While these advances