State Contamination in Memory-Augmented LLM Agents

Yian Wang, Agam Goyal, Yuen Chen, Hari Sundaram · 2026

Read State Contamination in Memory-Augmented LLM Agents by Yian Wang, Agam Goyal, Yuen Chen, Hari Sundaram (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16746.txt.

Introduction

Wang and colleagues study a failure mode they call memory laundering in agents that reuse persistent state. Toxic or adversarial context can be compressed into memory summaries that pass standard toxicity detectors, while still steering later outputs toward hostile framing. Paired counterfactual rollouts show toxic-origin summaries staying below toxicity thresholds yet raising downstream toxicity. The team measures this with the sub-threshold propagation gap. Sanitizing toxic state before summarization cuts the gap sharply, while cleaning only the finished summary leaves laundered influence intact.

Catalog record

CREATORYian Wang, Agam Goyal, Yuen Chen, Hari Sundaram
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87902
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16746

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

State Contamination in Memory-Augmented LLM Agents Yian Wang, Agam Goyal, Yuen Chen, Hari Sundaram Abstract LLM agents increasingly rely on persistent state, including transcripts, summaries, retrieved context, and memory buffers, to support long-horizon interaction. This makes safety depend not only on individual model outputs, but also on what an agent stores and later reuses. We study a failure mode we call memory laundering: toxic or adversarial context can be compressed into memory summaries that no longer appear toxic under standard detectors, while still preserving hostile framing or conflict structure that influences future generations. Using paired counterfactual multi-agent rollouts, we show that toxic-origin memory summaries can remain below common toxicity thresholds while nevertheless increasing downstream toxicity relative to matched neutral baselines. To measure this hidden influence, we introduce the sub-threshold propagation gap (SPG), which quantifies downstream behavioral differences conditioned on memory states that a deployed monitor would classify as safe. Our experiments show that toxicity propagates through distinct state channels: raw transcript reuse drives overt downstream toxicity, while compressed memory carries hidden sub-threshold influence. We further find that mitigation depends critically on intervention placement. Sanitizing toxic state before summarization substantially reduces the hidden propagation gap, whereas cleaning only the completed summary can leave laundered influence intact. These results suggest that safety in memory-augmented agents should be treated as a state-control problem over evolving context, with sanitization applied before unsafe information is compressed into persistent memory. 1 Introduction LLM systems increasingly operate over long-running tasks: coordinating with users, collaborating with other agents, summarizing discussions, retrieving prior information, and carrying context forward over time ( 30 ; 33 ; 34 ; 18 ) . This ability to remember and reuse context is central to agent usefulness, but it also makes memory part of the system’s safety surface ( 29 ; 17 ) . However, this same ability also changes the problem of safety. In a single-turn chatbot, an unsafe response can often be treated as a bad output to detect, block, or rewrite ( 26 ; 9 ) . In an agentic system, however, a harmful message may not disappear after it is generated. It can be stored, summarized, retrieved, or passed along to other agents, becoming part of the context that shapes future behavior ( 12 ; 25 ) . Safety therefore depends not only on what an agent says at one moment, but also on what the system remembers from that moment and how that memory influences later generations, and the topological configuration they are connected under ( 37 ; 4 ) . Figure 1 : Memory laundering in stateful agents. (a) In single-turn chatbots, safety monitoring is typically applied directly to the generated response. In memory-augmented agents, harmful influence can instead be compressed into external agent state, such as summaries or transcripts, that appears safe under standard toxicity checks while still conditioning downstream agents toward unsafe behavior. (b) In our work, we show that memory sanitization before summarization helps mitigate memory laundering. (c) For a representative example, a toxic source message from an agent is compressed into a memory summary that scores below the classifier threshold, yet a downstream agent conditioned on that memory produces a hostile response. The summary removes explicit insults while preserving the confrontational framing of the exchange. Additional examples are shown in § D . Figure 1 illustrates the failure mode studied in this paper. Many agent systems compress long conversations into short summaries so that future agents can stay informed without reading the full history ( 36 ; 28 ; 5 ; 11 ) . This compression is usually treated as a practical engineering step: it saves context length and preserves the important parts of the interaction. We show that it can also act as a laundering step. A toxic message may be summarized into language that no longer looks toxic to a standard classifier, while still preserving the adversarial framing, conflict structure, or hostile stance that steers downstream agents toward toxic behavior. We call this failure mode memory laundering : toxic influence is compressed into a memory summary that appears safe under standard monitoring but remains behaviorally potent, causing downstream agents to produce more toxic responses than matched neutral baselines. Figure 1 gives a concrete example: a toxic message is compressed into a memory summary that appears safe to a toxicity classifier, yet downstream agents conditioned on that memory still produce hostile responses. We evaluate memory laundering with paired counterfactual multi-agent rollouts over 200 paired seeds. Although toxic-condition summaries fall below standard toxicity thresholds, they substantially increase downstream toxicity. We capture this hidden influence with the sub-threshold propagation gap (SPG): the toxic–neutral downstream difference conditioned on memory states satisfying \mathrm{tox}(M_{t})<\tau , i.e., the regime a deployed monitor would label safe. Here, state means external context reused by future generations: the raw transcript, compressed memory summaries, retrieved context, and other persistent context objects. It does not refer to the model’s parameters or hidden activations. Memory laundering occurs when this conditioning state is contaminated, compressed, and later reused. Our results show that toxic influence propagates through two distinct state channels. Raw transcript backflow primarily drives overt downstream toxicity, while compressed memory carries hidden sub-threshold influence. This distinction matters because sanitizing before memory update closes the laundered channel, whereas sanitizing only the completed summary can be too late because harmful framing may already have been compressed below the classifier threshold. Contributions. (1) We identify and empirically characterize memory laundering , a previously unexamined safety failure mode in memory-augmented LLM agents: summarization can transform overtly toxic context into classifier-clean memory states that nevertheless steer downstream agents toward toxic behavior. (2) We introduce the sub-threshold propagation gap (SPG), a paired counterfactual metric for measuring behavioral influence inside the regime a deployed memory monitor would classify as safe. Using SPG with the average downstream toxicity shift ( \Delta\mu ) and tail-toxicity statistics ( P95_{\text{tox}} ), we show that transcript exposure drives overt toxicity, while compressed memory carries hidden sub-threshold influence. (3) We show that mitigation depends on where sanitization is placed: toxic state must be sanitized before it is summarized, because cleaning only the completed summary can miss laundered influence. We evaluate a model-agnostic state-control framework that composes transcript control, memory control, and DPO, and show that the full system best suppresses mean, hidden, and tail propagation. 2 Related Work Memory and state in LLM agents. Recent LLM agent architectures increasingly rely on external state: conversation histories, retrieved documents, memory buffers, reflections, and summaries that are reused across turns ( 1 ; 30 ) . Prior work has shown that such state improves long-horizon interaction, planning, and coordination ( 34 ) . Generative Agents store natural-language memories, synthesize them into reflections, and retrieve them to guide later behavior ( 23 ) ; Reflexion uses verbal feedback stored in episodic memory to improve subsequent decisions ( 27 ) ; MemGPT treats long-context interaction as a memory-management problem ( 22 ) ; and AutoGen composes multiple conversable agents through shared interaction patterns ( 32 ) . These systems motivate our setting: memory is not merely a log of past interaction, but an input that shapes future behavior. However, prior work largely treats memory as a capability mechanism or an object to retrieve

Read the complete text