Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
Vedant Patel · 2026
Read Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents by Vedant Patel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27472.txt.
Introduction
Patel isolates a failure mode of language model agents that long conversations expose: they keep using facts that have already been superseded, in a machine learning paper published in 2026. When a user moves house or a price changes, the agent must act on the current value. On a knowledge-update benchmark drawn from real conversations, swapping full context for a bounded self-maintained memory drops a frontier model's accuracy from ninety-two percent to seventy-seven percent in the reported tests. Memory maintenance, not comprehension, is the bottleneck, and stronger models do not close it.
As conversations grow twenty-four-fold, accuracy falls further still, and granting proportionally more memory yields no recovery. Patel then releases Supersede, a reinforcement-learning environment rewarding answers from current values and penalising stale ones. Fine-tuning a small open model there almost doubles its held-out supersession accuracy on unseen conversations.
Catalog record
| CREATOR | Vedant Patel |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.27472
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents Vedant Patel Abstract Large language model (LLM) agents increasingly operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised. Acting correctly requires using the current value of a fact and discarding values that have been superseded . We isolate this ability on real conversational data and show that it is a distinct, unsolved failure. On the knowledge-update subset of LongMemEval, replacing an agent’s full context with a bounded, self-maintained memory drops accuracy from 92 % 92\% to 77 % 77\% even on a frontier model (gpt-5.4), a gap that is statistically significant (paired McNemar p < 0.005 p<0.005 ) and persists across model scale while full-context accuracy saturates near 92 % 92\% . The bottleneck is therefore memory maintenance, not comprehension, and it is not closed by a stronger model. We then ask whether this is merely an artifact of an undersized memory, and find it is not: as the conversation grows 24 × 24\times , accuracy falls further (from 68 % 68\% to 28 % 28\% ), and granting the agent proportionally more memory yields no detectable recovery ( 28 % → 28 % 28\%\!\rightarrow\!28\% , n = 25 n=25 ). The failure scales with the length of the conversation, not with the compression ratio. We release Supersede , an open reinforcement-learning environment (built on the verifiers / prime-rl stack) that turns this measurement into a training signal: agents are rewarded for answering from the current value and penalized for stale ones. Finally, we close the loop and show the gap is trainable: GRPO fine-tuning a small open model (Qwen2.5-3B) on this environment nearly doubles its held-out supersession accuracy on real, unseen conversations ( 9.0 % → 16.7 % 9.0\%\!\rightarrow\!16.7\% , a single run), along a monotonic checkpoint curve that indicates the learned policy, not the harness, carries the gain. To our knowledge this is the first trainable environment whose reward targets temporal fact-currency, and the first evidence that the supersession gap can be trained down rather than only measured. Code & Environment \cdot Prime Intellect Hub \cdot Model & Data 1 Introduction Pre-training has largely consumed the open web, and the frontier of LLM capability has shifted toward agents that act over long horizons using external memory rather than a single context window. A defining property of long-horizon interaction is that information is not static: facts are introduced, then later updated or retracted. An assistant that is told a user lives in Detroit, and later that they moved to the suburbs, must answer a subsequent location question with the current value. We call the correct handling of such updates supersession , and the accuracy an agent loses when it must sustain it under a bounded, self-maintained memory the supersession gap . This paper measures that gap, shows the obvious remedies do not close it, and then trains it down. Recent benchmarks have begun to measure long-term memory ( Maharana et al. 2024 ; Wu et al. 2025 ; He et al. 2026 ; Hu et al. 2025 ; Uddin et al. 2026 ) , and a forgetting-aware metric has been proposed to penalize reliance on obsolete information ( Uddin et al. 2026 ) . These works, however, score frozen models: they quantify the problem but do not provide a mechanism to train it away. In parallel, a line of work applies reinforcement learning to memory agents ( Yu et al. 2025 ; Chen et al. 2026 ) , but rewards final-answer correctness or evidence relevance, never temporal currency. No existing work sits in the intersection: a trainable environment whose reward is supersession-correctness. This paper makes five contributions, structured as one investigation: we ask in turn whether the failure exists, whether a bigger model fixes it, whether a bigger memory fixes it, and whether training fixes it — and answer no, no, no, and yes. The first three answers establish that the easy escapes are closed; the fourth shows the principled one is open. 1. An isolation result. On real conversational data, we hold memory load fixed and vary only whether the agent has full context or a bounded, self-maintained memory. Bounded memory degrades knowledge-update accuracy significantly (Section 5.1 ), isolating memory maintenance, not reading comprehension, as the cause. 2. A frontier confirmation. The gap survives on gpt-5.4 ( 92\%\!\rightarrow\!77\% , p=0.0033 ) and does not shrink to zero as the model improves; full-context accuracy, by contrast, saturates. A bigger model does not fix it. 3. Scale, not size. As the conversation grows 24\times , the failure deepens ( 68\%\!\rightarrow\!28\% ), and giving the agent proportionally more memory yields no detectable recovery at n=25 (Section 5.2 ). The failure tracks the length of the conversation, not the compression ratio, so a bigger memory does not fix it either. 4. An open environment. We release Supersede 1 1 1 Code, environment, trained model, and dataset are publicly available: https://github.com/Vrin-cloud/supersede (environment), https://huggingface.co/vedant33/supersede-qwen2.5-3b-grpo-lora (model), and https://huggingface.co/datasets/vedant33/supersede-rl-episodes (dataset); the environment is also published on the Prime Intellect Environments Hub. , a verifiers / prime-rl environment with a programmatic supersession-aware reward, validated end-to-end. It reframes the forgetting-aware metric of Uddin et al. 2026 as a learning signal , so the failure can be trained against rather than only measured. 5. Training closes it. Fine-tuning a small open model (Qwen2.5-3B) on the environment with GRPO nearly doubles held-out supersession accuracy on real, unseen conversations ( 9.0\%\!\rightarrow\!16.7\% , Section 5.3 ; a single training run, multi-seed significance in ongoing work), monotonically as training progresses. Where a bigger model and a bigger memory did not, a trained memory policy does, turning the diagnosis into a fix. 2 Related Work Long-term memory benchmarks. LoCoMo ( Maharana et al. 2024 ) evaluates very long multi-session dialogue but does not annotate a dedicated update category. LongMemEval ( Wu et al. 2025 ) introduces an explicit knowledge-update question type in which a fact stated in one session is changed in a later one; we adopt this subset as ground truth. MemoryArena ( He et al. 2026 ) shows that models near-perfect on recall benchmarks drop sharply when memory must drive actions, motivating evaluation beyond passive recall. MemoryAgentBench ( Hu et al. 2025 ) and MemBench ( Tan et al. 2025 ) add knowledge-updating and conflict-resolution competencies. Memora and its FAMA metric ( Uddin et al. 2026 ) explicitly penalize reliance on superseded or deleted memory, and temporal-conflict QA ( Özer & Yıldız 2025 ) probes whether models resolve facts that change over time. All of these are evaluation-only. Memory systems and RL for memory. Mem0 ( Chhikara et al. 2025 ) manages a memory store with prompted add / update / delete operations: a heuristic, not a learned, policy. A line of work instead learns memory management with RL: MemAgent ( Yu et al. 2025 ) rewards final-answer correctness, LongRLVR ( Chen et al. 2026 ) rewards evidence relevance within a single long document, AgeMem ( Yu et al. 2026 ) learns store/update/discard operations under task-outcome rewards, and Memory-T1 ( Du et al. 2025 ) learns to select temporally relevant sessions from the full history. Two things separate Supersede. First, these methods