MEME: Multi-entity & Evolving Memory Evaluation
Seokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected] · 2026
Read MEME: Multi-entity & Evolving Memory Evaluation by Seokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12477.txt.
Description
Jung and four colleagues test whether agent memory systems cope when facts involve many entities and change over time. Their MEME benchmark defines six tasks across those two axes, including cascade reasoning, absence reasoning and deletion, areas earlier benchmarks left unscored. Across six memory systems and one hundred controlled episodes, every system collapses on dependency reasoning, scoring only a few percent, despite decent static retrieval. Prompt tuning, deeper retrieval and stronger models barely help, except for one costly file-based agent.
Catalog record
| CREATOR | Seokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 110368 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12477