MEME: Multi-entity & Evolving Memory Evaluation

Seokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected] · 2026

Read MEME: Multi-entity & Evolving Memory Evaluation by Seokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12477.txt.

Description

Jung and four colleagues test whether agent memory systems cope when facts involve many entities and change over time. Their MEME benchmark defines six tasks across those two axes, including cascade reasoning, absence reasoning and deletion, areas earlier benchmarks left unscored. Across six memory systems and one hundred controlled episodes, every system collapses on dependency reasoning, scoring only a few percent, despite decent static retrieval. Prompt tuning, deeper retrieval and stronger models barely help, except for one costly file-based agent.

Catalog record

CREATORSeokwon Jung Alexander Rubinstein Arnas Uselis, [2pt] Sangdoo Yun Seong Joon Oh, [6pt] KAIST AI Tübingen AI Center, University of Tübingen NAVER AI Lab[2pt] [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110368
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.12477

Browse the full library · About the Tower of Babel