Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents
Chongrui Ye Yuxiang Liu Yu Wang Haofei Yu, Yining Zhao Ge Liu Julian McAuley Jiaxuan You, [2pt] University of Illinois Urbana-Champaign · 2026
Read Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents by Chongrui Ye Yuxiang Liu Yu Wang Haofei Yu, Yining Zhao Ge Liu Julian McAuley Jiaxuan You, [2pt] University of Illinois Urbana-Champaign (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20616.txt.
Introduction
Chongrui Ye and seven colleagues present Auto-Dreamer, a learned offline consolidator for language-agent memory. Agents working through streams of tasks pile up experience, but most memory systems record each session alone. Their design splits fast per-session capture from slow cross-session consolidation, treating a chosen memory region as read-only evidence. The consolidator then inspects entries and linked source paths and writes a compact replacement set that abstracts across sessions.
Catalog record
| CREATOR | Chongrui Ye Yuxiang Liu Yu Wang Haofei Yu, Yining Zhao Ge Liu Julian McAuley Jiaxuan You, [2pt] University of Illinois Urbana-Champaign |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102711 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20616
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents Chongrui Ye Yuxiang Liu Yu Wang Haofei Yu, Yining Zhao Ge Liu Julian McAuley Jiaxuan You, [2pt] University of Illinois Urbana-Champaign Abstract Language agents increasingly operate over streams of related tasks, yet existing memory systems struggle to convert accumulated experience into reusable knowledge. Retrieval-augmented and structured memory methods record per-session observations effectively, but often couple acquisition and consolidation into a single online process, leaving the agent without a global view across sessions to discover recurring patterns, abstract shared procedures, or prune redundant entries. Inspired by complementary learning systems theory, we propose Auto-Dreamer, a learned offline consolidator for language-agent memory. Auto-Dreamer decouples fast per-session memory acquisition from slow cross-session consolidation. Given a selected working region of a typed memory bank, the consolidator treats the region as read-only evidence, performs bounded tool-use to inspect entries and provenance-linked source trajectories, and synthesizes a fresh compact replacement set that abstracts across sessions and supersedes the original region. We train Auto-Dreamer via GRPO, using end-to-end agent performance as the reward signal to learn how to consolidate memories acquired through fast online experience. Trained on ScienceWorld trajectories alone, Auto-Dreamer outperforms fixed, RL-trained, and prompted memory baselines on ScienceWorld by 7 points while using an active memory bank 12 × \times smaller than the strongest baseline, and continues to lead on held-out ALFWorld and WebArena without retraining — using 6 × \times less memory than the strongest baseline on ALFWorld. 1 Introduction Language agents are increasingly deployed over streams of related tasks rather than isolated interactions [ 27 , 33 ] . In such settings, long-term memory is not merely a retrieval cache for past entities or user preferences; it is the mechanism by which an agent converts raw experience into reusable procedures, environment knowledge, and behavioral priors that improve future decision making. A memory system must therefore solve two distinct problems: it must rapidly acquire useful information from each new trajectory, and it must periodically reorganize accumulated experience into a form that is compact, non-redundant, and useful for future tasks. Recent work has made substantial progress on individual components of language-agent memory [ 10 , 11 ] , including retrieval-augmented episodic stores [ 43 , 21 ] , structured memory systems [ 29 , 34 ] , procedural skill libraries [ 6 , 26 , 20 ] , reflection-based methods [ 23 , 16 ] , and RL-trained memory managers [ 37 , 32 , 35 , 30 ] . Despite this progress, two challenges remain. First, a consolidation problem: existing methods typically couple acquisition and consolidation into a single online update process, so each update is made with limited evidence from the current session. This makes it difficult to discover recurring patterns, abstract reusable procedures that generalize across sessions, resolve contradictions, or prune redundant entries. Second, a memory-utility problem: RL-trained memory methods optimize online construction or retrieval rather than offline consolidation under an explicit downstream utility objective, so they do not directly learn which memories are load-bearing, which entries are redundant, or how to trade off success against memory compactness. We take inspiration from complementary learning systems (CLS) theory of human memory, in which a fast hippocampal system encodes individual episodes and a slower neocortical system gradually extracts shared structure across episodes [ 18 , 12 , 17 ] . We adopt CLS not as a biological claim about language models, but as an operational design principle for separating fast acquisition from slow cross-session consolidation. We introduce Auto-Dreamer , a learned offline consolidator for language-agent memory. 1 1 1 Auto-Dreamer is distinct from the Dreamer family of world models [ 7 , 8 ] ; our method operates on memory entries and source trajectories, not the latent environment dynamics. Auto-Dreamer is the slow-timescale counterpart to a fast per-session writer. Given a typed memory bank produced by the writer, it performs a multi-step tool-use rollout: searching memory, inspecting candidate entries, retrieving raw source trajectories for provenance, and synthesizing new entries that abstract across sessions. Its core operation is region rewriting : the consolidator treats a selected working region as read-only evidence and synthesizes a fresh replacement set that supersedes the original region. This replacement semantics makes compactness structural rather than auxiliary: old entries do not persist by default, and information survives only if it is re-synthesized into the replacement set. As a result, abstraction, deduplication, contradiction resolution, and omission-based forgetting become default behaviors. We train Auto-Dreamer with GRPO [ 22 ] using a composite reward that combines downstream task performance with a counterfactual utility term estimated by random memory masking, which penalizes redundant entries while rewarding load-bearing ones. The task agent and the per-session writer remain fixed throughout training, isolating the contribution of the consolidator. We evaluate Auto-Dreamer in two regimes: continual-memory deployment, where the bank starts empty and grows over the task stream, and fixed-bank consolidation, where a pre-built bank is rewritten once. The results support three conclusions. First, Auto-Dreamer improves task success while maintaining substantially smaller active memory banks: in continual deployment, it achieves 41.1% success on ScienceWorld [ 28 ] , 7 points above the strongest baseline with 12 \times less memory; 60.2% on held-out ALFWorld [ 24 ] with 6 \times less memory than the strongest baseline; and 52.3% on held-out WebArena [ 44 ] , leading all baselines. Second, the learned consolidator transfers beyond its training distribution: although trained only on ScienceWorld trajectories, it improves performance on held-out ALFWorld and WebArena without further updates, including across a writer-backbone shift from Qwen3-14B [ 25 ] to Gemini-3.1-flash-lite-preview [ 4 ] . Third, controlled fixed-bank experiments and ablations show that the gains come from offline consolidation itself: region rewriting improves the quality of a given memory bank, while the counterfactual utility term suppresses redundant memories without sacrificing task performance. Our contributions are summarized as follows: • A two-timescale formulation of language-agent memory. We distinguish fast per-session acquisition from slow cross-session consolidation and formulate the latter as a learned decision problem over accumulated evidence. • Region rewriting as a compactness-inducing consolidation primitive. We formulate offline consolidation as provenance-grounded region rewriting: a selected working region is treated as read-only evidence and replaced by a synthesized replacement set. This differs from per-entry CRUD by making cross-session abstraction, deduplication, and omission-based forgetting the default update semantics. • RL training with region-local credit. Because region rewriting produces a self-contained replacement set, we can evaluate it directly and assign local credit from downstream task performance without supervised memory labels. We further use counterfactual masking to favor load-bearing memories and suppress redundant entries, improving the task utility of the compact bank. 2 Related works Memory systems for language agents. A growing body of work designs memory architectures for language agents. Early systems organize memory around atomic units or flat stores, such as A-MEM [ 34 ] , Mem0 [ 2 ] , MemOS [ 13 ] , and SimpleMem [ 15 ] . More recent work introduces richer typed memory spanning episodic, semantic, and procedural stores, including EverMemOS [ 9 ] , MIRIX [ 29 ] , Nemori [ 19 ] , and PlugMem [ 36 ] . A complementary line focuses on extracting reusable procedures or strategies from trajectories: Memp [