MEM1 : Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents

Zijian Zhou, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, Paul Pu Liang · 2025

Read MEM1 : Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents by Zijian Zhou, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, Paul Pu Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15841.txt.

Introduction

Zijian Zhou and co-authors present a 2025 paper on memory for long-running language agents. Agents that search, watch, and answer over many turns often keep every past turn. That makes memory grow without bound, costs more, and can hurt reasoning. Their system, MEM1, keeps a compact inner state instead of the full history.

At each turn, the state merges old memory with new facts and drops what is stale. Training used tasks built by joining existing data sets into long chains. On a hard multi-hop task, a small MEM1 model beat a model twice its size by a wide margin. It also used far less memory. The skill held past the training length.

Catalog record

CREATORZijian Zhou, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, Paul Pu Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15841

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MEM1 : Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents Zijian Zhou, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, Paul Pu Liang Abstract Modern language agents must operate over long-horizon, multi-turn interactions, where they retrieve external information, adapt to observations, and answer interdependent queries. Yet, most LLM systems rely on full-context prompting, appending all past turns regardless of their relevance. This leads to unbounded memory growth, increased computational costs, and degraded reasoning performance on out-of-distribution input lengths. We introduce MEM1 , an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks. At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning. This state integrates prior memory with new observations from the environment while strategically discarding irrelevant or redundant information. To support training in more realistic and compositional settings, we propose a simple yet effective and scalable approach to constructing multi-turn environments by composing existing datasets into arbitrarily complex task sequences. Experiments across three domains, including internal retrieval QA, open-domain web QA, and multi-turn web shopping, show that MEM1 -7B improves performance by 3.5 × 3.5\times while reducing memory usage by 3.7 × 3.7\times compared to Qwen2.5-14B-Instruct on a 16 16 -objective multi-hop QA task, and generalizes beyond the training horizon . Our results demonstrate the promise of reasoning-driven memory consolidation as a scalable alternative to existing solutions for training long-horizon interactive agents, where both efficiency and performance are optimized. 0 0 footnotetext: * Equal contribution. Correspondence: [email protected], [email protected] 1 Introduction Large language models (LLMs) have shown remarkable performance in single-turn tasks such as question answering, summarization, and code generation [ 7 , 48 , 3 ] . However, emerging real-world applications increasingly operate over multiple turns—searching documents, interacting with environments [ 67 ] , and making decisions based on evolving external information [ 50 ] . Examples include research agents such as OpenAI and Gemini Deep Research [ 34 , 16 ] that automate complex tasks by iteratively gathering information, and web-navigation agents such as OpenManus [ 47 ] and BrowserUse [ 31 ] , which must complete goals across dozens of interactive turns. Unlike traditional tasks where the input is static or self-contained, long-horizon settings often involve answering a sequence of related questions, requiring the agent to continuously retrieve new information, revise beliefs, and adapt to evolving contexts over time. For instance, consider a research assistant tasked with “What’s the evidence for X?”. Subsequent queries like “Who published it?” require further information retrieval, while “Is the source credible?” calls for self-reflection and assessment. Each query builds on the previously collected and accumulated information. Similarly, a shopping assistant may be first asked “Which product is cheapest?”, then “What are its reviews?”, and “Is it compatible with my device?”. These interactions span multiple turns, featuring evolving contexts and compound reasoning. In systems designed for long-horizon settings, a common approach is to append all past observations, actions, and thoughts to the context at every turn [ 52 , 58 ] . This forces the model to operate with an unboundedly growing context, which introduces three key challenges. (1) Growing inference cost and memory usage . Transformer-based LLMs typically incur O(N^{2}) compute cost (or O(N) with Key-Value caching) and O(N) memory usage as the context length N increases [ 49 ] . Consequently, deploying these models requires reserving large GPU memory on modern inference frameworks to accommodate the growing context [ 25 , 65 ] , often leading to significant wastage of computing resources. (2) Generalization limits beyond the training horizon . Ongoing conversations with context length exceeding that in the training data become out-of-distribution for the model. The model struggles to manage and reason over such unfamiliar long-horizon inputs [ 60 ] . (3) Overloaded and inefficient context . The accumulation of irrelevant or redundant content dilutes the model’s attention. This reduces its ability to reason effectively, even when relevant information is technically still present within the prompt [ 2 , 27 , 53 ] . Recent progress in long-context modeling largely targets static inputs ( e.g., long documents) and does not address multi-turn interaction with external environments [ 6 , 17 ] . Some other approaches introduce external memory modules ( e.g., summarizers or retrievers) [ 60 , 26 , 11 , 54 ] , but these are typically trained separately and cannot be optimized end-to-end with the agent’s policy. This also introduces additional engineering overhead, as engineers must manage and integrate two separate models. Meanwhile, existing works on tool-using agent systems trained with reinforcement learning leave memory management unsolved, letting the prompt length grow unboundedly [ 22 , 66 ] . A natural question is raised: Can a language model learn to consolidate its memory as part of its reasoning process so that it retains only what is essential for solving the task? Figure 1 : Comparison of memory management between MEM1 and existing reasoning agents. While existing agents for long-horizon tasks [ 22 , 61 , 66 ] continuously append thoughts (typically enclosed in <think></think>), actions, and observations, resulting in an ever-growing context, our MEM1 agent learns to keep updating an internal state (enclosed in <IS></IS>) that blends thought and memory, discarding the contents from previous steps to achieve constant memory usage during the task. On the other hand, while existing environments and datasets focus on single-objective tasks, our task augmentation method effectively scales up these tasks to enable long-horizon agent training. Motivated by this question, we present MEM1 : M emory- E fficient M echanism via learning 1 -step integrated reasoning and consolidation—a method for training LLM agents that maintain constant memory usage across arbitrarily long horizons. Figure 2 : A conceptual comparison of context length between the MEM1 agent and existing reasoning agents when handling long-horizon tasks. Our agent learns to discard the previous context (except for the prompt and initial query) immediately after generating a new internal state and action, resulting in near-constant memory usage. As illustrated in Fig. 1 , at each turn, the model updates a consolidated state composed of prior memory and newly obtained information. This consolidated state becomes the agent’s only retained memory, allowing all external tool outputs to be discarded after use, which prevents prompt expansion altogether as illustrated by Fig. 2 . A key insight of our method is that inference-time reasoning [ 52 , 13 , 30 , 62 ] serves a dual function: it not only provides deeper insight into the current query but also acts as a form of “working memory” [ 4 ] , extracting key components from gathered information to build an evolving understanding. By unifying reasoning and memory consolidation, MEM1 offers an elegant solution where the agent learns to reason and remember within a shared representational space without requiring additional modules or architectural changes. We train this behavior end-to-end with reinforcement learning (RL) [ 45 , 68 ] , optimizing for task success via verifiable rewards [ 40 ] . Although not explicitly optimized for memory efficiency through reward signals, the agent learns to manage memory as part of its policy, resulting in near-constant memory usage across long horizons. Additionally, we

Read the complete text