HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning

Zeju Li, Ziyang Zheng, Yizhou Zhou†, Qiang Xu† · 2026

Read HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning by Zeju Li, Ziyang Zheng, Yizhou Zhou†, Qiang Xu† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28349.txt.

Introduction

Zeju Li, Ziyang Zheng, Yizhou Zhou, and Qiang Xu built HMARS, a memory system that helps language models reason over very long material. Standard retrieval fetches a few text chunks and can discard the decisive evidence before reasoning even starts, especially when relevance only becomes clear in wider context.

HMARS instead manages long contexts like organized memory. Sub-agents guard bounded regions, mid-level agents coordinate regional context for each query, and a frontier model reasons over the evidence pages they return. The team built two diagnostic benchmarks stressing evidence breadth and how relevance depends on context. Across long-document and multi-turn tasks, HMARS beat retrieval, reranking, full-context, graph-based, and agentic rivals. Analysis showed the wins came from gathering the needed evidence more completely.

Catalog record

CREATORZeju Li, Ziyang Zheng, Yizhou Zhou†, Qiang Xu†
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28349

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning Zeju Li, Ziyang Zheng, Yizhou Zhou†, Qiang Xu† Abstract Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across documents, dialogues, and accumulated interaction histories. Standard retrieval-augmented generation reduces this problem to top- K K chunk retrieval, but such passive access can discard relevant evidence before reasoning begins, especially when relevance depends on broader context. We propose HMARS , a hierarchical multi-agent memory system that treats long contexts as managed memory rather than a flat retrieval corpus. Sub-agents maintain grounded access to bounded memory regions, mid-agents manage regional context and provide query-specific coordination, and a frontier model performs final reasoning over retrieved evidence pages. To evaluate this view, we construct two diagnostic benchmarks targeting evidence breadth and context-dependent relevance. Across long-document and multi-turn memory tasks, HMARS achieves the best overall performance against retrieval, reranking, full-context, graph-based, and agentic long-context baselines. Evidence coverage analysis further shows that its gains come from retrieving the required supporting evidence more completely, rather than merely changing the final answer prompt. 1 Introduction Large language models ( Yang et al. 2025 ; Comanici et al. 2025 ; Achiam et al. 2023 ; Singh et al. 2026a ) are increasingly expected to reason over long documents, multi-turn dialogues, and accumulated interaction histories. However, simply expanding context windows does not solve long-context reasoning: inference remains expensive, attention over long inputs is unreliable, and relevant evidence is often buried among distractors. Retrieval-augmented generation (RAG) ( Jiang et al. 2024 ; Lewis et al. 2020 ) remains the default solution ( Bai et al. 2024 ; Bai et al. 2025 ) : it chunks documents, embeds chunks independently, retrieves the top- K , and passes them to a generator. This passive interface is efficient, but assumes relevance is local and statically measurable. The assumption breaks when evidence is scattered across distant regions or becomes relevant only under broader context—for example, an early “$2,500 budget” may be essential to a later question about “housing constraints” despite low lexical overlap. RAG stores context; it does not provide a memory system for managing, accessing, and reasoning over evidence. Figure 1: Passive RAG chunks documents, embeds chunks independently, and retrieves top- K evidence, but may miss context-dependent evidence. HMARS treats long inputs as managed memory, where sub-agents assess local regions, mid-agents coordinate shared context, and a frontier model performs final reasoning. Recent work has begun to move beyond passive retrieval, but still falls short of a memory system for long-context reasoning. Memory-augmented agents ( Chhikara et al. 2025 ; Xu et al. 2026 ; Zhou et al. 2025 ; Zhang et al. 2026 ; Sun et al. 2026 ) mainly study what to store, update, compress, or forget over long-horizon interactions. Long-context retrieval and reading methods ( Jiang et al. 2024 ; Sarthi et al. 2024 ; Lee et al. 2024 ) improve access through larger chunks, summary hierarchies, or page lookup, but remain passive or centralized: some module still decides which parts of the input are allowed to reach the reasoner. What is missing is a memory system that organizes long inputs into local regions, lets each region participate in query-conditioned access, retrieves grounded evidence, and leaves final reasoning to a stronger model. We propose HMARS , a Hierarchical Multi-Agent Memory System for long-context reasoning. HMARS treats long documents and interaction histories as managed memory rather than a flat chunk corpus. Lightweight sub-agents retain grounded access to bounded memory regions, mid-agents manage shared regional context and provide query-specific coordination, and a frontier base model performs final reasoning over retrieved evidence. Unlike top- K retrieval, HMARS does not pre-filter away document regions: every sub-agent performs query-conditioned assessment over its local memory, while mid-agent guidance helps retrieve focused, grounded evidence, shown in Figure 1 . To evaluate this memory-system view, we construct two diagnostic benchmarks that stress evidence breadth and context-dependent relevance rather than short sequential hops. The first is derived from LongBench-v2 ( Bai et al. 2025 ) , with questions filtered or generated to require evidence across distant chunks and low RAG recall; the second is a cumulative interaction benchmark inspired by RealMem ( Bian et al. 2026 ) , where key facts are scattered across multi-turn interactions and become relevant only under later context. Across these settings, HMARS improves most where passive top- K retrieval and centralized memory access fail, supporting our core claim: long-context reasoning should be framed not only as retrieving more chunks, but as managing, accessing, retrieving, and reasoning over memory. Our contributions are threefold. First, we build harder diagnostic benchmarks for long-context reasoning and multi-turn interaction memory, where questions require broader evidence coverage across distant chunks, regions, or turns. Second, we frame these challenges as failures of passive chunk-based memory access, especially under evidence breadth and context-dependent relevance. Third, we introduce HMARS , a hierarchical multi-agent memory system that separates local memory access, mid-level coordination, grounded evidence retrieval, and frontier-model reasoning. 2 Related Work 2.1 Long-Context Retrieval and Hierarchical Reading Retrieval-augmented generation remains the dominant interface for long-context QA ( Singh et al. 2026b ; Jin et al. 2025 ; Gupta et al. 2024 ) . Recent work improves this pipeline through adaptive retrieval and critique ( Asai et al. 2024 ) , retrieval correction, larger retrieval units ( Jiang et al. 2024 ) , hierarchical summary trees ( Sarthi et al. 2024 ) , or gist memory with page lookup ( Lee et al. 2024 ) . These methods improve access to long context, but still rely on passive or centralized selection: a retriever selects chunks, a summary tree is searched, or a single reader controls lookup. HMARS instead treats long context as an agent-managed memory hierarchy, where local regions participate in query-conditioned assessment and return grounded pages without early top- K exclusion. 2.2 Multi-Agent Long-Context Processing Multi-agent frameworks have also been explored for long-context reasoning ( Wang and Chen 2025 ; Jiang et al. 2025 ; Zhao et al. 2024 ) . Chain-of-Agents ( Zhang et al. 2024 ) , for example, assigns chunks to sequential worker agents that pass compressed communication units before a manager produces the final answer. These methods show that agent collaboration can reduce input bottlenecks, but they still treat long-context processing primarily as sequential reading : each query triggers a pass over the input, and information is propagated through compressed messages. HMARS instead treats the input as a persistent memory system: sub-agents retain grounded page access to local regions, mid-agents provide shared regional context and query-specific coordination, and the base model reasons over retrieved evidence. 2.3 Memory-Augmented LLM Agents A growing line of work studies explicit memory systems for LLM agents. Mem0 ( Chhikara et al. 2025 ) extracts and retrieves salient conversational memories; A-Mem ( Xu et al. 2026 ) organizes memories as dynamically linked notes; MEM1 ( Zhou et al. 2025 ) jointly learns memory consolidation and reasoning; HiMem ( Zhang et al. 2026 ) builds hierarchical Episode and Note memories; and reflective or learned memory managers study how agents should update, compress, or delete memories over time ( Tan et al. 2025 ; Yan et al. 2025 ; Yu et al. 2025 ) . While these works

Read the complete text