Multi-Agent Memory from a Computer Architecture Perspective: Visions and Challenges Ahead Conference: Architecture 2.0 Workshop on AI for Computing Systems Design; March 23, 2026; Pittsburgh, PA, USA

Zhongming Yu, Naicheng Yu, Hejia Zhang, Wentao Ni, Mingrui Yin, Jiaying Yang, Yujie Zhao, Jishen Zhao · 2026

Read Multi-Agent Memory from a Computer Architecture Perspective: Visions and Challenges Ahead Conference: Architecture 2.0 Workshop on AI for Computing Systems Design; March 23, 2026; Pittsburgh, PA, USA by Zhongming Yu, Naicheng Yu, Hejia Zhang, Wentao Ni, Mingrui Yin, Jiaying Yang, Yujie Zhao, Jishen Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10062.txt.

Introduction

Zhongming Yu and coauthors frame the memory of multi agent language model systems as a computer architecture problem. As agents begin to work in teams, their shared memory needs grow fast in complexity. The paper separates shared and distributed memory designs and proposes a three layer hierarchy of input output, cache and memory. Two protocol gaps stand out, cache sharing and structured access control, with memory consistency named the hardest open challenge.

Catalog record

CREATORZhongming Yu, Naicheng Yu, Hejia Zhang, Wentao Ni, Mingrui Yin, Jiaying Yang, Yujie Zhao, Jishen Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE8900
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10062

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Agent Memory from a Computer Architecture Perspective: Visions and Challenges Ahead Conference: Architecture 2.0 Workshop on AI for Computing Systems Design; March 23, 2026; Pittsburgh, PA, USA Zhongming Yu, Naicheng Yu, Hejia Zhang, Wentao Ni, Mingrui Yin, Jiaying Yang, Yujie Zhao, Jishen Zhao Abstract . As LLM agents evolve into collaborative multi-agent systems, their memory requirements grow rapidly in complexity. This position paper frames multi-agent memory as a computer architecture problem. We distinguish shared and distributed memory paradigms, propose a three-layer memory hierarchy (I/O, cache, and memory), and identify two critical protocol gaps: cache sharing across agents and structured memory access control. We argue that the most pressing open challenge is multi-agent memory consistency. Our architectural framing provides a foundation for building reliable, scalable multi-agent systems. 1. Introduction Large language model (LLM) agents ( 25 ; 20 ) are quickly moving from “single agent” tools ( 18 ) to multi-agent systems ( 6 ; 7 ) : tool-using agents ( 22 ) , planner–orchestrator stacks ( 10 ) , debate teams ( 14 ; 2 ) , and specialized sub-agents that collaborate to solve tasks ( 11 ; 19 ) . At the same time, the context these agents operate within is becoming more complex: longer histories, multiple modalities, structured traces, and customized environments. This combination creates a bottleneck that looks surprisingly familiar to computer architects: memory . In computer systems, performance and scalability are often limited not by compute but by memory hierarchy, bandwidth, and consistency. Multi-agent systems are heading toward the same wall—except their “memory” is not raw bytes, but semantic context used for reasoning. This position paper frames multi-agent memory as a computer architecture problem and highlights key protocol and consistency gaps. Figure 1. Two fundamental multi-agent memory architectures for managing growing context complexity: shared memory and distributed memory. Overview graphic motivating multi-agent memory challenges. 2. Why Memory Matters: Context Is Changing LLM evaluations show that “real” context ability involves more than simple retrieval; it requires multi-hop tracing, aggregation, and sustained reasoning as context length scales. Multimodal benchmarks add images, diagrams, and videos. Structured tasks introduce executable traces and schemas. Interactive environments make environment state + execution part of the memory problem. The result is not a static prompt but a dynamic, multi-format, partially persistent memory system. \bullet Longer context windows: Suites like RULER ( 8 ) emphasize reasoning over long histories, not just retrieval. \bullet Multimodal inputs: Benchmarks such as MMMU ( 28 ) and VideoMME ( 4 ) require joint reasoning over images and videos. \bullet Structured data & traces: Text-to-SQL datasets like Spider ( 27 ) and BIRD ( 12 ) show that agents increasingly operate over structured, executable traces. \bullet Customized environments: Evaluations such as SWE-bench ( 9 ) and OSWorld ( 23 ) stress long-horizon state tracking and grounded actions. As such, context is no longer a static prompt; it is a dynamic memory system with bandwidth, caching, and coherence constraints. 3. Shared vs. Distributed Agent Memory Here we name two basic prototypes that mirror classical memory systems. In shared memory , all agents access a shared pool (e.g., a shared vector store or document database). In distributed memory , each agent owns local memory and synchronizes selectively. Shared memory makes knowledge reuse easy but requires coherence support; without coordination, agents overwrite each other, read stale information, or rely on inconsistent versions of shared facts. Distributed memory improves isolation and scalability but requires explicit synchronization; state divergence becomes common unless carefully managed. Most real systems sit between these extremes: local working memory with selectively shared artifacts. Figure 2. Agent memory hierarchy and protocol framing. Diagram of agent I/O, cache, and memory layers with protocol links. 4. An Architecture-Inspired Memory Hierarchy Computer architecture teaches a practical lesson: systems are not designed around “one memory.” Instead, they are built as memory hierarchies with layers optimized for latency, bandwidth, capacity, and persistence. A useful mapping for agents is as follows: Agent I/O layer: Interfaces that ingest and emit information (audio, text documents, images, network calls). Agent cache layer: fast, limited-capacity memory for immediate reasoning (compressed context, recent tool calls, short-term latent storage such as KV caches and embeddings). Agent memory layer: large-capacity, slower memory optimized for retrieval and persistence (full dialogue history, vector DBs, graph DBs, and document stores). This framing emphasizes a key principle: agent performance is an end-to-end data movement problem . If relevant information is stuck in the wrong layer (or never loaded), reasoning accuracy and efficiency degrade. As in hardware, caching is not optional. 5. Protocol Extensions for Multi-Agent Scenarios Architecture layers need protocols . Many systems rely on connectivity protocols, but inter-agent bandwidth remains limited by message passing. This layer is best viewed as agent context I/O , e.g. MCP ( 1 ) . That is necessary—but not sufficient. Two missing pieces stand out. Missing piece 1: Agent cache sharing protocol. Recent work explores KV cache sharing ( 13 ; 5 ; 26 ) , but we lack a principled protocol for sharing cached artifacts across agents. The goal is to enable one agent’s cached results to be transformed and reused by another, analogous to cache transfers in multiprocessors. Missing piece 2: Agent memory access protocol. Agentic memory frameworks ( 15 ; 24 ; 3 ; 17 ) propose many strategies for maintaining and optimizing LLM agents’ memory. Yet even when some frameworks support shared state, the standard access protocol (permissions, scope, granularity) remains under-specified. Key questions include: Can one agent read another’s long-term memory? Is access read-only or read-write? What is the unit of access: a document, a chunk, a key-value record, or a trace segment? Figure 3. Consistency model comparison from traditional memory architecture to multi-agent memory. Comparison chart of memory consistency models. 6. The Next Frontier: Multi-Agent Consistency In computer architecture, memory consistency models ( 21 ) specify the order in which memory operations issued by one processor become visible to others—governing not what value a read returns, but when and in what order concurrent writes may be observed. Decades of systems research have produced a precise hierarchy of such models (SC, TSO, Release Consistency) together with formal verification techniques. Agent memory systems face an analogous challenge, yet no equivalent formalism exists. For a single LLM agent, consistency requires that updates to memory—across working, episodic, and semantic stores—propagate in a causally order. Prior work on generative agents ( 16 ) acknowledges the need for temporally coherent retrieval, but does not formalize the ordering guarantees under which retrieval is correct. The problem compounds in multi-agent settings, where multiple agents concurrently read from and write to shared memory. Here, consistency decomposes into two requirements. Update-time visibility and ordering determines when an agent’s writes become observable to others, and in what order concurrent writes from different agents may be observed—directly analogous to the ordering contracts of hardware consistency models. Read-time conflict resolution governs how an agent should reconcile conflicting or stale artifacts that arise when records evolve across versions under concurrent revision. Both requirements are harder than their hardware counterparts: memory artifacts are semantically heterogeneous (evidence, tool traces, plans), inter-agent dependencies are implicit rather than declared, and conflicts are often semantic and coupled to environment state. Despite this complexity, multi-agent

Read the complete text