Aeon : High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents
Mustafa Arslan · 2026
Read Aeon : High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents by Mustafa Arslan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15311.txt.
Introduction
Mustafa Arslan gives long-running language model agents a memory that acts less like a pile of notes. His 2026 arXiv paper deals with a core limit of these models. Self-attention costs grow with the square of the context length, and quality fades in the middle of long inputs.
Most fixes use flat retrieval over a vector store. That treats memory as an unstructured bag of embeddings. Arslan presents Aeon, a neuro-symbolic memory manager for long-horizon agents. It aims for high performance by giving the stored past a structure the agent can reason over.
Catalog record
| CREATOR | Mustafa Arslan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-15311
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Aeon : High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents Mustafa Arslan Abstract Large Language Models (LLMs) are fundamentally constrained by the quadratic computational cost of self-attention and the “Lost in the Middle” phenomenon, where reasoning capabilities degrade as context windows expand. Existing solutions, primarily “Flat RAG” architectures relying on vector databases, treat memory as an unstructured bag of embeddings. This approach fails to capture the hierarchical and temporal structure of long-horizon interactions, leading to “Vector Haze”—the retrieval of disjointed facts lacking episodic continuity. We propose Aeon , a Neuro-Symbolic Cognitive Operating System that redefines memory not as a static store, but as a managed OS resource. Aeon structures memory into a Memory Palace (a spatial index implemented via Atlas , a SIMD-accelerated Page-Clustered Vector Index that combines small-world graph navigation with B+ Tree-style disk locality to minimize read amplification) and a Trace (a neuro-symbolic episodic graph). We introduce the Semantic Lookaside Buffer (SLB) , a predictive caching mechanism that exploits conversational locality to achieve sub-millisecond retrieval latencies. Benchmarks demonstrate that Aeon achieves < 1 <1 ms retrieval latency on conversational workloads while ensuring state consistency via a zero-copy C++/Python bridge, effectively enabling persistent, structured memory for autonomous agents. 1 Introduction The rapid evolution of Large Language Models (LLMs) has been defined by a relentless scaling of parameters and training data, yet the fundamental architecture remains bound by the Context Bottleneck . The Transformer’s self-attention mechanism, while transformative, imposes a quadratic time and space complexity, O(N^{2}) , relative to the input sequence length. Although recent optimization techniques—such as sparse attention, RingAttention, and hardware-aware kernel fusion—have theoretically extended context windows to 1 million tokens and beyond, the utility of this context does not scale linearly. Empirical evidence highlights a distinct degradation in reasoning capabilities over these extended horizons, a phenomenon widely characterized as being “Lost in the Middle” [ 8 ] . When relevant information is buried in the center of a massive context window, model performance deteriorates often below the baseline of closed-book inference. As autonomous agents are tasked with increasingly complex, long-horizon objectives spanning days or weeks, the reliance on these transient, volatile context windows becomes untenable. The model cannot simply attend to all of history; it must select what is potentially relevant before attention is even applied. However, the selection mechanism itself is often limited by the same architectural constraints it seeks to mitigate. We argue that the simplistic view of context as a sliding window is insufficient for general-purpose autonomy. Instead, we require a memory architecture that is as persistent, structured, and deterministically managed as the storage hierarchies found in classical operating systems. The prevailing industry response to the context limitation has been the widespread adoption of Retrieval-Augmented Generation (RAG). In its most common form, “Flat RAG,” this approach offloads information preservation to vector databases that perform Approximate Nearest Neighbor (ANN) search over unstructured lists of embeddings. Technologies such as HNSW (Hierarchical Navigable Small World) graphs or inverted file indices allow for efficient retrieval of semantically similar chunks. However, while effective for simple, one-shot question-answering tasks, Flat RAG fails to model the structure of extended interaction. It treats memory as a featureless plane—a “bag of vectors”—where the temporal evolution of a conversation, the causal lineage of decisions, and the hierarchical relationship between concepts are lost. We term this failure mode “Vector Haze”: the retrieval of semantically similar but episodically disjointed facts that confuse rather than aid the agent. A Flat RAG system has no notion of “where” it is in a conversation; it only knows “what” matches the current query vector in high-dimensional space. It lacks the ability to backtrack to a previous state, to branch a conversation into potential futures, or to understand the narrative arc that led to the current moment. For an agent to maintain coherence over thousands of turns, it requires more than a semantic search engine; it requires a state machine that understands the topology of its own experience. We propose a paradigm shift from treating memory as a passive database retrieval problem to treating it as an active resource management problem within a Cognitive Operating System . In this view, memory management becomes analogous to virtual memory management in traditional OS kernels. We introduce Aeon , a system that formalizes these operations. Allocation corresponds to the deliberate writing of new semantic concepts into a structured Atlas ; paging transforms into the loading of relevant semantic clusters into a Semantic Lookaside Buffer (SLB) for immediate, low-latency access; and context switching is re-framed as the deterministic movement between branches of a decision tree. By enforcing these rigorous abstractions, Aeon transforms the probabilistic chaos of vector search into a deterministic navigational process. This empowers the agent to utilize a “Memory Palace”—a spatial index where information is stored not just by what it means, but by where it belongs relative to other concepts in the agent’s ontology. This spatial locality enables the system to predictively pre-fetch memory, much like a CPU pre-fetches instructions based on spatial and temporal locality, drastically reducing the latency penalty of retrieval and allowing the agent to "think" at the speed of conversation. Our contributions in this paper are threefold. First, we introduce Atlas , a high-performance, memory-mapped B+ Tree that organizes uniform vectors into a navigable, hierarchical index. Unlike HNSW or IVFPQ indices which optimize purely for recall at the expense of insert performance and structure, Atlas optimizes for semantic locality and stable modification. It utilizes a custom SIMD-accelerated math kernel to perform metric comparisons, ensuring that the tree structure strictly adheres to the geometry of the embedding space. Second, we present the Trace , a neuro-symbolic directed acyclic graph (DAG) that explicitly tracks the agent’s episodic state. The Trace records the traversal path of the agent, creating distinct nodes for User inputs, System responses, and intermediate thoughts, linked by typed edges (CAUSAL, NEXT, REFERS_TO). This enables capabilities such as backtracking and context anchoring that are impossible in flat vector stores. Finally, we demonstrate the implementation of a zero-copy architecture bridging C++23 and Python. By harnessing the nanobind library, Aeon exposes internal C++ memory structures as read-only NumPy arrays, eliminating serialization overhead. This allows Aeon to achieve retrieval latencies of <1 ms on standard conversational workloads, validating the feasibility of this Cognitive OS approach for real-time, interactive agents that require both the speed of systems programming and the flexibility of high-level reasoning. 2 Related Work We position Aeon within the broader landscape of neural memory systems, contrasting its Cognitive Operating System architecture against existing approaches in retrieval, memory management, and neuro-symbolic reasoning. While individual components of Aeon have precedents in isolation, we argue that the lack of a unified, high-performance kernel has limited the emergence of truly long-horizon agents. 2.1 Retrieval-Augmented Generation (RAG) The dominant paradigm for grounding Large Language Models (LLMs) is Retrieval-Augmented Generation (RAG), typically implemented using Dense Passage Retrieval (DPR) [ 7 ] and Approximate Nearest Neighbor (ANN) search indices like FAISS [ 5 ] or HNSW [ 9 ] . These systems rely on what we characterize as "Flat RAG": a single, monolithic vector space where every query is treated as an independent event, disconnected from temporal or causal context. The primary limitation of Flat RAG