HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents
Jinchang Zhu, Jindong Li, Cheng Zhang, Jiahong Liu, Menglin Yang · 2026
Read HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents by Jinchang Zhu, Jindong Li, Cheng Zhang, Jiahong Liu, Menglin Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16839.txt.
Introduction
Zhu and colleagues propose HeLa-Mem, a bio memory architecture treating memory as a dynamic Hebbian graph. Fixed context windows cannot keep coherence across long chats, and old systems store history as loose embedding vectors. Three life tricks are central yet largely absent: association, consolidation and spreading firing. An episodic graph evolves through co-firing patterns, while a semantic store fills via Hebbian boiling of densely linked hubs. On LoCoMo, HeLa-Mem beats prior work across four question kinds while using far fewer context tokens.
Catalog record
| CREATOR | Jinchang Zhu, Jindong Li, Cheng Zhang, Jiahong Liu, Menglin Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37831 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16839
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents Jinchang Zhu, Jindong Li, Cheng Zhang, Jiahong Liu, Menglin Yang Abstract Long-term memory is a critical challenge for Large Language Model agents, as fixed context windows cannot preserve coherence across extended interactions. Existing memory systems represent conversation history as unstructured embedding vectors, retrieving information through semantic similarity. This paradigm fails to capture the associative structure of human memory, wherein related experiences progressively strengthen interconnections through repeated co-activation. Inspired by cognitive neuroscience, we identify three mechanisms central to biological memory: association , consolidation , and spreading activation , which remain largely absent in current research. To bridge this gap, we propose HeLa-Mem , a bio-inspired memory architecture that models memory as a dynamic graph with Hebbian learning dynamics. HeLa-Mem employs a dual-level organization: (1) an episodic memory graph that evolves through co-activation patterns, and (2) a semantic memory store populated via Hebbian Distillation, wherein a Reflective Agent identifies densely connected memory hubs and distills them into structured, reusable semantic knowledge. This dual-path design leverages both semantic similarity and learned associations, mirroring the episodic-semantic distinction in human cognition. Experiments on LoCoMo demonstrate superior performance across four question categories while using significantly fewer context tokens (Figure 1 ). Code is available on GitHub . 1 Introduction Large language models have demonstrated remarkable capabilities in language understanding and generation, enabling increasingly sophisticated interactive agents ( 24 ; 15 ) . However, sustaining coherent behavior over long time horizons remains a fundamental challenge ( 25 ) . Due to their reliance on fixed-length context windows, LLMs struggle to maintain consistent representations of past interactions as dialogues extend or span multiple sessions. This limitation often leads to fragmented memory ( 21 ; 6 ; 12 ; 14 ) , resulting in factual inconsistencies, diminished personalization, and unstable agent behavior ( 11 ) . Addressing long-term memory coherence is therefore essential for LLM agents operating in settings that require persistent user adaptation, multi-session knowledge retention, or stable persona maintenance. Figure 1: Performance vs. token efficiency on LoCoMo, averaged across GPT-4o-mini and GPT-4o. HeLa-Mem achieves strong performance with fewer tokens, landing in the upper-left ideal region. Current memory mechanisms for LLM agents can be broadly categorized into three methodological paradigms. Knowledge-organization methods , such as A-Mem ( 22 ) , structure memory into interconnected semantic networks to enable adaptive management. Retrieval mechanism-oriented approaches , exemplified by MemoryBank ( 26 ) , integrate semantic retrieval with memory forgetting curves for long-term updating. Architecture-driven methods , including MemGPT ( 18 ) , employ hierarchical memory structures with explicit read and write operations to dynamically manage limited context windows. Despite their demonstrated effectiveness, these approaches are typically developed in isolation, each prioritizing a single dimension—memory structure, retrieval strategy, or update mechanism—while largely overlooking their mutual interaction and joint contribution to long-term coherence. More fundamentally, this component-wise optimization often overlooks a critical aspect of long-term coherence: the dynamic evolution of memory structure. Human memory is not a static database where items are stored and retrieved in isolation; rather, it is a dynamic system where connections are continuously reorganized by experience. For example, a topic discussed today might trigger a memory from a month ago, not necessarily because they share surface-level keywords, but because they are part of the same evolving narrative arc. Current systems, by treating storage and retrieval as separate static processes, fail to capture this evolving connectivity, leading to agents that “remember” facts but lack the “continuity” of a developing relationship. To better understand how long-term coherence can be maintained, we draw on a fundamental principle of biological memory: Hebbian learning . In biological systems, experiences that are repeatedly co-activated gradually develop stronger associations, a phenomenon often summarized as “neurons that fire together wire together” (illustrated in Figure 2 ). This associative organization allows related memories to be efficiently reactivated through spreading activation and supports the gradual consolidation of episodic experiences into more stable semantic knowledge. Together, association, consolidation, and spreading activation form a tightly coupled memory process that enables biological systems to maintain coherent representations over extended time scales—capabilities that remain largely absent from current artificial memory designs. Building on this perspective, we propose HeLa-Mem ( He bbian L earning a ssociative Mem ory), a unified memory architecture for LLM agents. HeLa-Mem represents conversational history as a dynamic graph with Hebbian learning dynamics and operates through coordinated mechanisms of online association, reflective consolidation, and dual-path retrieval, establishing a unified memory management framework that captures both fine-grained details and high-level patterns. The primary contributions of our work are: • We propose HeLa-Mem , a bio-inspired framework that utilizes an Online Encoding & Association mechanism to model conversation history as a dynamic Hebbian graph, where co-activated memories strengthen connections to capture latent context. • We introduce a Reflective Consolidation framework using Hebbian Distillation , which identifies hub clusters and transforms them into structured semantic knowledge, preventing graph explosion while retaining key information. • We implement a Dual-Path Retrieval strategy that leverages spreading activation to traverse Hebbian edges, achieving the best average rank (1.25) across all question categories. • Comprehensive experiments on the LoCoMo benchmark validate HeLa-Mem’s effectiveness, achieving superior performance across four categories while using significantly fewer context tokens. Figure 2: Conceptual illustration of Hebbian learning in associative memory. Two memory nodes (Neuron A and B) representing distinct experiences (e.g., a daytime event and a nighttime event) develop strengthened synaptic connections when co-activated through shared context. This "neurons that fire together, wire together" principle forms the theoretical foundation of HeLa-Mem’s dynamic memory graph. 2 Related Work 2.1 Memory for LLM Agents Figure 3: The architectural overview of HeLa-Mem. The framework consists of three modules: (1) Hebbian Association for dynamic graph construction (Section 3.2 ); (2) Reflective Consolidation for semantic knowledge distillation (Section 3.3 ); and (3) Retrieval and Response using a Dual-Path strategy (Section 3.4 ). Existing Large Language Models face fundamental challenges in handling complex scenarios requiring long-term coherence ( 6 ) . Advancements in memory systems addressing this problem can be broadly grouped into three categories. Knowledge-organization methods focus on capturing and structuring intermediate reasoning states. Think-in-Memory ( 16 ) stores evolving chains-of-thought, enabling consistency through continual updates. A-Mem ( 22 ) organizes knowledge into an interconnected note network that spans sessions. Retrieval mechanism-oriented approaches, pioneered by RAG ( 10 ) , enrich the model with external memory libraries. MemoryBank ( 26 ) logs conversations, events, and user traits in a vector database and refreshes them using a forgetting-curve schedule. Generative Agents ( 19 ) keep memories in natural language and add a reflection loop for relevance filtering. EmotionalRAG ( 7 ) retrieves memory entries by combining semantic similarity with the agent’s emotional state. Architecture-driven designs alter the core control flow to manage context explicitly. MemGPT ( 18 ) adopts an OS-like hierarchy with dedicated read/write calls. SCM ( 13 ) introduces dual buffers and a memory controller that gates selective recall. Mem0 ( 2 ) dynamically extracts and consolidates salient information for scalable long-term memory. MemoryOS ( 8 ) introduces a three-tier hierarchical storage with short-term, mid-term, and long-term memory units, employing segment-page organization for dynamic updating. 2.2 Hebbian Learning in Neural Networks Hebbian learning, summarized as “neurons that