The Price of Meaning: Why Every Semantic Memory System Forgets

Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan, Ashwin Gopinath · 2026

Read The Price of Meaning: Why Every Semantic Memory System Forgets by Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan, Ashwin Gopinath (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27116.txt.

Introduction

Ray Barman and co-authors prove that semantic memory systems must forget, as a matter of geometry. Organizing memory by meaning enables analogy and conceptual retrieval. For kernel-threshold memories over semantic feature spaces, they derive four limits. Useful representations have finite effective rank, and finite local dimension forces rival memories nearby. As memory grows, retention decays to zero in power-law fashion. False recall of associative lures cannot be tuned away, either. Tests across five architectures confirm the pattern. Systems escaping interference do so by giving up semantic generalization.

Catalog record

CREATORSambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan, Ashwin Gopinath
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56455
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27116

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Price of Meaning: Why Every Semantic Memory System Forgets Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar, Nikhil Narasimhan, Ashwin Gopinath Abstract Every major AI memory system in production today, from vector databases to RAG pipelines to the weights of large language models, organises information by meaning. That organisation is what makes these systems useful: it lets them generalise, draw analogies, and retrieve by concept rather than by keyword. But it comes at a price. We show that the same geometric structure that enables semantic generalisation also makes interference, forgetting, and false recall inescapable. Here we formalise and test that tradeoff for a broad class of semantically continuous kernel-threshold memories : systems whose retrieval score is a monotone function of an inner-product in a semantic feature space, whose representations are learned under a rate or distortion budget, and whose semantic manifold has finite local intrinsic dimension. Within this class, we derive four results. First, semantically useful representations have finite semantic effective rank. Second, finite local dimension implies positive competitor mass in retrieval neighbourhoods. Third, under growing memory, retention decays to zero; with power-law arrival statistics and population heterogeneity, this yields population-level power-law forgetting curves. Fourth, for associative lures satisfying a δ \delta -convexity condition below the decision margin, false recall cannot be eliminated by threshold tuning within the same score family. We then test these predictions across five memory architectures: vector retrieval, graph memory, attention-based retrieval, BM25-based filesystem retrieval, and parametric memory. Pure semantic retrieval systems express the geometric vulnerability directly as forgetting and false recall. Systems with explicit reasoning can partially override these symptoms behaviourally, but convert smooth degradation into brittle failure modes. Systems that escape interference completely do so by sacrificing semantic generalisation. The result is not an argument against scale. It is an argument that scale alone is not enough. Making a vector database ten times larger, an LLM ten times bigger, or an embedding space ten times wider does not remove the interference; it moves the system along a tradeoff surface where forgetting and usefulness are coupled. For memory, progress requires not only scale but new architectures, training objectives, and interference-management mechanisms. The price of meaning is interference, and no architecture we tested avoids paying it. Organising memory by meaning makes forgetting and false recall inevitable. Scaling up does not fix it. Introduction Every deployed retrieval-augmented generation system, every long-term agent memory, and every knowledge graph built on dense embeddings shares a design choice: organise information by meaning. Items that are semantically related sit near each other in representation space. This is what makes these systems capable of generalisation, analogy, and conceptual transfer rather than mere keyword lookup. But it also means that when the system tries to retrieve one memory, its semantic neighbours compete for the same retrieval slot. That competition is interference, and this paper asks whether any semantic memory system can avoid it. Our previous work, HIDE 3 , showed that one simple retrieval architecture (cosine similarity over sentence embeddings) reproduces several canonical memory phenomena, including forgetting under interference ( b=0.460\pm 0.183 ), DRM-style false recall ( \text{FA}=0.583 ), spacing effects, and tip-of-tongue states ( 3.66\% ). (We note that different dimensionality estimators yield different values for the same model (participation ratio \approx 158 , Levina–Bickel \approx 10.6 , PCA-projected \approx 16 ), a discrepancy we reconcile in the Dimensionality section; all place these systems in the interference-vulnerable regime.) The natural objection is architectural: perhaps those phenomena are artefacts of one particular embedding-and-threshold system rather than consequences of semantically organised memory more broadly. This paper addresses that objection. We identify a theorem class, semantically continuous kernel-threshold memories , within which interference is not a bug of one architecture, but a structural consequence of semantic organisation under finite effective dimensionality. We then show empirically that related pressures appear across multiple modern memory architectures, even when their behavioural expression differs. This paper argues that within a broad and practically important theorem class, these phenomena follow from the structure of semantically organised retrieval itself. We call a memory system semantically useful if it supports retrieval by conceptual relatedness rather than exact lexical identity alone. This is a functional definition: the target regime is memory that supports inference, analogy, and conceptual transfer. The theorem developed here applies not to all possible memories, but to a specific class of semantically continuous retrieval systems. To obtain fully rigorous results, we make explicit the theorem class. Our proofs apply to semantically continuous kernel-threshold memories : systems whose retrieval rule is a monotone function of an inner-product score in a semantic feature space (Axiom A1), whose semantically useful representation is optimised under a rate or distortion budget (Axiom A3), and whose semantic manifold has finite local intrinsic dimension (Axiom A4). This class includes dense vector retrieval, embedding-based graph memory, and hidden-state similarity retrieval. Architectures equipped with an external symbolic verifier or exact episodic record fall outside this theorem class and are treated separately as behavioural workarounds rather than counterexamples. The claim is therefore not that every conceivable memory system must exhibit the same behavioural signatures. It is that a large and practically central class of modern memory systems inherits a common geometric vulnerability. Architectures can differ in how they express that vulnerability, and some can partially compensate for it behaviourally, but those compensations are not free. We close the gap with four theorems and a unifying No-Escape Theorem. Within the kernel-threshold theorem class, any system satisfying Axioms A1–A5 exhibits interference-driven forgetting, false recall, and partial retrieval states. The logical chain is: semantic kernel + rate-distortion optimality \Rightarrow finite semantic effective rank (Theorem 1) \Rightarrow positive cap mass (Theorem 2) + growing memory \Rightarrow inevitable forgetting (Theorem 3); power-law arrival + population heterogeneity \Rightarrow power-law forgetting curve. Independently: associative \delta -convexity \Rightarrow lure inseparability under threshold tuning (Theorem 4) (Fig. 1 ). We verify every link empirically across five architecturally distinct memory systems: a vector database (BGE-large 21 ), an attention-based context window (Qwen2.5-7B 13 ), a filesystem agent memory with BM25 + LLM re-ranking, a graph memory with PageRank (MiniLM 15 ; similar contrastive architectures underpin CLIP 14 ), and parametric knowledge in LLM weights. The effective dimensionality convergence (from d_{\text{nom}}=3{,}584 to d_{\text{eff}}=17.9 for Qwen hidden states) mirrors the low-dimensional structure in biological neural populations 19 , 7 . We emphasise what the theorem does not say. It bounds the existence of these phenomena, not their magnitude . Engineering can and should optimise parameters to minimise unwanted interference; the gap between “inevitable” and “catastrophic” is where engineering contributes. The forgetting exponent, the false alarm rate, and the TOT probability are continuous functions of system parameters; the theorem says these functions are bounded away from zero for systems in the kernel-threshold theorem class satisfying Axioms A1–A5. Murdock’s serial position effect 11 , Cepeda et al.’s 6 distributed practice findings, Brown and McNeill’s 5 tip-of-tongue phenomenology, and Nadel and Moscovitch’s 12 consolidation theory all describe the same geometric substrate from different vantage points. The most important finding is not that all five architectures show the same phenomena (they do not, at the behavioural level) but that the geometric vulnerability holds across the tested architectures under the SPP formalism while the behavioural expression depends

Read the complete text