Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo
Peiran Li · 2026
Read Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo by Peiran Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24040.txt.
Introduction
Li proposes a way to update what a language model knows without retraining the whole system. The starting point is MeMo, a model design that stores knowledge in explicit correlation matrix memories, which are structured tables linking sequences and tokens. Writing from Berlin in 2026, Li asks how edits such as replacing or rolling back a fact can run directly on those stored memories instead.
Each high-level edit becomes an ordered transaction, a bundle of small memory operations that can also record how to undo itself. Two extra memory tables support this: one maps version changes to transaction records, and another stores reusable change contents plus the inverse steps. Inputs can be whole sequences or structured differences between versions. Li also sketches how to test update success, rollback, traceability, and whether an edit stays local instead of disturbing unrelated knowledge.
Catalog record
| CREATOR | Peiran Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24040
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo Peiran Li Abstract MeMo proposes language models with explicit multi-layer correlation matrix memories (CMMs), where memorization, retrieval, and forgetting are architectural operations. This paper asks how such memories can reduce the need for retraining when knowledge changes. For changes expressible as MeMo memory associations, the model’s accessible knowledge can be updated by editing explicit memories rather than retraining the whole model. We propose a version-aware operation layer in which high-level operations such as replace , obsolete , keep-history , rollback , and trace are compiled into MeMo-native primitive calls over sequences and tokens. The key observation is that a version-aware operation is rarely a single MeMo association. It is an ordered transaction of primitive edits, for example forgetting one sequence-token chain, memorizing another, preserving a historical chain, and recording an inverse program. The framework introduces two auxiliary CMMs: a Version CMM (V-CMM) for mapping version transitions to transaction handles, and a Transaction CMM (T-CMM) for storing reusable change contents and inverse programs. It supports both direct sequence-level edits and structured diff-level inputs, and outlines an evaluation route for update success, rollback, traceability, locality, and transaction reuse. [orcid=0009-0002-5807-3432, [email protected], ] keywords MeMo ,correlation matrix memory ,transaction memory ,version-aware operations ,memory editing ,neuro-symbolic AI † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: MeMo Workshop on Mechanistic Interpretability & Neuro-symbolic Approaches by-design, Rome (Italy), 24/6/2026 † † address: Freie Universität Berlin, Kaiserswerther Str. 16-18, 14195 Berlin, Germany 1 Introduction Large Language Models are often adapted to new or changed knowledge through continued training, retrieval augmentation, or parameter-level model editing 2 ; 5 ; 6 ; 7 . These methods can be effective, but they are not ideal when a knowledge change is small and can already be expressed as an explicit memory association. In that case, the natural goal is to update the model’s accessible knowledge by editing memory directly, rather than retraining on a fully revised corpus. MeMo is a useful starting point because it externalizes memory through correlation matrix memories and makes memorization and forgetting explicit operations rather than only side effects of training 9 . Its final architecture is Multi-layer MeMo: a stack of CMMs composes input sequences and supports retrieval from explicit memory structures. This paper keeps that Multi-layer MeMo architecture intact. We ask what operation layer is needed when explicit memories are long-lived, versioned, and repeatedly edited. MeMo provides primitive actions such as memorizing, retrieving, and forgetting a sequence-value association, but knowledge evolution also needs replacement, historical preservation, obsolescence, trace, and rollback. The central idea is to treat such operations as transactions over MeMo primitive edits. A transaction is not a single association (S,y) ; it is an ordered program that may remove (S,y_{old}) , add (S,y_{new}) , save historical memories, and record an inverse. We therefore design a two-stage auxiliary memory. A Version CMM (V-CMM) maps version transitions to transaction handles, and a Transaction CMM (T-CMM) maps transaction handles to executable edits. MeMo CMMs store what is remembered; V-CMM and T-CMM store which memory edits belong to a version change and how those edits should be executed. The proposed algebra has two input levels. At the lower level, it supports direct sequence-token updates, such as replacing the memorized continuation of a MeMo sequence. At the higher level, structured knowledge changes, for example ontology moves or obsoletions, are compiled into sets of sequence-token transactions. Change languages such as KGCL provide a realistic interface for such structured changes 3 , while the contribution remains a version-aware operation algebra for Multi-layer MeMo. 2 Multi-layer MeMo as Substrate A CMM stores associations by outer products, following classical correlation matrix memory and related distributed-memory ideas 4 ; 1 ; 8 . For example, C=\sum_{i}k_{i}v_{i}^{\top}. (1) A query multiplies the memory matrix, and forgetting subtracts a corresponding outer product. Multi-layer MeMo extends this mechanism with several CMM layers that compose input sequences into higher-level sequence representations 9 . We denote the original stack by: \mathcal{C}_{\mathrm{MeMo}}=\{C^{(1)},C^{(2)},\ldots,C^{(L)},C^{(out)}\}. (2) At the interface level, we abstract the existing implementation through three primitive procedures: \displaystyle\mathrm{memo}(S,y), \displaystyle\mathrm{forget}(S,y), \displaystyle\mathrm{retrieve}(S), (3) where S is an input sequence and y is the associated token or value. The internal multi-layer CMM machinery decides how the relevant layers are updated. The operation layer introduced below delegates to these primitives instead of directly modifying the layer design. 3 Version-aware Operations 3.1 Direct sequence-level operations The lowest-level input to the proposed algebra is already MeMo-native: a sequence-token edit. At the operation level, a primitive edit can be represented as: e_{j}=(S_{j},y_{j},\lambda_{j}),\qquad\lambda_{j}\in\{+1,-1\}, (4) where \lambda_{j}=+1 denotes memorization and \lambda_{j}=-1 denotes forgetting. When an implementation exposes layer-specific updates, the executable form is (\ell_{j},S_{j},y_{j},\lambda_{j}) ; otherwise \ell_{j} is left implicit and the standard multi-layer MeMo routine decides which internal CMMs to update. A high-level operation is an ordered transaction of such edits: \tau=\langle e_{1};e_{2};\ldots;e_{m}\rangle. (5) The order matters when later edits depend on continuations created by earlier ones. For example, a direct replacement of a MeMo continuation is: \displaystyle\textsc{replace}(S,y_{old},y_{new})=\langle \displaystyle(S,y_{old},-1);\ \displaystyle(S,y_{new},+1)\rangle. (6) This operation changes the current view but does not erase the old value as a historical fact. Version preservation adds explicit versioned sequences: \displaystyle\textsc{keep-History}=\langle \displaystyle(S_{v_{t}},y_{old},+1);\ \displaystyle(S_{v_{t+1}},y_{new},+1)\rangle. (7) Thus, forgetting from the current view is not necessarily erasure. It can mean deactivation from the latest view while preserving earlier memories for version-specific queries and rollback. Multi-token continuations. Although examples are sometimes written as if y were a single value, MeMo’s native primitive is next-token memory. A continuation Y=[y_{1},\ldots,y_{n}] must therefore be decomposed into a chain of primitive edits: \displaystyle\mathrm{chain}(S,Y,\lambda)=\langle \displaystyle(S,y_{1},\lambda);(Sy_{1},y_{2},\lambda); (8) \displaystyle\ldots;(Sy_{1}\cdots y_{n-1},y_{n},\lambda)\rangle. A replacement between continuations becomes: \textsc{replace}(S,Y_{o},Y_{n})=\mathrm{chain}(S,Y_{o},-1);\mathrm{chain}(S,Y_{n},+1). (9) The T-CMM records either this expanded edit set for small transactions or, in the scalable implementation, a reusable template and parameter handles from which the edit chain is generated at execution time. In both cases, the original MeMo stack still receives only standard (sequence,token) operations. 3.2 Structured keys and higher-level changes A structured assertion can be represented as (v,s,r,o) , where v is a version or view, s is a subject, r is a relation, and o is an object value. To make it compatible with Multi-layer MeMo, the structured key is serialized into an input sequence: \sigma(v,s,r)=[\texttt{VERSION},v,\texttt{SUBJECT},s,\texttt{RELATION},r]. (10) For structured resources, s , r , and o denote stable identifiers rather than mutable labels; labels are used only for verbalization. Writing the assertion is a standard MeMo call: \mathrm{memo}(\sigma(v,s,r),o), (11) and removing it from a view is: \mathrm{forget}(\sigma(v,s,r),o). (12) A structured change such as a class move or definition change is first normalized into one or more sequence-level transactions. For instance, replacing the latest value of (s,r) from o_{old} to o_{new} becomes: \displaystyle\langle \displaystyle(\sigma(\mathrm{v_{latest}},s,r),o_{old},-1);\ \displaystyle(\sigma(\mathrm{v_{latest}},s,r),o_{new},+1); \displaystyle(\sigma(v_{t},s,r),o_{old},+1);\ \displaystyle(\sigma(v_{t+1},s,r),o_{new},+1)\rangle. (13) The same mechanism supports obsolete : an old current status is deactivated, a new obsolete status is memorized, an optional replacement is memorized, and the previous status is preserved under the source version. 4 Version and Transaction Correlation Memories 4.1 Why CMMs instead of only logs? A version-aware operation is not itself a standard MeMo memory entry. MeMo stores next-token associations of the form (S,y) . A transaction may instead