The Coordinate System Problem in Persistent Structural Memory for Neural Architectures

Abhinaba Basu · 2026

Read The Coordinate System Problem in Persistent Structural Memory for Neural Architectures by Abhinaba Basu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22858.txt.

Introduction

Basu introduces an architecture for persistent memory in neural networks and draws two lessons from it. The network routes sparse attention through a lasting pheromone field over latent slots. The core finding: lasting memory needs a stable coordinate system. Any coordinate system learned with the model proves unstable. Fixed random Fourier features give stable outer coordinates, but routing bias alone does not transfer. Using the pheromone as a learning-rate prior removes negative transfer. Basu names coordinate stability and graceful transfer as two separate needs.

Catalog record

CREATORAbhinaba Basu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91458
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22858

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Coordinate System Problem in Persistent Structural Memory for Neural Architectures Abhinaba Basu Abstract We introduce the Dual-View Pheromone Pathway Network (DPPN), an architecture that routes sparse attention through a persistent pheromone field over latent slot transitions, and use it to discover two independent requirements for persistent structural memory in neural networks. Through five progressively refined experiments using up to 10 seeds per condition across 5 model variants and 4 transfer targets, we identify a core principle: persistent memory requires a stable coordinate system, and any coordinate system learned jointly with the model is inherently unstable . We characterize three obstacles—pheromone saturation, surface-structure entanglement, and coordinate incompatibility—and show that neither contrastive updates, multi-source distillation, Hungarian alignment, nor semantic decomposition resolves the instability when embeddings are learned from scratch. Fixed random Fourier features—derived from the Johnson-Lindenstrauss lemma and Bochner’s theorem—provide extrinsic coordinates that are stable, structure-blind, and yet informative, but with 10 seeds, position-only Fourier coordinates show no significant transfer advantage with routing-bias pheromone ( p > 0.05 p>0.05 for all conditions), revealing that coordinate stability is necessary but not sufficient. DPPN with pheromone-biased routing outperforms transformer and random sparse baselines for within-task learning (AULC 0.700 ± 0.010 0.700\pm 0.010 vs. 0.680 ± 0.010 0.680\pm 0.010 vs. 0.670 ± 0.010 0.670\pm 0.010 ), confirming the architecture’s value. Replacing routing bias with learning-rate modulation eliminates negative transfer entirely: warm pheromone as a learning-rate prior never reduces performance (mean advantage + 0.002 +0.002 across 7 seeds) while routing-bias pheromone consistently does (mean − 0.002 -0.002 across 10 seeds). The diagnostic cascade methodology—where each experiment’s finding motivates the next experiment’s design—may itself serve as a template for systematic architectural investigation. When both coordinates and signal computation are fully extrinsic, the first positive structural transfer emerges: a structure completion function over stable coordinates produces + 0.006 +0.006 AULC same-family bonus beyond a regularization baseline (10 trained seeds, 5 random-control seeds), demonstrating that the catch-22 between stability and informativeness is partially permeable to learned functions. The contribution is the identification of two independent requirements for persistent structural memory: (a) coordinate stability—the coordinate system must be fixed before statistics are accumulated, and (b) graceful transfer mechanism—learned functions or learning-rate modulation, not routing bias, because wrong priors must degrade gracefully rather than actively interfere. 1 Introduction A chess player who masters the principle of central control does not relearn it for each new opening. The structural insight—that occupying the center enables tactical flexibility—transfers across games with entirely different surface moves. An ant colony that discovers a short path between nest and food source does not recompute this route from scratch each morning; pheromone trails persist overnight, biasing the next day’s foragers toward previously successful paths [ 2 ] . Both are instances of persistent structural memory : knowledge about which pathways work , accumulated over experience and reused when the same structural patterns recur under different surface conditions. Deep learning architectures lack this capacity. A transformer [ 1 ] recomputes every attention pattern from scratch on each forward pass. When a model trained on Task A encounters Task B—which shares the same structural dependencies but uses different tokens, features, or modalities—it must rediscover the shared structure from scratch. There is no memory of which computational routes proved useful that could transfer between tasks. We set out to build one. Inspired by ant colony optimization, we constructed a pheromone field over the latent routing space of a transformer-like architecture: a persistent, non-gradient statistic that accumulates evidence about which structural pathways lead to correct predictions, survives when model weights are reset, and biases future routing toward historically productive patterns. If it worked, it would be the first mechanism that enables structural transfer without any shared parameters between source and target tasks. The path from concept to working transfer revealed five distinct obstacles—each illuminating a requirement that any persistent memory system must satisfy. Across five experiments, each fixing the previous obstacle and revealing the next, we traced every obstacle to one root cause: persistent memory requires a stable coordinate system, and any coordinate system learned jointly with the model is inherently unstable. The pheromone field records which slot transitions are useful, but the slots themselves are defined by learned projections that change unpredictably across training runs and weight resets. Two independently trained models assign the same structural patterns to different slots—their pheromone fields are maps drawn in different coordinate systems, and no amount of post-hoc alignment recovers the correspondence (3.5% correlation vs. 3.1% chance). In a fifth experiment, we test a solution suggested by the Johnson-Lindenstrauss lemma: fixed random Fourier features that operate on raw positional inputs, providing coordinates that are stable across runs, structure-blind (containing no task information), and yet distance-preserving. With 3 seeds, the position-only variant appeared to show a directionally correct transfer pattern, but with 10 seeds the effect washed out to uniformly -0.002 (not significant). Coordinate stability is necessary but not sufficient: a further experiment shows that the transfer mechanism matters independently. Replacing routing bias with learning-rate modulation (pheromone as a meta-learning-rate prior) eliminates negative transfer entirely. We make three contributions: 1. We introduce the Dual-View Pheromone Pathway Network (DPPN), an architecture that routes sparse attention through a persistent pheromone field over latent slot transitions (Section 3 ). 2. We conduct five experiments (Sections 5 – 9 ), each diagnosing a distinct obstacle. Together they constitute a diagnostic cascade : pheromone saturation \to surface-structure entanglement \to coordinate mismatch \to embedding instability \to the coordinate system problem. 3. We identify the coordinate system problem as the fundamental obstacle (Section 10 ) and show that coordinate stability is necessary but not sufficient: even with stable coordinates, routing-bias pheromone does not transfer (10 seeds, p>0.05 ). We provide evidence that the transfer mechanism matters independently: learning-rate modulation eliminates negative transfer while routing bias does not (Section 12 ). 2 Research Gap and Evolution of Ideas The coordinate system problem we identify in this paper did not emerge in a vacuum. It sits at the intersection of four research threads—memory-augmented networks, persistent state models, transfer learning, and random feature theory—none of which, individually, confronted the specific obstacle we characterize. This section traces the evolution of ideas that converges on the coordinate stability requirement, and identifies the gaps between existing lines of work that our contribution fills. 2.1 Phase 1: External Memory (2014–2016) The Neural Turing Machine [ 8 ] demonstrated that neural networks can learn to read from and write to external memory using differentiable content-based addressing. The Differentiable Neural Computer [ 9 ] extended this with temporal linking, allowing the network to traverse memory in the order it was written, and dynamic allocation, preventing overwriting. Memory Networks [ 20 ] introduced multi-hop attention over an external memory bank with learned addressing. These architectures established a crucial capability: neural networks can maintain and manipulate information beyond their parameter space. However, the memory in all three cases is episodic —it stores specific content (input patterns, intermediate computations) rather than structural knowledge (which computational routes are useful). The NTM’s memory matrix records what was written; it does not record which read-write patterns proved effective across many inputs. The DNC’s temporal linking provides structural memory in a limited sense (it remembers the order of

Read the complete text