Pay Attention Later: From Vector Space Diffusion to Linearithmic Spectral Phase-Locking

Alper Yıldırım, İbrahim YÜCEDAĞ · 2025

Read Pay Attention Later: From Vector Space Diffusion to Linearithmic Spectral Phase-Locking by Alper Yıldırım, İbrahim YÜCEDAĞ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01208.txt.

Introduction

Alper Yıldırım and İbrahim Yücedağ propose a new attention scheme for transformers in a 2025 paper. Standard transformers pay what the authors call a semantic alignment tax. That tax is a heavy optimization cost. The model must organize a chaotic starting state into a coherent geometric map, through slow local gradient diffusion.

They hypothesize that this diffusive learning causes catastrophic rigidity. A model organized that way cannot adapt to novel concepts without destroying its pretrained reasoning. Their alternative moves from vector space diffusion toward a spectral phase-locking scheme with far better scaling. If the idea works, models could keep learning new concepts without full retraining.

Catalog record

CREATORAlper Yıldırım, İbrahim YÜCEDAĞ
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pay Attention Later: From Vector Space Diffusion to Linearithmic Spectral Phase-Locking Alper Yıldırım, İbrahim YÜCEDAĞ Abstract Standard Transformers suffer from a “Semantic Alignment Tax”, a prohibitive optimization cost required to organize a chaotic initialization into a coherent geometric map via local gradient diffusion. We hypothesize that this reliance on diffusive learning creates “Catastrophic Rigidity”, rendering models unable to adapt to novel concepts without destroying their pre-trained reasoning capabilities. To isolate this phenomenon, we introduce Iterative Semantic Map Refinement (ISMR), a diagnostic protocol revealing that alignment is a fixed geometric barrier that scaling cannot solve; a 20-layer model overcomes this barrier no faster than a 1-layer model. We introduce the Phase-Resonant Intelligent Spectral Model (PRISM). PRISM encodes semantic identity as resonant frequencies in the complex domain ( ℂ d \mathbb{C}^{d} ) and replaces quadratic self-attention with linearithmic O ⁡ ( N ​ log ⁡ N ) O(N\log N) Gated Harmonic Convolutions. We validate PRISM on the WMT14 translation task. While the Standard Transformer maintains a slight edge in general competence on static benchmarks (23.88 vs 21.40 BLEU), it fails the “Plasticity-Stability” stress test completely. When injected with novel concepts, the Transformer suffers Catastrophic Forgetting, degrading by -10.55 BLEU points while achieving only 60% acquisition. In contrast, PRISM demonstrates Lossless Plasticity , achieving 96% 5-shot acquisition with negligible degradation (-0.84 BLEU). These results suggest that harmonic representations effectively decouple memory from reasoning, offering a structural solution to the plasticity-stability dilemma in real-time knowledge adaptation. 1 Introduction Effective relational reasoning prioritizes logical relationships over surface features, a capacity often attributed to strong inductive biases regarding the structure of information ( Webb et al. 2024 ) . However, effective reasoning requires a fundamental prerequisite: Semantic Geometry. A child operates on a structured mental map where concepts like “King” and “Queen” already possess meaningful proximity. In contrast, dominant neural architectures like the Transformer ( Vaswani et al. 2017 ) lack this pre-existing structure. Initialized with random weights, these models begin in a state of representational chaos, expending a significant portion of their compute budget just to discover a coherent semantic space. We identify the “Semantic Alignment Tax” not as a capacity limitation, but as a structural mismatch: the inherent inefficiency of capturing non-local, harmonic semantic topology via local gradient diffusion. Based on our empirical analysis, we propose that future architectures may need to decouple representation from reasoning to eliminate this cost. We specifically explore the theoretical potential of extending the geometric logic of Rotary Position Embeddings (RoPE) (Su et al., 2024) to semantic space, modeling identity not as static coordinates, but as resonant frequencies in the complex domain. Neuroscientific studies suggest that biological systems avoid this tax by supporting non-local computation, where functional networks emerge as harmonic waves on the connectome ( Atasoy et al. 2016 ) . Crucially, capturing these long-range interactions requires modeling dynamics with complex harmonics rather than simple diffusion ( Deco et al. 2025 ; Naselaris et al. 2009 ) . We contend that current scaling strategies merely mask this tax rather than solving it. As a proof-of-concept, we employ Iterative Semantic Map Refinement (ISMR), a diagnostic methodology designed to isolate the impact of the semantic map’s quality. Our findings demonstrate that the alignment tax is a fixed geometric barrier, motivating a shift toward harmonic, signal-based representations. 2 Related Work: Fixing the Processor vs. Fixing the Representation The struggle for data efficiency in Transformers has led to two distinct philosophical approaches: (1) building more complex processors (i.e., new attention mechanisms) to untangle messy information, and (2) improving the representations that the processor acts on. 2.1 Architectural Inductive Biases: The “Processor” Fix A prominent line of research ( Altabaa et al. 2024 ; Webb et al. 2024 ) argues that Transformers lack a “relational bottleneck.” This theory posits that the brain achieves efficiency by architecturally separating abstract reasoning from raw sensory features. The first implementation of this, the Abstractor ( Altabaa et al. 2024 ) , was a subtractive architecture. It forced a “hard bottleneck” by compressing inputs into abstract symbols and discarding token-level features. This seems highly effective for purely relational toy tasks (e.g., “Object Sorting”) where the goal is to identify a logical rule regardless of the object’s identity. However, as the authors note, this “hard bottleneck” is detrimental for partially-relational tasks like NMT, where specific token identities (e.g., “King”) are semantically essential and cannot be discarded. This insight led to an important architectural evolution: the Dual Attention Transformer (DAT) ( Altabaa & Lafferty 2025 ) . DAT is an additive architecture that abandons the hard bottleneck, instead running a standard “sensory” Transformer in parallel with a new, complex “relational attention” processor. The Benchmark Mismatch. While DAT represents a significant engineering improvement, its evaluation relies heavily on the “Relational Games” benchmark (DAT, Fig. 2). We argue that success on these benchmarks—which test abstract logic (e.g., symmetry detection)—may not be a transferable metric for an architecture’s suitability for semantic geometry tasks like language. This distinction is evidenced by the authors’ own language ablation (Appendix D.3, Fig. 12). When testing their original, simpler “Abstractor” (RCA) mechanism within the DAT architecture on language modeling, it performed only on par with a standard Transformer. This suggests that the impressive gains of DAT on language stem from its new, highly complex “Relational Attention” (RA) implementation and added tunability, rather than the original “bottleneck” principle itself. In our view, this body of work represents a “Processor-First” fix: attempting to build a more complex engine to navigate a map that is still fundamentally disordered. 2.2 Representation-First Approaches: The ”Representation” Fix Our work explores a complementary thesis: that for tasks dependent on semantic geometry, the fundamental bottleneck is not the processor’s reasoning ability, but the structural integrity of the map it operates on. Several existing methods implicitly support this view. Pre-trained embeddings are the most common strategy. As shown by Qi et al. ( Qi et al. 2018 ) , initializing with pre-trained vectors is particularly critical in low-resource scenarios, yielding gains of up to 20 BLEU points. This effectively allows the model to “pre-pay” the alignment tax. However, this strategy is mechanically fragile; Kim et al. ( Kim et al. 2024 ) demonstrate that without strict standardization, pre-trained embeddings introduce statistical mismatches that can destroy sequence information. Rotary Position Embeddings (RoPE) ( Su et al. 2024 ) offer a more fundamental precedent for our position. RoPE demonstrated that positional relationships are best encoded not by additive perturbations (which require learning), but by rotating the vector in the complex plane ( e^{im\theta} ). This multiplicative interaction allows relative distance to be preserved automatically. We argue that this success is not accidental; it is evidence that relational properties—whether positional or semantic—require complex harmonic representations rather than static Euclidean coordinates. Embedding Prediction and Disentanglement ( Hirasawa & Komachi 2019 ; Kumar & Tsvetkov 2019 ; Li et al. 2022 ) alter the learning objective to predict continuous vectors or force latent variables into specific statistical priors (e.g., Gaussian mixtures). While Li et al. ( Li et al. 2022 ) successfully demonstrate that structured priors can aid low-resource generation, we contend that these methods essentially “patch” the statistical distribution without correcting the underlying optimization dynamics. They still rely on the encoder

Read the complete text