Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models
Nathan Roll · 2026
Read Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models by Nathan Roll (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26494.txt.
Introduction
Roll asks whether quantum language models truly use quantum effects, or just run classical tricks on quantum hardware. He runs the first mechanistic study of these models, mixing gate ablation, entanglement tracking, and density-matrix swaps. The test task needs long-range links between tokens. One-qubit models prove exactly simulable by classical means and copy the classical plan. Two-qubit models instead store context in entanglement between qubits, backed by three causal tests. On real hardware, noise kills the entanglement plan while the classical one survives.
Catalog record
| CREATOR | Nathan Roll |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50343 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26494
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models Nathan Roll Abstract Quantum language models have shown competitive performance on sequential tasks, yet whether trained quantum circuits exploit genuinely quantum resources—or merely embed classical computation in quantum hardware—remains unknown. Prior work has evaluated these models through endpoint metrics alone, without examining the memory strategies they actually learn internally. We introduce the first mechanistic interpretability study of quantum language models, combining causal gate ablation, entanglement tracking, and density-matrix interchange interventions on a controlled long-range dependency task. We find that single-qubit models are exactly classically simulable and converge to the same geometric strategy as matched classical baselines, while two-qubit models with entangling gates learn a representationally distinct strategy that encodes context in inter-qubit entanglement—confirmed by three independent causal tests ( p < 0.0001 p<0.0001 , d = 0.89 d=0.89 ). On real quantum hardware, only the classical geometric strategy survives device noise; the entanglement strategy degrades to chance. These findings open mechanistic interpretability as a tool for the science of quantum language models and reveal a noise–expressivity tradeoff governing which learned strategies survive deployment. 1 Introduction How recurrent models preserve contextual information across intervening material has been studied since Elman’s simple recurrent networks ( Elman 1990 ) . Classical architectures address this through gating ( Hochreiter & Schmidhuber 1997 ) , and the strategies they learn have been probed in detail: agreement dependencies in LSTMs ( Linzen et al. 2016 ; Gulordava et al. 2018 ) , structured recurrences in state space models ( Gu et al. 2022 ) . The recurring lesson is that mechanistic understanding of how a model solves the memory problem reveals more than benchmark accuracy alone. Quantum language models, recurrent networks whose hidden states are quantum mechanical, change the picture. Their state spaces grow exponentially with system size: two entangled qubits occupy a 15-dimensional space that no product of classical vectors can represent. Quantum recurrent architectures have shown faster convergence on sequential benchmarks ( Chen et al. 2022 ; Li et al. 2025 ; Takaki et al. 2021 ) , and compositional quantum NLP frameworks have been run on real hardware ( Lorenz et al. 2023 ; Kartsaklis et al. 2021 ) . But no one has looked at the memory mechanisms these models learn, the question that has been most productive in classical language modeling ( Linzen et al. 2016 ) . We ask the same question for quantum models: do trained quantum language models actually use quantum resources for memory, or do they merely perform classical computation on quantum hardware? Expressivity analyses characterize what quantum circuits could compute ( Abbas et al. 2021 ; Cerezo et al. 2021 ) , kernel methods connect circuit design to feature spaces ( Schuld & Killoran 2019 ; Havlíček et al. 2019 ) , and rigorous speed-ups exist for specific problems ( Liu et al. 2021 ) , but none of this tells us what a circuit does compute after training. The situation parallels LSTMs: their expressivity was established long before anyone probed what trained LSTMs actually encode ( Linzen et al. 2016 ) . For classical models, mechanistic interpretability has since identified circuits ( Olah et al. 2020 ; Elhage et al. 2021 ) , induction heads ( Olsson et al. 2022 ) , superposition ( Elhage et al. 2022 ) , dictionary-learning-based decompositions ( Bricken et al. 2023 ) , and causal structure ( Vig et al. 2020 ; Geiger et al. 2021 ; Wang et al. 2023 ; Conmy et al. 2023 ; Nanda et al. 2023 ) . For quantum models, nothing comparable exists. We address this directly. We train minimal quantum and classical recurrent networks on the Parity-Switch Grammar , a synthetic task requiring context identity to be maintained across distractor tokens, and perform a mechanistic analysis of the learned strategies. By comparing single-qubit models (which cannot entangle) against two-qubit models (with and without CNOT gates) and parameter-matched classical baselines, we can delineate the classical–quantum boundary. Our investigation yields four principal findings: C1. Two-qubit QRNNs with CNOT gates learn an entanglement-based memory strategy that is representationally distinct from any single-qubit mechanism. Context identity is encoded in inter-qubit entanglement entropy, and ablating the CNOT gate causally destroys this encoding, forcing reversion to the classical geometric strategy. C2. Single-qubit QRNNs are exactly classically simulable via the SU(2) \to SO(3) double cover. Both quantum and classical baselines converge to an identical geometric strategy—hemisphere preservation on the Bloch sphere—establishing a precise classical reference point. C3. A shared-parameter tradeoff theorem proves that parameter sharing between context and distractor gates makes simultaneous context encoding and distractor invariance impossible, explaining why minimal architectures fail. C4. Hardware validation on IBM Eagle-generation processors reveals a noise–expressivity tradeoff : the geometric strategy survives perfectly while entanglement-based memory degrades to chance under two-qubit gate noise. Figure 1: Hemisphere separation distinguishes successful from failing quantum memory. Bloch sphere trajectories for 10-distractor sequences. Left: Shared-parameter model (MinimalQLM). Contexts A and B overlap after distractor tokens, causing misclassification (76% accuracy). Right: Decoupled model (DecoupledQLM). Contexts remain in opposite hemispheres throughout, achieving 100% generalization. A classical SO(3) baseline produces identical trajectories, confirming this mechanism is not inherently quantum. 2 Related work Memory in recurrent language models. Linzen et al. 2016 probed LSTMs for agreement using controlled syntactic tasks; Gulordava et al. 2018 showed RNN LMs track grammatical features even in implausible contexts; Marvin & Linzen 2018 extended this across construction types; state space models ( Gu et al. 2022 ) approach the same problem through structured linear recurrences. The common thread is that controlled experiments probing how models encode information yield more than benchmark accuracy alone. We apply this approach to quantum recurrent models. Quantum recurrent architectures. The Quantum LSTM ( Chen et al. 2022 ) augmented classical LSTM cells with variational quantum circuits, demonstrating competitive or faster convergence on certain time-series benchmarks. Subsequent work introduced gated quantum RNNs ( Li et al. 2025 ) , variational QRNNs for temporal data ( Takaki et al. 2021 ) , and hybrid architectures ( Xu 2025 ) . Compositional quantum NLP (DisCoCat) takes a fundamentally different approach, using category-theoretic compositional semantics ( Coecke et al. 2010 ) to derive quantum circuit representations of linguistic structure, implemented in the lambeq framework ( Kartsaklis et al. 2021 ) and validated on superconducting hardware ( Lorenz et al. 2023 ) . None of these examine what trained circuits actually compute internally. We dissect the learned strategies rather than comparing convergence curves. Classical–quantum equivalence and dequantization. Data re-uploading renders single-qubit circuits universal classifiers ( Pérez-Salinas et al. 2020 ) , with provable finite VC dimension ( Mauser et al. 2025 ) . However, single-qubit operations are classically simulable ( Yu et al. 2022 ) —a consequence of the standard SU(2) \to SO(3) double cover, since every single-qubit unitary maps to a 3D rotation. Our dequantization result (Section 3.3 ) confirms this constructively and leverages it as a controlled baseline. Whether entanglement is necessary for quantum computational advantage remains actively debated ( Preskill 2012 ; Biham et al. 2004 ) ; our entanglement ablation experiments provide direct empirical evidence within the variational circuit setting. Mechanistic interpretability. Classical mechanistic interpretability has found modular structure in neural networks: