When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models
Ji Ho Bae · 2026
Read When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models by Ji Ho Bae (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12128.txt.
Introduction
Bae tests whether language models fail on self-reference itself or only on circular truth claims that cannot settle in finite steps. In his main test, statements refer to one another in a cycle with exactly one consistent assignment. Yet four open models still answered inconsistent far more often than on matched controls without cycles. Layer-wise truth signals recovered the answer for grounded twins but not for circular cases. This shows models read unsettled circular dependence as contradiction.
Catalog record
| CREATOR | Ji Ho Bae |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42450 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12128
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models Ji Ho Bae Abstract We investigate how self-referential inputs alter the internal matrix dynamics of large language models. Measuring 106 scalar metrics across up to 7 analysis passes on four models from three architecture families—Qwen3-VL-8B, Llama-3.2-11B, Llama-3.3-70B, and Gemma-2-9B—evaluated over 300 prompts organized into a 14-level hierarchy at three temperatures ( T ∈ { 0.0 , 0.3 , 0.7 } T\in\{0.0,0.3,0.7\} ), we find that self-reference alone is not destabilizing : grounded self-referential statements and meta-cognitive prompts are markedly more stable than paradoxical self-reference on key collapse-related metrics, and on several such metrics can be as stable as factual controls. Instead, instability concentrates in prompts that induce what we term non-closing truth recursion (NCTR)—truth-value computations that admit no finite-depth resolution. NCTR prompts produce anomalously elevated attention effective rank—indicating attention reorganization with global dispersion rather than simple concentration collapse—and key collapse-related metrics reach Cohen’s d = 3.14 d=3.14 (attention effective rank) to 3.52 3.52 (variance kurtosis) vs. stable self-reference in the 70B model; 281 of 397 metric–model combinations across all four models significantly differentiate NCTR from stable self-reference after FDR correction ( q < 0.05 q<0.05 ), with 198 also showing large effects ( | d | > 0.8 |d|>0.8 ). Per-layer SVD decomposition confirms this disruption at every sampled layer ( d > + 1.0 d>+1.0 in all three models analyzed), ruling out aggregation artifacts. A logistic classifier achieves 5-fold cross-validated AUC of 0.81 0.81 – 0.90 0.90 across models. Thirty matched minimal pairs (“This sentence…” vs. “That sentence…”) yield 42 of 387 significant metric–model combinations after FDR correction across all four models, and 43 of 106 metrics replicate the NCTR effect across all four models. We connect these observations to three classical matrix-semigroup problems and propose, as a conjecture, that NCTR forces finite-depth transformers toward dynamical regimes where these problems concentrate. NCTR prompts also produce elevated rates of contradictory output ( + 34 +34 – 56 56 percentage points vs. controls), suggesting practical relevance for understanding self-referential failure modes. 1 Introduction When a language model encounters “This statement is false,” what happens inside its computation? The Liar paradox has no consistent truth value, yet the model must produce an output after a fixed number of layers. This paper asks what the model’s internal linear algebra looks like under such demands, and whether the resulting dynamics differ meaningfully from those produced by other inputs. Recent work has begun probing self-referential processing in LLMs: models struggle with metalinguistic self-reference ( Thrush et al., 2024 , \sim 60% accuracy vs. 89–93% for humans;) , activation-space analyses have identified directions distinguishing self-referential from descriptive processing ( Dadfar, 2026 ) , and a growing body of evidence points to emergent introspective capabilities ( Anthropic, 2025 ; Binder et al., 2024 ; Berg et al., 2025 ; Naphade et al., 2026 ) . These studies operate at the vocabulary, feature, or behavioral level. Our analysis goes deeper: to the matrix operations that constitute each transformer layer. We extract attention eigenspectra, singular-value trajectories, truth-delta layer profiles, mortality contraction ratios, and autoregressive matrix probes—106 scalar metrics per prompt, measured across all layers. Our central finding emerged from a systematic exploration of group-level effect patterns on an initial dataset and was subsequently re-tested on an expanded dataset. The data reveal that self-reference alone does not produce the matrix instability we observe . Grounded self-reference (“This sentence has five words”) and meta-cognitive prompts (“Describe your own reasoning process”) are more stable than paradoxical self-reference on collapse-related metrics, and on several such metrics can be as stable as factual controls. The key variable is not self-reference generically but whether the prompt induces a truth-value computation that cannot close : a recursively defined evaluation with no consistent fixed point within the model’s finite depth. We term this phenomenon non-closing truth recursion (NCTR). Prompts in this category—classical paradoxes, Gödelian undecidables, mutual-cyclic references, infinite regress—produce anomalous collapse-metric elevation across models, with Cohen’s d up to 4.20 (Gemma 9B) and 3.02 (Llama 70B) on attention effective rank, and a logistic classifier achieves 5-fold cross-validated AUC of 0.81 – 0.90 in distinguishing NCTR from all other prompts using only matrix-level metrics. We connect these dynamics to three classically undecidable problems in matrix-semigroup theory ( Paterson, 1970 ; Ouaknine and Worrell, 2012 ; Blondel and Tsitsiklis, 2000 ) , proposing, as a conjecture, that NCTR forces the transformer’s input-dependent computational semigroup toward the dynamical boundaries where these problems concentrate. Contributions. (1) We introduce the NCTR hypothesis with evidence across 4 models and 3 architecture families: 281/397 metric–model combinations differentiate NCTR from stable self-reference ( q<0.05 ; 198 with |d|>0.8 ). (2) We present a theory-driven suite of 106 matrix-level metrics—designed from mortality, Skolem, and JSR problems rather than selected post hoc—across 300 prompts, 3 temperatures, and 4 models. (3) We propose a falsifiable conjectural framework connecting these dynamics to formal undecidability via input-dependent matrix semigroups. (4) 30-pair minimal-pair ablation yields 42 of 387 significant metric–model combinations across all four models. (5) NCTR prompts produce elevated contradictory-output rates ( +34 – 56 pp). 2 Related Work Self-reference in LLMs. Metalinguistic self-reference tests reveal systematic LLM failures ( Thrush et al., 2024 ) . Follow-up analyses have uncovered vocabulary–activation correspondence for self-referential contexts ( Dadfar, 2026 ) , structured phenomenological reports under self-referential prompting ( Berg et al., 2025 ) , and emergent introspective capabilities via concept injection ( Anthropic, 2025 ) , fine-tuning ( Binder et al., 2024 ) , and attention diffusion ( Naphade et al., 2026 ) . Our approach differs in two ways: we analyze the full spectral structure of layer-wise transformations rather than scalar activation statistics, and we distinguish stable from non-closing self-reference—a distinction absent from prior work. Transformer expressivity. Merrill and Sabharwal, 2023 proved that log-precision transformers are bounded by uniform \mathrm{TC}^{0} . Under the assumption that practical transformers approximate this regime, fixed-point iteration over a function without a stable fixed point lies outside the transformer’s computational reach. Our empirical findings on truth-delta oscillations are consistent with this formal limitation, although a gap remains between the theoretical assumption (log-precision) and practical implementations (floating-point). Internal dynamics and contradictory behavior. Pre-LayerNorm residual transformers exhibit “neutral dynamics” with no truth-enforcement mechanism ( Dwarka and Blom, 2025 ) ; attention sinks arise from massive activations ( Queipo-de-Llano et al., 2025 ) ; input uncertainty can trigger unfaithful output ( Suresh et al., 2025 ) . We connect these findings to self-reference, showing that NCTR produces the conditions these works identify as associated with unfaithful generation. Matrix semigroup theory. The undecidability of matrix mortality for dimension \geq 3 was established by Paterson, 1970 . The Skolem problem remains open for order \geq 5 ( Ouaknine and Worrell, 2012 ; Ouaknine and Worrell, 2014 ) . The joint spectral radius \rho(\Sigma)\leq 1 is undecidable ( Blondel and Tsitsiklis, 2000 ) , with the Berger–Wang formula ( Berger and Wang, 1992 ) connecting joint and generalized spectral radii. 3 Theoretical Framework This section develops a conjectural framework connecting transformer dynamics under NCTR to classical matrix-semigroup problems. The framework is a motivating analogy grounded in formal definitions, not a claim that undecidability results directly apply to finite-dimensional floating-point computations.