Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models

Hassan Ugail, Newton Howard · 2026

Read Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models by Hassan Ugail, Newton Howard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11622.txt.

Introduction

Hassan Ugail and Newton Howard read language models as systems that change in time. Their 2026 preprint seeks working regimes inside LLMs. Text writing runs through vast inner workings in the model. How those workings unfold in time is poorly understood.

Most work on reading models stresses static states instead. Causal interventions are the other common lens in the field. Both leave temporal structure largely unexplored, they argue. The team borrows concepts from neuroscience for the task. Temporal integration and metastability mark neural organization there. They adapt those ideas to transformer models here. A combined time metric is computed from activations. It is tracked across step-by-step text writing in the model.

Catalog record

CREATORHassan Ugail, Newton Howard
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-11622

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models Hassan Ugail, Newton Howard Abstract Large language models perform text generation through high-dimensional internal dynamics, yet the temporal organisation of these dynamics remains poorly understood. Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored. Drawing on neuroscience, where temporal integration and metastability are core markers of neural organisation, we adapt these concepts to transformer models and discuss a composite dynamical metric, computed from activation time-series during autoregressive generation. We evaluate this metric in GPT-2-medium across five conditions: structured reasoning, forced repetition, high-temperature noisy sampling, attention-head pruning, and weight-noise injection. Structured reasoning consistently exhibits elevated metric relative to repetitive, noisy, and perturbed regimes, with statistically significant differences confirmed by one-way ANOVA and large effect sizes in key comparisons. These results are robust to layer selection, channel subsampling, and random seeds. Our findings demonstrate that neuroscience-inspired dynamical metrics can reliably characterise differences in computational organisation across functional regimes in large language models. We stress that the proposed metric captures formal dynamical properties and does not imply subjective experience. K eywords Large Language Models \cdot Dynamical Systems \cdot Neuroscience-inspired metrics \cdot Neural Networks \cdot Interpretability 1 Introduction Large language models (LLMs) have achieved remarkable performance across natural language processing tasks, yet the internal dynamics underlying their computations remain incompletely understood. Current interpretability approaches predominantly examine static representations through probing classifiers, visualise attention patterns, or apply causal interventions to identify component contributions [ 1 , 2 ] . While valuable, these methods rarely address how activations evolve temporally as dynamical systems—a perspective that has proven central to understanding biological neural computation. In neuroscience, temporal organisation of neural activity has emerged as a fundamental principle distinguishing cognitive states. Hierarchical temporal integration, characterised by long-range correlations and scale-free dynamics, differentiates wakefulness from unconscious states [ 3 , 4 ] . Metastability—the tendency of neural systems to transiently coordinate and decoordinate across spatial scales—reflects the brain’s capacity to balance integration and segregation, enabling flexible transitions between processing states [ 5 , 6 ] . Complexity measures incorporating these properties successfully discriminate sleep stages, anaesthetic depth, and pathological conditions [ 7 , 8 , 9 ] . Recent theoretical frameworks have formalised these concepts into composite metrics integrating multiple dynamical dimensions. For instance, perturbational complexity indices quantify causal propagation under external stimulation [ 7 , 10 ] , while resting-state metrics combine temporal correlation structure with cross-frequency coordination [ 11 , 12 ] . These approaches converge on the principle that rich internal dynamics require simultaneously high integration (coordinated activity across scales) and high differentiation (diverse, non-redundant states) [ 13 , 14 ] . Despite conceptual parallels between biological and artificial neural systems, such dynamical analyses have rarely been applied to LLMs. Previous work on recurrent networks examined fixed points and bifurcations [ 15 ] , but transformers’ discrete autoregressive generation presents distinct challenges. Whether transformer activations exhibit distinguishable dynamical regimes across functional conditions, and whether neuroscience-inspired metrics can quantify these differences, remains unexplored. This study addresses this gap by adapting established dynamical measures to characterise GPT-2-medium internal states during text generation. We focus on two core properties: hierarchical integration, quantified via detrended fluctuation analysis (DFA) of temporal correlations [ 16 ] , and metastability, measured as temporal variability in phase synchronisation [ 5 ] . We construct a composite index \Psi^{\prime} combining these components and evaluate its capacity to distinguish five experimental conditions spanning structured reasoning, trivial repetition, destabilised noisy sampling, and two forms of architectural perturbation. We test three hypotheses. First, that \Psi^{\prime} reliably distinguishes functional regimes, with structured reasoning exhibiting higher dynamical organisation than repetitive or noisy conditions—analogous to wakeful versus unconscious brain states. Second, that architectural perturbations produce intermediate reductions in \Psi^{\prime} , indicating partial rather than catastrophic degradation. Third, that \Psi^{\prime} remains robust under recording perturbations (layer subsets, channel subsampling, random seeds), demonstrating system-level properties rather than measurement artefacts. Critically, we emphasise that our aim is not to assess consciousness or subjective experience in artificial systems. Rather, we examine whether formal dynamical properties identified in neuroscience—temporal integration and metastable coordination—can characterise computational regimes in LLMs in a principled, reproducible manner. This represents, to our knowledge, one of the first systematic applications of composite dynamical metrics to transformer internal states. 2 Background 2.1 Dynamical organisation in neural systems The relationship between temporal dynamics and cognitive function has been extensively studied in neuroscience. Scale-free neural activity, characterised by power-law temporal correlations, is disrupted during deep sleep and anaesthesia [ 3 ] . Detrended fluctuation analysis, which estimates the Hurst exponent H characterising correlation scaling across timescales, has proven effective for quantifying this organisation [ 16 ] . Values H>0.5 indicate persistent long-range dependencies, H\approx 0.5 corresponds to uncorrelated fluctuations, and H<0.5 indicates anti-persistence. Empirical studies show that wakeful brain activity typically exhibits H\approx 0.6 – 0.8 , while anaesthesia shifts toward either uncorrelated ( H\approx 0.5 ) or pathologically rigid ( H>1.0 ) dynamics [ 4 ] . Metastability refers to flexible coordination dynamics, quantified as temporal variability in global synchronisation [ 5 , 6 ] . The Kuramoto order parameter R(t)=|\langle\exp(i\theta_{i}(t))\rangle_{i}| captures instantaneous phase alignment across recording sites, with R\approx 1 indicating synchrony and R\approx 0 indicating independence. Metastability M=\text{std}(R(t)) quantifies fluctuation magnitude. High metastability indicates frequent transitions between integrated and segregated states, supporting flexible information processing. Reduced metastability characterises anaesthesia, brain injury, and psychiatric conditions [ 17 , 18 ] . Complexity measures combining these properties have demonstrated clinical utility. Perturbational complexity indices (PCI), which quantify spatiotemporal response patterns to transcranial magnetic stimulation, reliably distinguish conscious from unconscious states [ 7 ] . Spontaneous EEG complexity decreases during propofol anaesthesia and deep sleep, increases during psychedelic states, and correlates with residual awareness in disorders of consciousness [ 8 , 19 , 20 ] . These findings converge on the principle that conscious-level processing requires both integration across scales and differentiation among components [ 13 , 9 ] . 2.2 Interpretability in language models Current interpretability research on LLMs employs diverse methodologies. Mechanistic interpretability seeks to identify specific circuits implementing algorithmic operations, such as indirect object identification or modular arithmetic [ 21 , 22 ] . Probing classifiers assess whether linear readouts can recover linguistic or semantic properties from intermediate representations [ 23 ] . Causal interventions, including activation patching and ablation studies, quantify individual component contributions to model outputs [ 2 , 1 ] . While these approaches have yielded valuable insights into transformer function, they predominantly analyse static snapshots or local causal relationships rather than temporal dynamics. Some recent work has examined how representations evolve across layers [ 1 ] , but systematic characterisation of activation time-series during generation remains rare. Given that transformers process sequences autoregressively—building representations incrementally across tokens—temporal organisation represents a fundamental but understudied aspect of their computation. 2.3 Bridging neuroscience and artificial systems Several considerations motivate applying neuroscience-inspired metrics to LLMs. First, both biological and artificial neural networks exhibit distributed parallel processing across multiple channels and sequential temporal evolution. Second, recent theoretical work emphasises modality-general principles of organisation that may apply across systems [ 11 , 9 ] . Third, formal dynamical measures—unlike mechanistic circuit analysis—require minimal assumptions about implementation details, potentially offering

Read the complete text