Markovian Circuit Tracing for Transformer State Dynamics
Abdullah X · 2026
Read Markovian Circuit Tracing for Transformer State Dynamics by Abdullah X (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20824.txt.
Introduction
Abdullah X introduces Markovian Circuit Tracing, a diagnostic pipeline for checking whether transformer activations hold coarse state-transition patterns. Many sequence computations are simpler to study as motion through internal states than as lone local circuits. The benchmark uses synthetic Hidden Markov Model tasks where latent states, shift matrices, belief vectors, and optimal predictions are known exactly. Across six model families and three seeds each, tiny causal transformers find near-Bayes next-token models, and extracted state patterns recover coarse transition signal.
Catalog record
| CREATOR | Abdullah X |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26392 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20824
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Markovian Circuit Tracing for Transformer State Dynamics Abdullah X Abstract Many sequence computations are easier to study as movement through internal states than as isolated local circuits. We introduce Markovian Circuit Tracing (MCT), a diagnostic pipeline for testing whether transformer activations contain coarse state-transition structure. The benchmark uses synthetic Hidden Markov Model (HMM) tasks where latent states, transition matrices, Bayesian belief vectors, Bayes-optimal predictions, and forced-state counterfactual targets are known exactly. Across six HMM families and three seeds per family, tiny causal transformers learn near-Bayes next-token predictors, with mean excess loss over Bayes of 0.0138. Residual activations contain partial Bayesian belief information in this controlled synthetic benchmark. State abstractions extracted from these activations recover coarse transition signal, strongest in persistent and lower-state regimes, and weaker in ambiguous-emission and six-state regimes. The clearest result comes from state forcing. Patching a recovered-state centroid reduces KL to the exact HMM counterfactual target from 0.1957 in the unpatched model to 0.0532 on average, beating wrong-state, mean-activation, random-activation, and shuffled-label controls. The contribution is a controlled benchmark and evaluation framework for transformer state-dynamics interpretability, with MCT as a simple reference pipeline. 1 Introduction Mechanistic interpretability studies how trained neural networks compute. Much recent work studies circuits, features, attention heads, activation patching, and causal interventions in transformers ( Olah et al. 2020 ; Elhage et al. 2021 ; Meng et al. 2022 ; Conmy et al. 2023 ) . This has produced useful explanations for local behaviors. A different kind of computation appears in partially observed sequence tasks. The model must update an internal summary over time, and that summary must carry information for future prediction. Hidden Markov Models provide a clean setting for this question. An HMM has an unobserved state sequence, a transition law, and visible emissions. A model trained on visible tokens should learn a predictive summary of the hidden process. In the exact Bayesian solution, that summary is a belief vector over latent states. This makes HMMs useful for interpretability. The experimenter knows the latent states, the belief vectors, the transition matrix, and the exact counterfactual output for forced latent states. We use this setting to test transformer state dynamics. Markovian Circuit Tracing maps activations to candidate internal states, estimates transitions over those states, tests Markov order, and performs state-forcing interventions. The goal is diagnostic. MCT asks whether a chosen activation-derived state abstraction carries usable transition information. It does not require the model to copy the sampled HMM state. In a partially observed task, a predictive belief-state abstraction can be the more natural target. This paper contributes a controlled transformer benchmark where automata extraction, predictive-state ideas, probing, activation patching, and causal abstraction can be tested together against exact ground truth. Prior work has studied these components separately. Here the benchmark gives exact latent labels, exact Bayesian beliefs, exact transition matrices, exact Bayes-optimal predictions, and exact forced-state counterfactual targets in one setting ( Baum & Petrie 1966 ; Rabiner 1989 ; Littman et al. 2001 ; Geiger et al. 2021 ; Chan et al. 2022 ) . The empirical result is partial but consistent. Transformers learn the prediction tasks well. Belief information is recoverable from activations in all HMM families, with large family variation. Residual and PCA-based state abstractions recover part of the transition structure, while exact belief clustering remains a stronger upper bound. State forcing gives the clearest evidence that recovered states are causally meaningful in this benchmark. The results support MCT as a benchmarked diagnostic for coarse predictive state dynamics, not as a full hidden-state recovery algorithm. Figure 1 : Markovian Circuit Tracing pipeline. A transformer is trained on visible HMM emissions. Activations are used for belief probing, state recovery, transition estimation, Markov-order testing, and counterfactual state forcing. 2 Background and related work Markov chains and HMMs. A finite Markov chain has states s_{t}\in\{1,\dots,K\} and transition matrix T with P(s_{t+1}=j\mid s_{t}=i)=T_{ij}. (1) An HMM adds visible emissions x_{t}\in\{1,\dots,V\} with emission matrix E , where E_{iv}=P(x_{t}=v\mid s_{t}=i) . The model observes x_{1:t} but not s_{t} . The exact Bayesian belief state is b_{t}(i)=P(s_{t}=i\mid x_{1:t}). (2) With row-vector notation, the next-token distribution is P(x_{t+1}\mid x_{1:t})=b_{t}TE. (3) This equation is central. A trained transformer may represent a predictive belief state rather than the sampled latent label. Circuit analysis and interventions. Circuit work analyzes internal mechanisms through components, pathways, and interventions ( Olah et al. 2020 ; Elhage et al. 2021 ; Conmy et al. 2023 ) . Activation patching and causal tracing test whether chosen activations causally affect model outputs ( Meng et al. 2022 ) . Causal scrubbing frames interpretability claims as hypotheses that should pass intervention tests ( Chan et al. 2022 ) . MCT uses the same intervention logic, but the HMM benchmark gives exact targets for the intervention. Probing and state extraction. Linear probes test whether representations contain recoverable information ( Alain & Bengio 2016 ; Hewitt & Liang 2019 ; Belinkov 2022 ) . Probes alone do not show causal use. Older work extracted finite-state structure from recurrent networks and automata-like models ( Cleeremans et al. 1989 ; Omlin & Giles 1996 ; Weiss et al. 2018 ) . Predictive state representations define state by future predictive content rather than by hidden labels ( Jaeger 2000 ; Littman et al. 2001 ; Boots et al. 2011 ) . Our benchmark connects these lines of work to transformer activations under exact HMM ground truth. Sparse features and causal abstraction. Sparse autoencoders and dictionary learning methods study interpretable feature bases in neural activations ( Elhage et al. 2022 ; Cunningham et al. 2023 ; Bricken et al. 2023 ) . Causal abstraction asks when neural computation supports a higher-level causal model ( Geiger et al. 2021 ) . The present paper uses simple residual clustering as a reference extractor. The benchmark is meant to support future extractors, including sparse and probabilistic methods. 3 Markovian Circuit Tracing Given an activation vector a_{\ell,t} at layer \ell and time t , MCT applies a state extractor z_{t}=g(a_{\ell,t})\in\{1,\dots,M\}. (4) The reference extractor is K-means over residual activations. The recovered transition matrix is \widehat{T}^{(z)}_{mn}=\frac{\sum_{t}\mathbf{1}\{z_{t}=m,z_{t+1}=n\}}{\sum_{t}\mathbf{1}\{z_{t}=m\}}. (5) When M=K , we align recovered clusters to true states only for evaluation. This Hungarian matching step is an oracle evaluation step. It is not used in a real unlabeled setting. MCT evaluates a recovered state sequence through four tests. 1. Belief recovery . A linear ridge probe maps a_{\ell,t} to the exact Bayesian belief vector b_{t} . 2. Transition recovery . The aligned \widehat{T}^{(z)} is compared with T using row-wise KL and Frobenius error when M=K . 3. Markov order . Empirical order-0, order-1, and order-2 models over z_{t} are evaluated by held-out negative log-likelihood. 4. State forcing . Recovered-state centroids are patched into the model and compared with exact HMM counterfactual targets. For a forced latent state i , the next-token target is P(x_{t+1}\mid do(s_{t}=i))=e_{i}TE, (6) where e_{i} is the one-hot row vector for state i . The intervention test uses several controls. We compare the unpatched model, the recovered centroid, a wrong recovered centroid, the mean activation, a random activation, a shuffled-label centroid, and a true-state centroid. Lower KL to e_{i}TE means the patch moves the model closer to the exact counterfactual target. 4 Benchmark design The benchmark