Do Models Read What They Write? Causal Registers in Scratchpad Reasoning
Benjamin Shih, John Winnicki, Eric Darve · 2026
Read Do Models Read What They Write? Causal Registers in Scratchpad Reasoning by Benjamin Shih, John Winnicki, Eric Darve (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29522.txt.
Introduction
Shih, Winnicki, and Darve test whether language models actually read the reasoning notes they write for themselves. Process supervision hopes a scratchpad exposes intermediate variables that later steps use. For oversight, writing a state is not enough; later computation must depend on that written state. Otherwise the notes are decoration.
Their controlled state-tracking task has a known update rule. Some models report only the final state, while others write intermediate states first. At evaluation time, the team edits the internal representation of one written state, leaving the visible scratchpad text unchanged. Because the rule is known, each edit implies exactly one correct consequence. The state-writing model predicts the implied next phase bit on most held-out examples, across both task variants. Controls trained without intermediate states stay near baseline. Additional checks rule out generic steering and simple copying. The same causal pattern replicates across model families.
Catalog record
| CREATOR | Benjamin Shih, John Winnicki, Eric Darve |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.29522
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Do Models Read What They Write? Causal Registers in Scratchpad Reasoning Benjamin Shih, John Winnicki, Eric Darve Abstract A central hope behind process supervision is that models can expose intermediate variables that matter for their later behavior. For this to help with alignment, a scratchpad must be tied to the computation: when the model writes a state, later steps should compute from that state. To test this requirement, we use a controlled state-tracking task with a known update rule, comparing models trained to report only the final state with models trained to write intermediate states before giving the final answer. At evaluation, we edit the internal representation of one written state while leaving the visible scratchpad text fixed. Because the transition rule is known, the edit has a single correct downstream consequence. In Qwen2.5-Coder-7B , the state-writing model predicts the next phase bit implied by the edited state on 80% and 91% of held-out examples across the two task variants, while pretrained and final-answer-only controls remain near baseline. Additional controls rule out generic next-token steering and copying another continuation: the prediction depends on both the edited state and the current move. The same causal-use pattern replicates across model families. Together, these results suggest a sharper goal for scratchpad oversight: not just to make intermediate reasoning legible, but to train written states that the model uses as part of its computation. 1 Introduction Scratchpads are attractive for alignment because they expose intermediate reasoning rather than only a final answer. In process supervision, an overseer might inspect a chain-of-thought, tool log, program state, or proof state and reward the process before the final answer is known. That hope depends on a coupling between what is written and what is computed. If a model writes a state and then uses that state as an input to the next step, the state can in principle be audited by changing it and checking downstream consequences. If the state is only plausible text correlated with the answer, it is much less useful for oversight. A useful metaphor is a register in a program: changing the stored value changes later execution. A transformer does not expose literal registers, but a scratchpad token could play a similar causal role if later predictions compute from the hidden state represented at that token. This is a minimal version of a broader evidence-distortion audit: when an intermediate trace contains information that should constrain a later claim, we would like to know whether the later computation actually depends on that information rather than merely co-occurring with it. We study a minimal version of that question in which the answer is unambiguous. We train models on finite state-tracking tasks where the state after each move is known. Some models are trained to output only the final state, while others are trained to write the full running state before the final answer. We then change the internal value associated with one written state while leaving the scratchpad text unchanged, and ask whether the next state follows from the edited value. The desired behavior is precise: the printed scratchpad is unchanged, the internal state is changed, and the next step changes in the way the update rule predicts. This turns the broad question, “does the model use its scratchpad?”, into a causal test with a right answer. Our main finding is that running-state supervision (fine-tuning) can make a written scratchpad state causally usable. In the supervised model, an internal edit to the phase bit at the current state token redirects the next-state computation in two related order-sensitive tasks. The same edit has little effect in the pretrained or final-answer-only controls. Additional controls show that the model combines the edited state with the current move, rather than favoring one answer token or copying a future continuation from another example. This lets us separate three ideas that are often conflated in discussions of scratchpad oversight: whether a state is written, whether it is internally represented, and whether the model actually computes from it. 1.1 Prior work and causal target Scratchpad and chain-of-thought methods show that intermediate tokens can improve multi-step reasoning ( Nye et al. 2021 ; Wei et al. 2022 ) , while faithfulness work shows that written reasoning can be incomplete, post-hoc, or influenced by irrelevant features ( Turpin et al. 2023 ; Lanham et al. 2023 ) . Probing and causal abstraction draw the related distinction between information that is present in an activation and information the model actually uses ( Hewitt and Manning 2019 ; Geiger et al. 2021 ; Geiger et al. 2024 ) . Work on world models, automata, and arithmetic circuits studies algorithmic state inside sequence models ( Li et al. 2023b ; Nanda et al. 2023b ; Liu et al. 2023 ; Kantamneni and Tegmark 2025 ; Feucht et al. 2026 ; Zhang et al. 2025 ) . Appendix G gives a fuller comparison. Methodologically, we use activation patching and representation editing ( Meng et al. 2022 ; Turner et al. 2023 ; Li et al. 2023a ; Zou et al. 2023 ; Todd et al. 2024 ; Ilharco et al. 2023 ) , but we use them for a different causal target than the usual one. Much of this literature asks whether an internal component affects a behavior, restores a corrupted answer, or steers the model toward a desired class of outputs. Here, the written scratchpad state has known semantics: it is the current state of a transition system. This lets us ask a stricter question. When we edit the internal value at the current state token, does the model use that edited value as the state from which it computes the next step? Success is not an arbitrary output change. The next phase bit must be the one obtained by applying the supplied move to the edited state. This is why two selectivity controls matter: one changes the supplied move while holding the edited state fixed, and the other injects a counterfactual state from a context with an incompatible future. Together, they test whether the model is performing the state update, rather than favoring a token or carrying over another example’s future. In this sense, our novelty is not a new patching primitive, but a counterfactual state-editing test for whether a written intermediate has become a state variable in the model’s own computation. 2 Background: residual streams, KV caching, and activation patching 2.1 Residual streams and the KV cache During cached autoregressive generation, the model state consists of two related parts: the visible token sequence and the KV cache. Suppose the visible prefix is A,B,C,D,E,F , and the model is about to assign probability to the next token G . The token G is not yet part of the input, so there is no residual stream for G yet. The model computes a next-token distribution from the prefix A,\ldots,F . If G is selected, its token id is appended to the sequence and processed on the next decode step. At each visible token position and each layer, the model has a hidden vector called the residual stream. We write r_{i}^{\ell} for the residual stream at position i entering layer \ell . Attention reads from these residual streams