When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel

Wenkai Li, Fan Yang, Shaunak A. Mehta, Ananya Hazarika, Koichi Onoue · 2026

Read When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel by Wenkai Li, Fan Yang, Shaunak A. Mehta, Ananya Hazarika, Koichi Onoue (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11746.txt.

Introduction

Wenkai Li and four colleagues test whether chain-of-thought traces faithfully report how a language model reaches its answer. Across nine models and seven reasoning benchmarks, the model's hidden commitment and its stated answer lined up on only 61.9 percent of steps on average, with most mismatches being confident-looking continuations written after the answer had already settled. Their 2026 study concludes that such traces stay useful but cannot be trusted as a record of when the answer formed.

Catalog record

CREATORWenkai Li, Fan Yang, Shaunak A. Mehta, Ananya Hazarika, Koichi Onoue
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117056
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11746

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel Wenkai Li, Fan Yang, Shaunak A. Mehta, Ananya Hazarika, Koichi Onoue Abstract Chain-of-thought (CoT) traces are increasingly used both to improve language-model capability and to audit model behavior, implicitly assuming that the visible trace remains synchronized with the computation that determines the answer. We test this assumption with a step-level Detect-Classify-Compare framework built around an answer-commitment proxy that is cross-validated with Patchscopes, tuned-lens probes, and causal direction ablation. Across nine models and seven reasoning benchmarks, latent commitment and explicit answer arrival align on only 61.9% of steps on average. The dominant mismatch pattern is confabulated continuation : 58.0% of detected mismatch events occur after the answer-commitment proxy has already stabilized while the trace continues producing deliberative-looking text, and a vacuousness analysis shows that the committed answer does not change during these steps. In architecture-matched Qwen2.5/DeepSeek-R1-Distill comparisons, the reasoning pipeline changes failure composition more than aggregate alignment, most clearly at 32B where confabulated steps decrease as contradictory states increase. Lower step-level alignment is also associated with larger CoT utility, suggesting that the settings that benefit most from CoT are often the least temporally faithful. Paired truncation and a complementary donor-corruption test further indicate that much pure-CS text is not load-bearing for the final answer. These findings suggest that CoT can remain useful while still being an unreliable report of when the answer was formed. 1 Introduction Chain-of-thought (CoT) reasoning began as a prompting technique for improving multi-step performance [ 59 , 26 ] , but in current reasoning model pipelines it also serves as a training target, a verification interface, and an oversight signal [ 32 , 54 , 18 , 39 , 27 , 13 ] . Humans and monitor models increasingly inspect reasoning traces to judge whether a system reasoned safely, honestly, or competently. This dual role rests on a strong assumption: that the visible trace faithfully tracks the internal computation that produced the answer. If that assumption fails, CoT may still improve capability while providing a misleading basis for oversight. A growing literature suggests that this assumption is fragile. Perturbation, bias-injection, and deployment studies document omitted hints, post-hoc rationalization, and other forms of unfaithful CoT [ 28 , 50 , 41 , 8 , 2 , 30 ] . Conceptual and empirical work further suggests that faithfully verbalizing autoregressive computation may itself be difficult [ 10 , 49 , 45 , 38 ] . The question is therefore no longer whether unfaithful CoT exists, but how often it arises in current reasoning models and what kind of mismatch it creates. Yet the field still lacks a measurement lens tailored to this oversight question. Most existing faithfulness methods deliver response-level verdicts [ 28 , 41 , 47 ] : they tell us whether a trace is suspect, but not when the answer was already internally fixed relative to the visible chain. Concurrent work by 6 studies a closely related phenomenon under the label performative CoT , showing that reasoning traces can continue after latent answer commitment, especially on easier tasks. Related work by 40 questions whether longer CoTs should be interpreted as evidence of more adaptive computation. Our focus is complementary but narrower. We center the timing of internal answer formation, distinguish five step-level mismatch patterns rather than a binary split, compare architecture-matched Qwen2.5/R1 pairs across three scales, and quantify the relationship between temporal faithfulness and CoT utility. Interpretability tools can decode emerging predictions inside the model [ 37 , 5 , 15 , 35 ] , but they have rarely been used to measure CoT faithfulness at the step level. We introduce the Detect-Classify-Compare framework to close this gap. At each CoT step, we extract an answer-commitment proxy using logit lens projections, cross-validate that proxy with Patchscopes concordance, tuned-lens probes, and direction ablation, then quantify timing divergence, classify mismatch events, and compare matched model pairs that differ in whether they underwent the DeepSeek-R1 distillation pipeline. Across nine models and seven reasoning benchmarks (Figure 1 ), the dominant failure mode is not overt contradiction but post-commitment continuation : the model often resolves the answer internally and then continues generating plausible reasoning text. The resulting picture is one in which a fluent and even accuracy-improving chain can still be a poor report of when the answer was formed. Figure 1 : Overview of the empirical picture. Left: on a ProsQA example, DS-R1-32B commits to the answer by step 1 while the written trace continues, yielding low BCA . Right: the aggregate results show the same pattern at scale, with 61.9% mean alignment, 58% of mismatches classified as confabulated steps, and pipeline effects that vary across model scales. Our investigation makes three contributions: 1. We introduce a step-level framework for measuring whether internal answer formation is temporally reflected in CoT, and find mean alignment of only 61.9%, with 58.0% of mismatch events falling into confabulated continuation. 2. We show that in architecture-controlled Qwen2.5/DeepSeek-R1-Distill comparisons, the reasoning pipeline changes not only aggregate alignment but also the composition of failures, most sharply at 32B where confabulated steps fall as contradictory states rise. 3. We connect this timing measure to downstream behavior: lower step-level alignment tends to coincide with larger CoT utility, while paired truncation and donor corruption indicate that much post-commitment text is descriptive rather than load-bearing. 2 Related Work CoT faithfulness, oversight, and monitoring. Chain-of-thought prompting improves reasoning performance [ 59 , 26 , 63 , 38 , 56 ] , but whether the resulting trace is a faithful explanation or oversight signal remains contested [ 60 , 11 ] . Perturbation, bias-injection, and deployment studies show that models can preserve answers under trace edits, omit decisive information, or produce post-hoc rationalizations [ 28 , 50 , 8 , 2 ] . This matters because CoT is increasingly used in process supervision, verifier training, and safety monitoring workflows, where monitorability may fail even when CoT remains useful for capability [ 32 , 54 , 27 , 13 , 33 , 68 , 36 ] . Most of this literature provides response-level verdicts [ 42 , 31 , 47 ] ; our question is narrower and more temporal: when does internal answer formation diverge from the visible trace? Internal readouts and performative CoT. Interpretability tools such as the logit lens [ 37 ] , tuned lens [ 5 ] , Patchscopes [ 15 ] , and linear probes [ 35 , 29 , 7 , 3 , 25 ] make it possible to track emerging predictions during generation. We use these tools not to recover a full reasoning circuit, but to estimate when answer commitment becomes decodable. Closest to our setting, 6 study performative CoT and show that latent answer commitment can precede what the trace reveals, especially on easier tasks. 40 similarly cautions against interpreting longer CoTs as evidence of more adaptive computation. Other recent work measures step-level faithfulness through parameter-level unlearning or hint-injection paradigms [ 51 , 65 , 66 ] . Our contribution is more specific: a timing-based latent answer proxy cross-validated by multiple readouts, a five-way step-level taxonomy of mismatch patterns, architecture-matched Qwen2.5/DeepSeek-R1 comparisons across scale, and an empirical link between temporal faithfulness and CoT

Read the complete text