LLM Reasoning Is Latent, Not the Chain of Thought

Wenshuo Wang · 2026

Read LLM Reasoning Is Latent, Not the Chain of Thought by Wenshuo Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15726.txt.

Introduction

Wang argues LLM thought lives in hidden state paths, not in the word chains we see. Three mixed strands get split apart: thought as hidden paths, thought as shown word chains, and seeming thought gains from plain extra serial work. Weighing lab, machine, and survey work, plus cases that split shown traces, hidden tweaks, and matched work boosts, the case backs hidden paths as the default. The field should take hidden-state motion as its default target and test thought with plans that pull the three strands apart.

Catalog record

CREATORWenshuo Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40828
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15726

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLM Reasoning Is Latent, Not the Chain of Thought Wenshuo Wang Abstract This position paper argues that large language model (LLM) reasoning should be studied as latent-state trajectory formation rather than as faithful surface chain-of-thought (CoT). This matters because claims about faithfulness, interpretability, reasoning benchmarks, and inference-time intervention all depend on what the field takes the primary object of reasoning to be. We ask what that object should be once three often-confounded factors are separated and formalize three competing hypotheses: H1, reasoning is primarily mediated by latent-state trajectories; H2, reasoning is primarily mediated by explicit surface CoT; and H0, most apparent reasoning gains are better explained by generic serial compute than by any privileged representational object. Reorganizing recent empirical, mechanistic, and survey work under this framework, and adding compute-audited worked exemplars that factorize surface traces, latent interventions, and matched budget expansions, we find that current evidence most strongly supports H1 as a default working hypothesis rather than as a task-independent verdict. We therefore make two recommendations: the field should treat latent-state dynamics as the default object of study for LLM reasoning, and it should evaluate reasoning with designs that explicitly disentangle surface traces, latent states, and serial compute. 1 Introduction Large language models now solve many arithmetic, symbolic, and planning-like tasks more effectively when they are given extra intermediate computation. In practice, this computation may appear as explicit chains of thought, self-consistency, deliberate search, or other inference-time expansions [ 1 , 2 , 3 , 4 , 5 ] . These gains have made reasoning a central object of current LLM research. They have also made it harder to say what the field is actually studying when it studies reasoning. This matters because claims about faithfulness, interpretability, reasoning benchmarks, and inference-time interventions all depend on what the field takes the primary object of reasoning to be. Current work supports at least three incompatible readings of this same phenomenon. One view treats surface chain-of-thought as the reasoning process itself. A second view treats many reasoning gains as consequences of extra serial compute, regardless of representational form. A third view treats multi-step reasoning as a latent process that can be only partly verbalized, or not verbalized at all [ 6 , 7 , 8 , 9 , 10 ] . The difficulty is that recent methods often move several explanatory factors at once, making experimental results hard to interpret as causal support for any specific view: Chain-of-thought prompting changes both visible traces and compute allocation; Latent reasoning methods often change both hidden-state dynamics and compute budget; Test-time scaling changes compute and usually changes the output path as well [ 4 , 5 , 9 ] . The first task, then, is to separate the objects that recent work often conflates. Section 2 does so by distinguishing surface traces, latent-state dynamics, and generic serial compute, and by turning three loose views into three concrete hypotheses: H2 treats multi-step reasoning as primarily mediated by explicit surface CoT; H0 treats most apparent reasoning gains as better explained by generic serial compute than by any privileged representational object; and H1 treats multi-step reasoning as primarily mediated by latent-state trajectories, with surface CoT serving only as a partial interface. Because H2, H0, and H1 assign explanatory priority to surface traces, serial compute, and latent trajectories, they make different predictions about where the strongest causal leverage should lie. Stated this way, the debate is no longer about whether CoT helps . It is about what such help is evidence of . Under that standard, the current record does not equally support all three views. The strongest case for H2 would require surface traces to provide the most stable causal leverage, yet ordinary CoT is often useful without being reliably faithful, and its role varies sharply across tasks [ 6 , 7 ] . The strongest case for H0 would require matched serial compute to explain most reasoning gains, yet extra budget alone does not explain why specific internal states, features, or trajectories can predict or alter reasoning behavior [ 4 , 5 ] . By contrast, recent work on latent-state monitoring and latent reasoning suggests that task-relevant commitment can arise in hidden-state dynamics that are only partly verbalized, or not verbalized at all [ 8 , 9 , 10 ] . Section 3 develops this comparison in detail. We therefore argue that latent-state dynamics should be treated as the default working object of study for LLM reasoning, rather than assuming faithful surface chain-of-thought by default. On this basis, the paper makes two concrete recommendations in Section 4 : (1) latent-state dynamics should be treated as the default object of study for LLM reasoning; and (2) future evaluations should explicitly disentangle surface traces, latent states, and serial compute rather than letting them move together. Section 5 then instantiates these recommendations in a two-tier adjudication program—a controlled regime matrix and a naturalistic transport suite—that compares surface, latent, and compute families under audited budgets. The resulting evidence shows that H1 receives the strongest support in ordinary regimes, while H2 and H0 re-emerge as local boundary regimes when visible traces are made constitutive or when serial compute dominates. Section 6 positions this paper against nearby empirical and survey work by clarifying what those papers establish, what they leave unresolved, and why a direct competition among H1, H2, and H0 is still missing. Section 7 closes by summarizing the resulting field-level claim. 2 Problem Formulation and Competing Explanations 2.1 The Big Question The core disagreement in current LLM reasoning research is not whether extra intermediate computation can help. It often can [ 1 , 2 , 4 , 9 ] . The harder question is what such gains are evidence of. To state the problem precisely, we distinguish three explanatory objects: S : the semantic content of surface chain-of-thought , namely the explicit natural-language intermediate trace; Z : the task-relevant latent-state trajectory , namely the hidden-state path that carries intermediate commitments during reasoning; B : generic serial compute , namely additional budget for iterative computation, search, sampling, or looping, regardless of representational form. By Z we mean task-relevant latent-state trajectories rather than arbitrary hidden activations. 1 1 1 More precisely, Z does not refer to arbitrary hidden activity anywhere in the network, nor to the claim that every hidden activation participates in reasoning. It refers to latent states that carry intermediate task-relevant commitments during multi-step inference. These states may be distributed across layers and positions rather than localized in a single feature or unit, and they need not be fully verbalizable at the surface. Our big question is therefore the following: Once S , Z , and B are distinguished, which of them should be treated as the primary scientific object of LLM reasoning? This question is methodological as well as empirical. Different answers imply different default targets for explanation, intervention, evaluation, and safety monitoring. 2.2 Competing Hypotheses We formalize the three leading views as three competing hypotheses. H1. Latent-trajectory mediation. Multi-step reasoning is primarily mediated by latent-state trajectories Z . Surface CoT is only a partial interface. It may sometimes report latent commitments, and it may sometimes perturb them, but it is not the default object of reasoning itself. Serial compute matters

Read the complete text