Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization
Jiecong Wang, Hao Peng, Chunyang Liu · 2026
Read Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization by Jiecong Wang, Hao Peng, Chunyang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21358.txt.
Introduction
Jiecong Wang, Hao Peng, and Chunyang Liu rethink how language models reason step by step. Writing each step as words works, but it costs a lot of compute. Forcing thought into single tokens can also collapse the range of paths a model considers.
Newer methods let the model think in smooth hidden states instead of words. The authors frame that hidden thinking as planning, split apart from speaking. Keeping the plan separate from the wording should keep more options alive longer. It also promises faster reasoning, since the model skips needless text. The work points toward models that think quietly, then answer once.
Catalog record
| CREATOR | Jiecong Wang, Hao Peng, Chunyang Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21358
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization Jiecong Wang, Hao Peng, Chunyang Liu Abstract Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and reasoning path collapse when grounded in discrete token spaces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, these methods typically operate as opaque end-to-end mappings from explicit reasoning steps to latent states, and often require a pre-defined number of latent steps during inference. In this work, we introduce PLaT ( P lanning with La tent T houghts), a framework that reformulates latent reasoning as planning by fundamentally decouple reasoning from verbalization. We model reasoning as a deterministic trajectory of latent planning states, while a separate Decoder grounds these thoughts into text when necessary. This decoupling allows the model to dynamically determine when to terminate reasoning rather than relying on fixed hyperparameters. Empirical results on mathematical benchmarks reveal a distinct trade-off: while PLaT achieves lower greedy accuracy than baselines, it demonstrates superior scalability in terms of reasoning diversity. This indicates that PLaT learns a robust, broader solution space, offering a transparent and scalable foundation for inference-time search. Our code can be found in https://github.com/yunsaijc/PLaT . 1 Introduction Figure 1: Comparison of PLaT and other reasoning strategies. CoT is an explicit chain-of-thought reasoning method, and the rest are implicit latent reasoning methods. Chain-of-Thought (CoT) reasoning 43 ; 18 ; 42 ; 51 has revolutionized the landscape of Large Language Models (LLMs) by decomposing intractable problems into sequences of intermediate steps 52 . This paradigm has unlocked impressive capabilities across complex domains, serving as the backbone for modern applications ranging from code generation 3 ; 4 ; 20 ; 22 to autonomous agents 47 ; 35 ; 33 . However, it faces a fundamental theoretical bottleneck: reasoning path collapse. At every generation step, the model is forced to sample a discrete token from the vocabulary, thereby pruning the probability of alternative valid reasoning paths 46 ; 50 ; 5 . This nature restricts the model from maintaining a superposition of multiple potential reasoning strategies in high-dimensional space, often leading to irrecoverable errors once a suboptimal token is chosen. Additionally, current models incur high costs by generating prohibitively long sequences of intermediate tokens 50 ; 37 ; 40 ; 11 . To mitigate the inefficiency, recent works have explored latent reasoning, where the model evolves hidden states internally before outputting a final answer 50 ; 44 ; 15 ; 34 ; 38 . While promising, they predominantly adopt an end-to-end implicit paradigm, optimizing latent states directly for the final generation. This leads to two critical limitations. First, the reasoning process is opaque: the intermediate states function as black boxes that cannot be reliably interpreted. Second, and more critically, these methods rely on a fixed number of latent steps during inference. This forces the model to expend the same computational effort regardless of problem difficulty, lacking the flexible nature of human System 2 thinking 21 . We argue that a robust reasoning system should mirror the cognitive distinction between thought and language. From a cognitive perspective, language serves merely as a low-dimensional projection (interface) of high-dimensional thought; the core reasoning process often occurs implicitly without verbalization 39 ; 10 ; 7 ; 9 . Ideally, the “brain” should maintain a superposition of potential reasoning trajectories within a continuous latent space, collapsing to discrete decisions only when an interface with the external world (the “mouth”) is required. Motivated to replicate this implicit process computationally, we draw inspiration from Multi-Token Prediction (MTP) 36 ; 29 ; 13 ; 27 ; 32 (that transformer hidden states are able to encode information about future tokens before they are generated) and propose to model reasoning as a sequence of latent planning states. In this work, we introduce PLaT ( P lanning with La tent T houghts), a framework that fundamentally decouples the reasoning process from verbalization. Our architecture comprises two distinct components: a latent Planner and a Decoder for verbalization. The Planner autoregressively evolves a trajectory of states in a high-dimensional continuous manifold, maintaining a probabilistic density over multiple logical possibilities until a decision is required. The Decoder grounds these latent plans onto the language space via a reconstruction objective. This empowers PLaT with dynamic termination of latent planning and intermediate interpretability of latent states, unlike prior methods that use a fixed number of latent steps. Our empirical evaluations on mathematical benchmarks reveal a distinctive behavioral pattern. We identify a trade-off between greedy precision and exploration potential: while PLaT achieves lower greedy accuracy compared to baselines, it exhibits superior scalability in reasoning diversity. PLaT outperforms baselines in Pass@k metrics with a steeper scaling slope, indicating that it learns a broader solution space rather than overfitting to a narrow trajectory. Furthermore, the latent states in PLaT can be decoded into text for interpretability without disrupting the continuous reasoning flow. Our main contributions are summarized as follows: • We reformulate latent reasoning as planning over latent space, shifting from implicit pattern matching to planning in continuous space. • We introduce a decoupled Planner-Decoder architecture that separates latent reasoning from language generation. This design naturally enables interpretable intermediate reasoning and dynamic inference termination. • Experiments show superior Pass@k scaling and reduced diversity saturation under search-based inference, framing a distinct trade-off between greedy precision and exploration potential. 2 Related Work Figure 2: Framework of the proposed PLaT paradigm. (1) SFT Stage: the Planner autoregressively steps forward to generate the latent states in the context of the question. The Decoder then utilizes the projected latent states as the prefix to verbalize them. (2) RL Stage: the Decoder decodes the same states with a sampling strategy to roll out different results. Equations that are valid in the corresponding reasoning process and correct answers reinforce the Decoder as a policy. 2.1 Chain-of-thought Reasoning Chain-of-Thought (CoT) prompting enables large language models to solve complex problems by explicitly decomposing them into intermediate reasoning steps 43 . Subsequent work shows that CoT can be activated with minimal prompting by a single instruction such as “Let’s think step by step” 18 . Beyond simple CoT strategies, self-consistency 42 samples multiple CoT trajectories and marginalizes over final answers, improving robustness by approximating a distribution over reasoning paths. Tree-of-Thought (ToT) 46 ; 23 ; 26 generalizes CoT into an explicit tree search, allowing the model to branch, evaluate, and backtrack over intermediate states. Graph-of-Thought (GoT) 48 ; 2 ; 49 further extends this paradigm to graph-structured reasoning, enabling the reuse and recombination of intermediate conclusions across different branches. Orthogonal to structured search, other works focus on refining intermediate reasoning steps through iterative revision or verification. These approaches improve correctness by critiquing, editing, or validating generated reasoning traces, often in a multi-pass manner 25 ; 24 ; 45 . 2.2 Latent Reasoning Recent latent reasoning methods aim to reduce the cost and path collapse of explicit CoT by shifting part of the reasoning process into continuous hidden states, while retaining the final answer in natural language. Early approaches explore partial internalization of reasoning, where additional non-semantic structures are introduced during inference. For example, pause tokens or planning tokens allow models to suspend