A Survey on Latent Reasoning
Rui-Jie Zhu ⋆,† , Tianhao Peng ⋆ , Tianhao Cheng ⋆ , Xingwei Qu ⋆ , Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian · 2025
Read A Survey on Latent Reasoning by Rui-Jie Zhu ⋆,† , Tianhao Peng ⋆ , Tianhao Cheng ⋆ , Xingwei Qu ⋆ , Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06203.txt.
Introduction
Rui-Jie Zhu and a large group of co-authors survey latent reasoning in large language models, in a 2025 paper. Chain-of-thought prompting made models spell out their steps, which helps accuracy but spends the model's bandwidth on words.
Latent reasoning instead runs the steps inside the model's hidden states, with no words in between. The survey maps the young field from the ground up. It starts with how stacked layers support such inner computation. It then covers methods based on repeated activation, carried hidden states, and training that folds written traces inward. A final part treats endless-depth schemes built on masked diffusion models. The authors aim to give a scattered literature one shared frame.
Catalog record
| CREATOR | Rui-Jie Zhu ⋆,† , Tianhao Peng ⋆ , Tianhao Cheng ⋆ , Xingwei Qu ⋆ , Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06203
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Survey on Latent Reasoning Rui-Jie Zhu ⋆,† , Tianhao Peng ⋆ , Tianhao Cheng ⋆ , Xingwei Qu ⋆ , Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian Yang, Wangchunshu Zhou, Chujie Zheng, Chongxuan Li, Yuyin Zhou, Zhoujun Li, Zhaoxiang Zhang, Jiaheng Liu † , Ge Zhang † , Wenhao Huang, Jason Eshraghian † UCSC, FDU, NJU, PKU, RUC, UoM, UW-Madison, PolyU, M-A-P Abstract Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model’s expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model’s continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: LatentCoT-Horizon . † † footnotetext: ⋆ Equal Contribution. † Corresponding Authors. Figure 1 : Explicit reasoning transmits discrete tokens ( \approx 15\,\text{bits} each), whereas latent reasoning exchanges full 2560-dimensional FP16 hidden states ( \approx 40,960\,\text{bits} each), revealing a \sim\!2.7\times 10^{3} -fold bandwidth gap between the two approaches. 1 Introduction Large Language Models (LLMs) have demonstrated remarkable capabilities in performing reasoning tasks, in some cases even exceeding human-level performance [ 64 , 133 , 47 , 82 ] . LLMs often reason more effectively when they produce a Chain-of-Thought (CoT) [ 115 ] , spelling out each intermediate step in natural language before arriving at a final answer. Initially viewed as a logical extension to prompt engineering, CoT gained traction once supervised instruction tuning exposed models to many annotated reasoning traces. It then became the norm when RL rewarded answer correctness [ 49 ] , which encouraged models to generate their own effective chains of thought. As a result, LLMs that “think in language before answering” have attained remarkable performance improvements. This principle now anchors leading reasoning models, including the Qwen3 series [ 118 ] , DeepSeek-R1 [ 41 ] , and Gemini 2.5 series [ 32 ] . However, just as humans do not always rely on language for their cognitive processes, LLMs spend most of their processing budget in the latent space. Enforcing a CoT to operate with natural language can constrain a model’s expressive range and can also impose redundant computation. Latent Chain-of-Thought (Latent CoT) has the potential to overcome these limits [ 23 , 44 ] . Unlike its explicit counterpart that relies on discrete tokens, latent CoT carries reasoning in continuous internal representations, often via recurrent mechanisms within the model. This offers richer expressivity and access to non-linguistic reasoning paths, potentially unlocking new frontiers in model reasoning. This survey examines the emerging landscape of Latent CoT and its potential to surpass language-based reasoning constraints. While explicit CoT forces thoughts into a string of tokens, Latent CoT shifts the entire reasoning process into the model’s continuous representational space. The aim is to expand expressiveness and raise the performance ceiling: freed from a finite vocabulary, a model can explore reasoning trajectories with no direct linguistic equivalent. We categorize and analyze the technical approaches that leverage these continuous representations to achieve more advanced reasoning. The structure of this survey is designed to provide a comprehensive understanding of Latent CoT and its various implementations. Our taxonomy breaks this down in Figure 2 . We begin by establishing a general formulation that captures most Latent CoT implementations, before classing techniques into more specific categories. These categories can be broadly divided into two types: 1) vertical recurrence for expanding computational depth, and 2) horizontal recurrence for increasing sequential capacity. Vertical recurrence applies feedback loops to activation values, and can be thought of ‘activation-based’ reasoning [ 22 , 71 ] . Alternatively, horizontal recurrence uses hidden states to propagate context across long sequences of reasoning trajectories [ 100 , 87 ] . We then explore fine-tuning strategies designed to compress or internalize explicit reasoning traces, which concludes the review of Latent CoT implementations. This sets the stage for understanding the mechanistic interpretability of latent reasoning to understand how these processes are realized within neural networks. This section examines the foundational role of network layers as the primary computational substrate for reasoning [ 137 , 92 ] . We explore the theory of Layer Specialization, which posits that different layers develop distinct, hierarchical functions—from feature extraction in shallow layers to complex logical operations in intermediate layers and final integration in deep layers—collectively forming an implicit computational pipeline analogous to an explicit CoT. Explicit CoT comes with the benefit of intermediate tokens which offers a degree of post-hoc interpretability, and we similarly aim to uncover the mechanisms that enable latent reasoning. Finally, we explore advanced paradigms at the frontier of LLM cognition, focusing on the pursuit of infinite-depth reasoning. This concept refers to a model’s ability to devote unbounded computational steps to refine a solution, moving beyond fixed-depth architectures. Our discussion centers on spatial infinite reasoning as realized by text diffusion models [ 124 , 74 ] . Unlike traditional autoregressive generation, these models operate on the entire output sequence in parallel, enabling global planning and iterative self- correction through bidirectional context. This approach facilitates globally consistent and reversible reasoning processes, offering a promising path toward more powerful and flexible AI systems. {forest} Figure 2 : Taxonomy of Latent Reasoning. 2 Preliminary: Latent Chain-of-Thought In this section, we present a unified mathematical framework for understanding various Latent CoT approaches. Unlike traditional CoT reasoning that generates explicit textual intermediate steps, latent CoT methods perform reasoning through continuous representations and hidden states within the model’s computational graph. We categorize these approaches based on how they propagate information across layers (spatial dimension) and time steps (temporal dimension). 2.1 General Framework We begin by establishing a general formulation for transformer-based reasoning systems. Consider a transformer model processing information at time step t and layer l . Let x_{t}^{l}\in\mathbb{R}^{d} denote the activation at layer l and time t . We introduce \mathbf{S}_{t}^{l} to represent the hidden state that captures historical information. The structure and dimensionality of \mathbf{S}_{t}^{l} are architecture-dependent and define how context is maintained. This state can manifest in several forms, including: • KV Cache: In standard Transformers, \mathbf{S}_{t}^{l} is the Key-Value (KV) cache, comprising a pair of matrices (\mathbf{K}_{t}^{l},\mathbf{V}_{t}^{l}) , where \mathbf{K}_{l},\mathbf{V}_{l}\in\mathbb{R}^{n\times d} and n is the sequence length of the context. Note that as t increases, so does n