Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Jia Liang, Liangming Pan · 2026
Read Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks by Jia Liang, Liangming Pan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00449.txt.
Description
Jia Liang and Liangming Pan study CODI, a continuous-thought distillation model, on polynomial iteration tasks with known states. Latent thought enables stepwise computation without rationales, yet its workings remain unclear. Using logit decoding and activation patching, the authors localize states and trace them to readout. In short cases, CODI forms bridge states across positions, while the final input takes a near direct route, with late fusion predictions at readout. As depth grows, CODI does not sustain a full rollout, instead compressing work into a partial pathway or losing it.
Catalog record
| CREATOR | Jia Liang, Liangming Pan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65972 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00449