Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks

Jia Liang, Liangming Pan · 2026

Read Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks by Jia Liang, Liangming Pan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00449.txt.

Description

Jia Liang and Liangming Pan study CODI, a continuous-thought distillation model, on polynomial iteration tasks with known states. Latent thought enables stepwise computation without rationales, yet its workings remain unclear. Using logit decoding and activation patching, the authors localize states and trace them to readout. In short cases, CODI forms bridge states across positions, while the final input takes a near direct route, with late fusion predictions at readout. As depth grows, CODI does not sustain a full rollout, instead compressing work into a partial pathway or losing it.

Catalog record

CREATORJia Liang, Liangming Pan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65972
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00449

Browse the full library · About the Tower of Babel