Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
Donald Ye Max Loffgren Om Kotadia Linus Wong · 2026
Read Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning by Donald Ye Max Loffgren Om Kotadia Linus Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11201.txt.
Description
Step-by-step accounts may not show how a model truly reasons. Donald Ye and four co-authors propose Normalized Logit Difference Decay, a metric that corrupts single thinking steps and measures the drop in answer confidence. Standardized scores allow fair match across designs.
Across three model families and syntactic, logical, and arithmetic tasks, they find a steady Thinking Horizon late in the chain, beyond which tokens barely affect the answer. Models can also hold correct internal forms while failing the task outright. The work shows scores alone cannot prove thinking, and offers a tool to measure when chain-of-thought actually matters.
Catalog record
| CREATOR | Donald Ye Max Loffgren Om Kotadia Linus Wong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11201