Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning

Donald Ye Max Loffgren Om Kotadia Linus Wong · 2026

Read Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning by Donald Ye Max Loffgren Om Kotadia Linus Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11201.txt.

Description

Step-by-step accounts may not show how a model truly reasons. Donald Ye and four co-authors propose Normalized Logit Difference Decay, a metric that corrupts single thinking steps and measures the drop in answer confidence. Standardized scores allow fair match across designs.

Across three model families and syntactic, logical, and arithmetic tasks, they find a steady Thinking Horizon late in the chain, beyond which tokens barely affect the answer. Models can also hold correct internal forms while failing the task outright. The work shows scores alone cannot prove thinking, and offers a tool to measure when chain-of-thought actually matters.

Catalog record

CREATORDonald Ye Max Loffgren Om Kotadia Linus Wong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11201

Browse the full library · About the Tower of Babel