When Shallow Wins: Silent Failures and the Depth–Accuracy Paradox in Latent Reasoning
Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary, [4pt] Independent · 2026
Read When Shallow Wins: Silent Failures and the Depth–Accuracy Paradox in Latent Reasoning by Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary, [4pt] Independent (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03475.txt.
Description
Subramanyam Sahoo and three others show that a top mathematics model often reaches right answers by unreliable roads. Latent reasoning models, which think internally rather than writing steps, now serve in tutoring and decision support. Their 2026 audit of a leading seven-billion-parameter math model finds most of its correct predictions, about four in five, emerge through computationally inconsistent pathways. Only a minority rest on stable, faithful reasoning.
Nearly one prediction in eleven is a silent failure: confident and wrong. Measured reasoning quality correlates slightly negatively with correctness, an artifact of a binary scoring threshold. Growing the model nearly fivefold in size brought zero accuracy gain on the evaluated slice, though the authors flag that slice as small. About a fifth of latent plans resemble ordinary chain-of-thought. Their conclusion: headline accuracy can mask deep unreliability, and evaluation must measure stability directly.
Catalog record
| CREATOR | Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary, [4pt] Independent |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45715 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.03475