Large Language Models Decide Early and Explain Later

Ayan Datta, Zhixue Zhao, Bhuvanesh Verma, Radhika Mamidi, Mounika Marreddy, Alexander Mehler · 2026

Read Large Language Models Decide Early and Explain Later by Ayan Datta, Zhixue Zhao, Bhuvanesh Verma, Radhika Mamidi, Mounika Marreddy, Alexander Mehler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22266.txt.

Description

Datta and five colleagues study when Qwen3-4B's final answer is actually fixed during long chain-of-thought reasoning. Using forced answer completion to read off intermediate predictions, they find predicted answers change in only 32 percent of queries. Once the final answer switch occurs, the model still generates an average of 760 extra reasoning tokens per query. Simple early-stopping rules, including probe-based stopping, cut token use by 500 per query at a cost of only 2 percent accuracy. Much chain-of-thought output is thus redundant, the authors argue.

Catalog record

CREATORAyan Datta, Zhixue Zhao, Bhuvanesh Verma, Radhika Mamidi, Mounika Marreddy, Alexander Mehler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62233
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22266

Browse the full library · About the Tower of Babel