Reasoning Can Be Restored by Correcting a Few Decision Tokens
Changshuo Shen, Leheng Sheng, Yuxin Chen, An Zhang, Xiang Wang · 2026
Read Reasoning Can Be Restored by Correcting a Few Decision Tokens by Changshuo Shen, Leheng Sheng, Yuxin Chen, An Zhang, Xiang Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16874.txt.
Introduction
Shen's team studies where base models fail versus stronger thinkers in token making. Gauging token-level gaps with likelihood splits, they find the thinking edge is sparse, focused on few early planning tokens. On Qwen3-0.6B, 8% of tokens drive gaps, focused early, 17-fold dense in planning, and tied to high base doubt. Their gap-guided step-in lets the thinking model take over one token at high-gap spots, then switches back. With a budget, this sparse handoff largely rebounds and can surpass same-size thinking-model scores.
Catalog record
| CREATOR | Changshuo Shen, Leheng Sheng, Yuxin Chen, An Zhang, Xiang Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69547 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16874
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning Can Be Restored by Correcting a Few Decision Tokens Changshuo Shen, Leheng Sheng, Yuxin Chen, An Zhang, Xiang Wang Abstract Large reasoning models (LRMs) substantially outperform their base LLM counterparts on challenging reasoning benchmarks , yet it remains poorly understood where base models go wrong during token-by-token generation and how to narrow this gap efficiently. We study the base–reasoning gap through quantifying token-level distributional disagreement between a base model and a stronger reasoning model using likelihood-based divergences . Across benchmarks, we find that the reasoning advantage is highly sparse and concentrates on a small set of early, planning-related decision tokens . For instance, on Qwen3-0.6B, only ∼ \sim 8% of generated tokens account for the salient disagreement, and these tokens concentrate early in the response, are strongly enriched in planning-related decisions ( 17 × 17\times ), and coincide with high base-model uncertainty—suggesting that base models fail mainly at early planning points that steer the subsequent reasoning trajectory. Building on this findings, we propose disagreement-guided token intervention, a simple inference-time delegation scheme that performs a one-token takeover by the reasoning model only at high-disagreement positions and immediately switches back to the base model. With a small intervention budget, this sparse delegation substantially recovers and can even surpass the performance of a same-size reasoning model on challenging reasoning tasks. Code is available at https://github.com/AlphaLab-USTC/RRTokenIntervention . † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : Disagreement-guided token intervention and its effectiveness. Intervening on 8% critical tokens, triggered by high token-level distributional disagreement through a strong LRM ( i.e., top figure), enables a 0.6B base model to recover most reasoning capabilities of the 8B strong LRM ( i.e., bottom figure) and outperform its post-trained 0.6B variant. Large reasoning models (LRMs) have recently demonstrated substantial advantages over their base LLM counterparts on challenging reasoning benchmarks ( 55 ; 19 ; 30 ; 10 ; 28 ; 11 ) , with gains commonly attributed to test-time scaling ( 23 ; 36 ; 44 ; 51 ) and reinforcement learning-based post-training such as RLVR ( 35 ; 48 ) . Beyond further improving performance, this widening base–LRM gap has motivated growing interest in a more fundamental question: what differs between base and reasoning models, and why does “reasoning mode” work? A growing body of evidence supports a latent-capability view in which base models may already encode considerable reasoning machinery, while post-training mainly activates, amplifies, or stabilizes it through minimal additional signals—such as activation-vector steering ( 40 ; 42 ; 58 ; 46 ) , confidence-based self-rewarding ( 53 ; 12 ) , distribution-level output amplification ( 54 ) , or even one-example unlocks ( 45 ) . To address the why question, prior work has proposed hypotheses at multiple levels. Reasoning capabilities may emerge from eliciting richer intermediate trajectories, as in chain-of-thought prompting ( 47 ; 57 ; 44 ) ; from inducing reasoning micro-behaviors such as backtracking ( 51 ; 52 ) , verification ( 24 ) , and self-correction ( 26 ; 32 ; 27 ) ; from steering reasoning direction by restricting policy gradient updates to high-entropy tokens ( 43 ) ; or from updating only a small subset of parameters, i.e., a top-singular reasoning subspace ( 4 ; 16 ) . Despite these insights, we still lack a simple token-level account of why “reasoning mode” works—namely, where reasoning ability actually manifests during token generation and which decision points truly drive the base–LRM performance gap. In this paper, we address this question by quantifying the base–LRM gap through token-level distributional disagreement between a base model and a stronger reasoning model using likelihood-based divergences such as cross-entropy and reverse Kullback–Leibler divergence ( 7 ) . We then characterize which tokens carry the reasoning advantage, how many such tokens exist, and what properties they exhibit. Across benchmarks, we find that reasoning ability is concentrated in a surprisingly small set of planning-related decision tokens that more likely appear early in the response. Concretely, only a small fraction of generated tokens determines whether the model commits to a reasoning trajectory: on Qwen3-0.6B ( 50 ) , merely \sim 8% of output tokens account for the salient base–LRM disagreement ( i.e., disagreement spike) ( cf. Figure 1 ), indicating that token-level disagreement is highly sparse. Moreover, these disagreement spikes appear disproportionately early in generation ( cf. Figure 2 ), contrary to the common intuition that reasoning gains primarily stem from long traces after the reasoning process unfolds. Finally, these critical tokens are tightly linked to planning under uncertainty: during recovery from base to reasoning rollouts, planning-related tokens become markedly more likely, increasing by 17\times in occurrence probability ( cf. Table 5 ), and they correlate strongly with the base model’s intrinsic uncertainty ( cf. Figure 2(c) ). This suggests that base models struggle on complex reasoning tasks primarily at the planning points—where they are least confident are precisely the planning decisions that steer the subsequent reasoning trajectory. Motivated by these findings, we ask whether a base model can attain strong reasoning performance with limited assistance from a reasoning model—by delegating only a small number of critical decisions. We propose a simple inference-time mechanism as shown in Figure 1 : at positions where the base–LRM disagreement score is large, we perform a one-token takeover—the reasoning model generates only the next token at that position, after which generation immediately switches back to the base model ( 29 ; 17 ) . We term this sparse delegation scheme disagreement-guided token intervention (Figure 3 ). Experiments show that with a small intervention budget, this approach can substantially recover—and in some settings even surpass—the performance of a same-size reasoning model on challenging reasoning tasks. Together, these findings motivate a token-level view of reasoning: a small number of early planning commitments can steer the entire trajectory and dominate the reasoning gap. 2 Related Work Reasoning capabilities in base models. Reasoning behaviors can emerge in small or base models when they are exposed to appropriate training signals, such as reasoning exemplars or Chain-of-Thought ( 47 ) traces ( 15 ) . Explicit alignment, including fine-tuning on structured rationales, often suffices to elicit multi-step problem solving in models that previously answered directly ( 15 ) . Reinforcement learning has been shown to gradually restructure output patterns, increasing response length and inducing more consistent use of intermediate reasoning steps ( 10 ; 45 ) . Activation-space methods such as steering or targeted editing demonstrate that reasoning patterns, like backtracking, can be causally induced in base models without weight updates ( 42 ; 46 ) . Inference-time approaches, such as CoT prompting or self-verification feedback, can also trigger reasoning behaviors that do not naturally surface in zero-shot settings ( 47 ; 8 ) . Collectively, these findings suggest that base models possess a wide range of reasoning capabilities that can be activated through post-training mechanisms, guidance signals, or structural interventions. Token-level distributional disagreement. Cross-entropy, entropy, and divergence offer effective signals for identifying decision-critical positions where models diverge or hesitate during generation. These signals can help locate forking tokens, error-prone steps, or places where weak models diverge sharply from stronger references. KL divergence , in both forward and reverse form,has been used to compare different models’ behavior. Forward