Reasoning Fails Where Step Flow Breaks
Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su, Xiaofeng Zhang · 2026
Read Reasoning Fails Where Step Flow Breaks by Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su, Xiaofeng Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06695.txt.
Introduction
Xu and six colleagues diagnose why long reasoning traces go astray. Their Step-Saliency tool pools attention-gradient scores into step-to-step maps across the question, thinking, and summary stages. Two failures recur. In shallow lock-in, early layers fixate on the current step and neglect earlier context. In deep decay, deep layers lose grip on thinking and the summary attends mostly to itself. Their StepFlow intervention repairs shallow patterns and adds a small deep-layer residual, lifting accuracy on math, science, and coding tasks without retraining.
Catalog record
| CREATOR | Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su, Xiaofeng Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51647 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06695
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning Fails Where Step Flow Breaks Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su, Xiaofeng Zhang Abstract Large reasoning models (LRMs) that generate long chains of thought now perform well on multi-step math, science, and coding tasks. However, their behavior is still unstable and hard to interpret, and existing analysis tools struggle with such long, structured reasoning traces. We introduce Step-Saliency , which pools attention–gradient scores into step-to-step maps along the question–thinking–summary trajectory. Across several models, Step-Saliency reveals two recurring information-flow failures: Shallow Lock-in , where shallow layers over-focus on the current step and barely use earlier context, and Deep Decay , where deep layers gradually lose saliency on the thinking segment and the summary increasingly attends to itself and the last few steps. Motivated by these patterns, we propose StepFlow , a saliency-inspired test-time intervention that adjusts shallow saliency patterns measured by Step-Saliency via Odds-Equal Bridge and adds a small step-level residual in deep layers via Step Momentum Injection. StepFlow improves accuracy on math, science, and coding tasks across multiple LRMs without retraining, indicating that repairing information flow can recover part of their missing reasoning performance. Code is available at https://github.com/XiaoyuXu-Vincent/step-saliency . † † footnotetext: † Corresponding author, ‡ Project leader 1 Introduction Large reasoning models (LRMs) generate an internal thinking segment before producing a final summary, forming a question–thinking–summary trajectory. These models achieve strong performance on multi-step mathematics, code generation, and scientific question answering 2 ; 35 ; 53 . Despite these gains, LRMs still exhibit persistent failure modes, including unfaithful chains of thought 34 ; 31 ; 5 , overconfident hallucinations 15 ; 9 , and brittleness on compositional tasks 40 ; 30 ; 7 ; 39 ; 11 . When an LRM makes a mistake, we lack a clear way to attribute the final error to the model’s internal reasoning trace. This motivates a diagnostic that tracks step-to-step influence across depth and links shifts in that influence to incorrect final answers 32 ; 8 ; 55 . Prior work offers several token-level diagnostics for inspecting LRMs during generation. Attention-based analyses visualize which tokens receive attention when producing later tokens 44 ; 56 , but attention weights are not always faithful indicators of what actually drives the prediction 21 . Saliency-based methods instead assign gradient-weighted importance to earlier tokens 3 ; 52 , yet along long reasoning traces the scores can be noisy and difficult to aggregate across positions. The main difficulty is not the lack of signals but the lack of a readable unit aligned with reasoning steps: token-level maps are dense, local, and do not naturally summarize step-to-step dependence. As Figure 1 shows, even when correct and error traces differ, the signal is dispersed across many tokens, making it difficult to summarize which steps remain influential later and how this changes with depth. Figure 1: From token- to step-level saliency. Token-level saliency maps are dense and noisy; Step-Saliency pools them into question/thinking/summary blocks. Correct traces show smooth question \rightarrow thinking \rightarrow summary flow, while errors exhibit shallow lock-in and weak thinking \rightarrow summary links. To address this limitation, we introduce Step-Saliency , a step-level diagnostic for long-form reasoning. We compute token saliency using attention–gradient influence and then pool the scores within each step. This yields a compact step \to step map that is easy to compare across layers. Applying Step-Saliency to several LRMs, we observe two common patterns in incorrect outputs. In shallow layers, influence concentrates on the current thinking step and its immediate neighbors, while earlier context is suppressed; we call this Shallow Lock-in . As depth increases, saliency on the thinking segment weakens for both correct and incorrect outputs, but it fades much faster on incorrect ones, and the summary becomes more self-focused; we call this Deep Decay (§ 3.2 ). Building on these observations, we propose StepFlow , a lightweight test-time intervention that modifies a model’s forward pass without retraining or backpropagation. StepFlow has two components that target the two failure patterns. Odds-Equal Bridge (OEB) is applied to the first few layers: when the model is generating a thinking step, it prevents step-level mass from collapsing onto the just-written tokens by keeping a non-trivial share on the question and earlier steps. Step Momentum Injection (SMI) is applied to the last few layers: at step boundaries, it carries a small residual summary of the previous step into the next step, so earlier reasoning remains available when the summary is produced. Together, OEB and SMI promote a steadier question–thinking–summary linkage during generation. We evaluate StepFlow on DeepSeek-R1-Distill (7B/14B/32B), GPT-OSS-20B, and QwQ-32B-Preview across six benchmarks: AIME24, AIME25, AMC23, MATH-500, GPQA-Diamond, and LiveCodeBench. StepFlow consistently improves accuracy across all backbones, with the largest gains on competition-style problems that require long reasoning chains. Our main contributions are as follows: • We introduce Step-Saliency , a diagnostic that aggregates token-level saliency into step \to step maps, making long reasoning traces interpretable at the step level. • We identify two depth-wise information-flow failure patterns in large reasoning models, which we term Shallow Lock-in and Deep Decay ; they reliably separate incorrect from correct traces in our saliency analysis. • We propose StepFlow , a test-time intervention that reshapes information flow via Odds-Equal Bridge and Step Momentum Injection, improving accuracy across multiple benchmarks without retraining. Figure 2: Step-Saliency patterns for shallow vs. deep layers and correct vs. error traces. Top: depth-collapsed step \to step saliency maps; darker red indicates stronger influence between steps. Bottom: schematic diagrams summarizing the observed patterns—red arrows denote narrow, local flow ( Shallow Lock-in and Deep Decay in error traces), while blue arrows denote broad, long-range flow in correct traces. 2 Related Work 2.1 Large Reasoning Models Chain-of-thought prompting (CoT) 50 substantially improved LLM reasoning and spurred Large Reasoning Models (LRMs) with strong multi-step reasoning and structured intermediate traces. Although LRMs demonstrate strong performance on multi-step reasoning benchmarks 24 ; 10 ; 53 ; 25 , their reasoning processes still suffer from instability and limited interpretability. Beyond standard CoT prompting, test-time scaling methods such as self-consistency and sampled-path selection improve robustness 49 ; 33 . Other work builds multi-step frameworks with tools or self-improvement 28 ; 29 ; 57 ; 14 . Our understanding of how information is propagated, transformed, and possibly lost inside LRMs remains incomplete, which makes it difficult to systematically analyze reasoning failures, incorrect inferences, and inconsistent answers 40 ; 30 ; 11 . 2.2 Layer-wise Reasoning Behavior Many studies employing layer-wise analysis have revealed that Transformer-based LLMs exhibit a clear functional division across network depth during reasoning 16 . In the shallow layers, the model primarily performs local and formal pattern recognition, such as extracting n-gram features 38 ; 45 and handling structural information 42 . In the deep layers, the model shifts toward cross-sentence and cross-concept integration, handling semantic alignment 37 , long-range dependency modeling 48 , and the global organization of the reasoning chain 13 . 2.3 Information Flow To study how information moves inside LLMs, prior work has proposed a range of information-flow analyses 59 ; 32 ; 8 . Attention-based methods inspect attention weights or head activations 44 ; 56 to show which tokens the model focuses on. They are often used to infer