ReasoningFlow : Semantic Structure of Complex Reasoning Traces
Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur, Julia Hockenmaier · 2025
Read ReasoningFlow : Semantic Structure of Complex Reasoning Traces by Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur, Julia Hockenmaier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02532.txt.
Introduction
Jinu Lee and co-authors introduce ReasoningFlow, a schema for dissecting the long reasoning traces that large models now produce, in a 2025 paper. Such traces mix planning, reflection, checking, and backtracking in one stream. ReasoningFlow parses a trace into a directed graph whose parts carry semantic labels. Distinct reasoning patterns then appear as recognizable subgraphs. The human readable form aims to support study, grading, and improvement of how reasoning models think.
Catalog record
| CREATOR | Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur, Julia Hockenmaier |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02532
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReasoningFlow : Semantic Structure of Complex Reasoning Traces Jinu Lee, Sagnik Mukherjee, Dilek Hakkani-Tur, Julia Hockenmaier Abstract Large reasoning models (LRMs) generate complex reasoning traces with planning, reflection, verification, and backtracking. In this work, we introduce ReasoningFlow , a unified schema for analyzing the semantic structures of these complex traces. ReasoningFlow parses traces into directed acyclic graphs, enabling the characterization of distinct reasoning patterns as subgraph structures. This human-interpretable representation offers promising applications in understanding, evaluating, and enhancing the reasoning processes of LRMs. 1 Introduction Large language models (LLMs) have shown remarkable step-by-step reasoning ability ( Wei et al. 2022 ; Kojima et al. 2022 ) . In step-by-step reasoning, LLMs first generate traces of intermediate steps that lead to the final answer instead of directly outputting the answer. Recently, LLMs specifically trained for strong reasoning ability (Large Reasoning Models (LRMs); e.g. , DeepSeek-R1 ( Guo et al. 2025 ) , QwQ ( Qwen 2024 ) ) have shown outstanding performance in challenging tasks such as olympiad-level math. These models generate long reasoning traces with complex structures such as planning, verification, and backtracking ( Gandhi et al. 2025 ) . While existing works analyzing the LRM-specific reasoning patterns adopt superficial methods such as keyword match ( Yeo et al. 2025 ) or zero-shot classification ( Gandhi et al. 2025 ) , they fail to address the deep semantic structures of the trace. Figure 1: ReasoningFlow annotates the semantic structure of a complex reasoning trace. Graph-based structure provides rich contextual information about complex cognitive processes like self-verification, as shown in the example. In this work, we propose ReasoningFlow , a fine-grained scheme for annotating the semantic structure of LRMs’ traces. By building an explicit directed acyclic graph structure, ReasoningFlow can characterize diverse reasoning patterns as corresponding subgraph patterns. ReasoningFlow enables deeper understanding of LRM-generated traces, possibly leading to explainable methods for trace validity evaluation and increasing reasoning efficiency. 2 Related works Reasoning traces of LLMs and human-written argumentative texts are both sequences of steps that lead to a specific conclusion ( Liu et al. 2023 ) . Therefore, it motivates investigating the logical structure of LLM-generated traces, as argumentation structures have enabled explainable evaluation ( Stab and Gurevych 2017 ) and compression ( Xu et al. 2020 ) in argumentative texts. One common approach to analyze the logical structure of traces is to use entailment graphs , where premise steps are linked to respective conclusions to form a directed acyclic graph (DAG) ( Ling et al. 2023 ; Mukherjee et al. 2025 ) . However, these works solely relied on LLMs for annotation without any human supervision, and their schema cannot extend to diverse reasoning patterns beyond the basic premise-conclusion relationship. Recent works on LRMs identify LRM-specific reasoning patterns by lexical match ( Yeo et al. 2025 ) or LLM-based classification ( Gandhi et al. 2025 ) . However, these classification-based approaches cannot provide further information about how statements relate ( e.g. , Which previous statement is being verified? ), ignoring the structural aspects of reasoning. 3 ReasoningFlow 3.1 Annotation scheme In ReasoningFlow , each question and reasoning trace is segmented into nodes , which are contiguous, non-overlapping text snippets ranging from clauses to sentences. We define 8 node classes based on their semantic roles in the reasoning, such as Plan, Reasoning, and Reflection (Table 1 ). Edges then connect these nodes to form a labeled DAG, which captures the left-to-right information flow in autoregressive generation ( Ling et al. 2023 ) . ReasoningFlow has three main edge types – reasoning, planning, and evaluation – with a total of 14 fine-grained labels (Table 2 ). ReasoningFlow reveals important reasoning structures, including deductive/inductive reasoning, verification, backtracking, and proof-by-contradiction, as subgraph patterns (Figure 2 ; Appendix B ). Unified graph schema of ReasoningFlow provides richer structural information than existing shallow methods ( Yeo et al. 2025 ; Gandhi et al. 2025 ) . Figure 2: Examples of different reasoning patterns shown in ReasoningFlow graphs. Detailed descriptions about node and edge labels can be found in Appendix A . 3.2 Dataset To construct sample annotations for ReasoningFlow , we sample 30 problems (15 math, 8 chemistry, 7 physics) and QwQ-32B-Preview ( Qwen 2024 ) traces from Sky-T1-data ( NovaSky 2025 ) . The authors manually annotated the entire DAG, from node segmentation, classification, and edge connection using a dedicated web-based interface. Detailed statistics of ReasoningFlow can be found in Appendix C . 4 Potential applications 4.1 Validity evaluation Existing methods for evaluating LLMs’ reasoning traces ( e.g. , Process Reward Models ( Lightman et al. 2023 ) , LLM-as-a-judge ( Gu et al. 2024 ) ) do not generalize to LRM traces due to their verbosity and complex structures ( Lee and Hockenmaier 2025 ) . When evaluating the validity of a step, edge annotations allow using only connected nodes instead of the entire trace ( Mukherjee et al. 2025 ) . Furthermore, ReasoningFlow ’s node annotations allow bypassing non-reasoning steps that PRMs often falsely regard as errors ( e.g. , planning, reflection) ( Kim et al. 2025 ) . 4.2 Increasing efficiency of LRMs LRMs are often criticized for generating overly lengthy reasoning traces for a relatively simple problem ( Sui et al. 2025 ) . As ReasoningFlow can determine which nodes are necessary for deriving the final answer by analyzing the ancestors in the DAG, it can be used to compress long traces for distillation ( Qu et al. 2025 ) or detect unpromising directions during the inference ( Sui et al. 2025 ) . 5 Conclusion In this paper, we introduce ReasoningFlow , an annotation scheme that can annotate complex semantic structures of reasoning traces generated by LRMs. ReasoningFlow annotates different semantic roles of nodes such as Reasoning, Planning, Facts, and Restatements, and connects them into a labeled directed acyclic graph that denotes interactions such as deductive reasoning, verification, and backtracking. ReasoningFlow can be potentially adopted to understand and improve LRMs’ behaviors, including precise evaluation and increasing efficiency. Appendix A ReasoningFlow scheme The objective of ReasoningFlow is to identify the logical structure of Chain-of-thought reasoning traces, especially ones generated by large reasoning models. ReasoningFlow has two main structural components, namely nodes and edges . Nodes Reasoning traces are segmented into nodes, which are syntactically non-overlapping segments from clauses to multi-line paragraphs. The authors manually determined the node boundary based on semantic atomicity. Basic segmentation units are sentences or lines. However, if the sentence contains clauses that might be assigned different node labels, they are separated. For instance, the sentence "I should also consider the possibility of forming 1-butene or other isomers , but given the structure, 2-methylbut-1-ene seems to be the only plausible product." can be segmented by the red comma, where the first segment is a Planning node and the second is a Reasoning node. Each node is assigned a unique label based on its semantic role. The detailed definitions of the node labels are presented in Table 1 . Node Description Context Context corresponds to text provided to the model before the first generated token. It typically includes questions and retrieved documents, if there are any. Planning Planning introduces the content of following nodes. It might be coarse, high-level