From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Training Retrieval-Augmented Generation Agents
Muzhi Li, Jinhu Qi, Yihong Wu, Minghao Zhao, Liheng Ma, Yifan Li, Xinyu Wang, Yingxue Zhang, Ho-fung Leung, Irwin King · 2025
Read From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Training Retrieval-Augmented Generation Agents by Muzhi Li, Jinhu Qi, Yihong Wu, Minghao Zhao, Liheng Ma, Yifan Li, Xinyu Wang, Yingxue Zhang, Ho-fung Leung, Irwin King (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23071.txt.
Introduction
Training retrieval agents lacks the step-by-step traces that good supervision needs. Muzhi Li and 11 co-authors synthesize such traces from evidence, working backward. Their paradigm, EviPath, starts from question-answer pairs with supporting evidence. Stage one plans subtasks by abductive reasoning, in dependency order. Stage two answers sub-questions with the evidence as a stand-in environment.
Stage three turns full traces into dialogue form for fine-tuning. An eight-billion-parameter model trained on this corpus beat strong baselines. Exact match on open-domain questions rose by about fifteen points. The corpus gives agents grounded, executable practice paths.
Catalog record
| CREATOR | Muzhi Li, Jinhu Qi, Yihong Wu, Minghao Zhao, Liheng Ma, Yifan Li, Xinyu Wang, Yingxue Zhang, Ho-fung Leung, Irwin King |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23071
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Training Retrieval-Augmented Generation Agents Muzhi Li, Jinhu Qi, Yihong Wu, Minghao Zhao, Liheng Ma, Yifan Li, Xinyu Wang, Yingxue Zhang, Ho-fung Leung, Irwin King Abstract Retrieval-augmented generation agents development is hindered by the lack of process-level supervision to effectively guide agentic capabilities like task decomposition, retriever invocation, and stepwise decision-making. While reinforcement learning offers a potential solution, it suffers from sparse rewards and the limited reasoning capabilities of large language models (LLMs). Meanwhile, existing data synthesis methods only produce chain-of-thought rationales and fail to model environmental interactions. In this paper, we propose EviPath , an evidence-anchored reasoning path synthesis paradigm for RAG agent development. EviPath comprises: (i) Abductive Subtask Planning, which decomposes the problem into sub-questions and iteratively plans an optimal solution path based on the dependencies between them; (ii) Faithful Sub-question Answering, which uses supporting evidence to construct a proxy environment to generate reasoning thoughts and answers for each sub-question; and (iii) Conversational Fine-Tuning, which formats the complete agent-environment interaction trajectory into a dialogue format suitable for Supervised Fine-Tuning. EviPath allows LLMs to learn complex reasoning and tool-use capabilities directly from synthesized data. Extensive experiments on widely-used question-answering benchmarks show that an 8B parameter model trained with EviPath-synthesized data significantly and consistently outperforms state-of-the-art baselines with a double-digit absolute EM gain of 14.7% in open-domain question answering. 1 Introduction Retrieval-augmented generation (RAG) agents, powered by large language models (LLMs) ( Guo et al. 2025 ) , can autonomously gather external knowledge and answer complex, multi-hop questions. Compared to vanilla RAG systems ( Lewis et al. 2020 ) , RAG agents minimize the need for human intervention, and adapt readily to downstream applications like math problem solving ( Zhu et al. 2025 ) , code generation ( Zhang et al. 2023 ) , and financial analysis ( Wang et al. 2025c ) . Despite their promise, RAG agents are hard to develop since ground truth reasoning trajectories are unavailable . Mainstream multi-hop question answering datasets Yang et al. 2018 ; Ho et al. 2020 ; Trivedi et al. 2022 provide final answers and supporting facts, while lacking step-wise supervision that is crucial to equip LLMs with agentic behaviors like question decomposition, search query reformulation, and plan refinement. As a result, existing RAG agents ( Li et al. 2025 ; Xu et al. 2025 ) still fail to deliver reliable performance. One approach to train RAG agents without process supervision is reinforcement learning (RL) ( Shao et al. 2024 ) , which optimizes the decision-making process based on outcome-based rewards. Nevertheless, these methods have notable limitations. First, reward signals are often sparse and delayed, making it difficult to assign credit to individual decisions. In addition, RAG agents contain various non-differentiable components like retrievers and databases, which makes end-to-end gradient backpropagation infeasible. Most critically, the effectiveness of RL relies heavily on the intrinsic reasoning capabilities of the model. Without sufficient prior knowledge, the LLM may fail to discover correct actions that lead to a positive reward, rendering trajectory exploration ineffective . Figure 1: The limitations of reinforcement learning- or data synthesis-based approaches in training RAG agents to answer open-domain questions. Another line of work ( Bai et al. 2024 ; Yang et al. 2025 ) mitigates data scarcity by synthesizing reasoning paths, which leverage LLMs to generate chain-of-thought (CoT) rationales that link questions to answers using the supporting evidence. However, existing data synthesis approaches cannot be adapted to agent training. First, the generated paths often function as post-hoc explanations of predefined answers, rather than a genuine, step-by-step problem-solving process. More importantly, simple CoT training does not endow RAG agents with the core agentic capability to interact with external environments , severely limiting their effectiveness in answering open-domain questions. Considering the aforementioned limitations, we propose EviPath , an Evi dence-anchored reasoning path synthesis framework based on abductive reasoning . EviPath proceeds in three stages that align with the Planner-Executor architecture ( Li et al. 2025 ) of RAG agents: Firstly, the (i) Abductive Subtask Planning stage applies abductive reasoning on the final answer and supporting evidence to reverse-engineer an optimal, dependency-aware reasoning plan, and then simulates the agent’s iterative execution process to generate thoughts and retrieval queries that form the planner’s reasoning path. Secondly, the (ii) Faithful Sub-question Answering stage operates in a simulated environment to bypass retrieval errors, where it identifies the exact evidence for each sub-question to synthesize grounded thoughts and derive an intermediate answer. Finally, the (iii) Conversational Fine-tuning stage packages the complete reasoning paths from previous steps into a user-assistant dialogue format for supervised fine-tuning (SFT). Extensive experiments on widely used QA benchmarks show that an 8B-RAG agent fine-tuned with EviPath trajectories significantly outperforms state-of-the-art baselines, achieving a double-digit absolute EM gain of 14.7% . Our contributions can be summarized in the following four aspects: • We are the first to formulate the synthesis of reasoning paths for RAG agents as an abductive reasoning problem . This novel perspective provides a structured approach for generating interactive, goal-oriented reasoning trajectories. • Building on this formulation, we propose EviPath : a novel framework that synthesizes reasoning paths to solve the dual challenges of data scarcity and reliance on complex reinforcement learning, establishing a data-centric paradigm for RAG agent development . • We construct 265k golden reasoning paths from multi-hop QA benchmarks, specifically designed to enhance agentic skills like high-level planning, retriever use, and context-aware reasoning. • We conduct extensive experiments on three widely used multi-hop QA datasets. The results show that RAG agents trained on EviPath-synthesized data significantly and consistently outperform all state-of-the-art retrieval-augmented generation agents. 2 Related Works 2.1 RAG Agents for QA RAG agents enhance LLMs with external evidence to mitigate hallucinations in knowledge-intensive QA ( Achiam et al. 2023 ; Cheng et al. 2024 ; Huang et al. 2025 ) . The paradigm has evolved from simple “retrieve-then-read” pipelines ( Lewis et al. 2020 ; Gao et al. 2023 ) to sophisticated workflows that interleave reasoning, tool use, and reflection ( Yao et al. 2023 ; Trivedi et al. 2023 ; Asai et al. 2024 ; Shao et al. 2023 ) . Recent works scale agentic QA through learned monologues ( Yang et al. 2024 ) , and modular designs that separate planning from execution ( Li et al. 2025 ; Xu et al. 2025 ; Jiang et al. 2025a ) . Some efforts also leverage reinforcement learning to optimize policies of retrieval and reasoning ( Jin et al. 2025 ; Song et al. 2025a ; Wu et al. 2025 ) . Orthogonal advances strengthen the retrieval side, including query reformulation ( Chan et al. 2024 ; Mao et al. 2024 ) , end-to-end multi-hop retrieval ( Zhang et al. 2024a ) , and knowledge graph integration ( Luo et al. 2025 ; Wang et al. 2025a ; Hao et al. 2025 ) . Despite making great progress, state-of-the-art methods seldom expose fine-grained process supervision that our work introduces to precisely guide LLMs in performing agentic operations like question decomposition, retrieval invocations, and reasoning. 2.2 Reasoning Path Synthesis Data synthesis is common for enhancing the reasoning capabilities of LLMs ( Wang et al. 2023 ; Xiong et al. 2024