Credible plan-driven RAG method for Multi-hop Question Answering

Ningning Zhang, Chi Zhang, Zhizhong Tan, Xingxing Yang, Weiping Deng, Wenyong Wang · 2025

Read Credible plan-driven RAG method for Multi-hop Question Answering by Ningning Zhang, Chi Zhang, Zhizhong Tan, Xingxing Yang, Weiping Deng, Wenyong Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16787.txt.

Introduction

Ningning Zhang and co-authors tackle questions that need several chained steps of searching and reasoning, in a 2025 paper. Retrieval-augmented generation lets a language model look facts up before answering. For multi-hop questions, where each answer feeds the next search, systems wobble in two ways: their reasoning path drifts, and their facts go unchecked. Earlier fixes handled one flaw or the other, rarely both.

Their framework, PAR-RAG, runs a three-stage cycle of plan, act, and review. It first judges the question's complexity and picks worked examples of matching difficulty, so the plan it writes has the right grain. Searching and reading then follow the plan. Finally, a double check hunts for errors mid-chain and corrects them, with stricter checks for harder questions. The authors report steadier reasoning and cleaner facts together.

Catalog record

CREATORNingning Zhang, Chi Zhang, Zhizhong Tan, Xingxing Yang, Weiping Deng, Wenyong Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16787

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Credible plan-driven RAG method for Multi-hop Question Answering Ningning Zhang, Chi Zhang, Zhizhong Tan, Xingxing Yang, Weiping Deng, Wenyong Wang Abstract Multi-hop question answering (QA) presents a considerable challenge for Retrieval-Augmented Generation (RAG), requiring the structured decomposition of complex queries into logical reasoning paths and the generation of dependable intermediate results. However, deviations in reasoning paths or errors in intermediate results, which are common in current RAG methods, may propagate and accumulate throughout the reasoning process, diminishing the accuracy of the system’s responses to complex queries. To address this challenge, we propose the P lan-then- A ct-and- R eview (PAR RAG) framework, which is organized into three key stages—planning, act, and review—and aims to offer an interpretable and incremental reasoning paradigm for accurate and reliable multi-hop question answering by mitigating error propagation. PAR RAG initially applies a top-down problem decomposition strategy, formulating a comprehensive plan that integrates multiple executable steps from a holistic viewpoint. This approach avoids the pitfalls of local optima common in traditional RAG methods, ensuring the accuracy of the entire reasoning path. Subsequently, PAR RAG incorporates a plan execution mechanism based on multi-granularity verification. By utilizing both coarse-grained similarity information and fine-grained relevant data, the framework thoroughly checks and adjusts intermediate results, ensuring process accuracy while effectively managing error propagation and amplification. Experimental results on multi-hop QA datasets demonstrate that the PAR RAG framework substantially outperforms existing state-of-the-art methods in key metrics, including EM and F1 scores. K eywords RAG \cdot Open-domain question answering \cdot Multi-hop \cdot Large large model 1 Introduction Multi-hop question answering (QA) [ 1 ] requires the integration of multiple information sources and multi-step reasoning to derive an answer and has wide-ranging applications in fields such as complex decision-making and deep analysis [ 2 ] . In recent years, Retrieval-Augmented Generation (RAG) [ 3 ] has made significant progress in multi-hop QA tasks [ 4 ] , achieving significant advancements [ 5 , 6 ] . However, it still faces several key challenges, particularly deviations in reasoning paths and errors in intermediate results. First, existing research highlights that prevalent failures in multi-hop QA tasks are often caused by deviations in reasoning paths, typically stemming from errors in executing the first hop or from inconsistencies in the reasoning paths [ 7 , 8 , 9 , 10 ] , underscoring the critical importance of problem decomposition. If the problem decomposition leads to an erroneous reasoning path, subsequent steps become challenging to correct, ultimately resulting in failures. Second, during the subproblem-solving process, the accuracy of intermediate results is vulnerable to issues such as interference from irrelevant information, position bias, or knowledge conflict [ 11 , 12 , 13 , 14 ] . Moreover, deviations in the reasoning path and errors in intermediate steps are propagated and amplified at each stage of the multi-step reasoning process, eventually triggering a butterfly effect [ 15 ] that substantially reduces the accuracy of answers to complex queries. Humans demonstrate distinct cognitive superiority over artificial intelligence (AI) when addressing complex multi-step reasoning tasks. Professionally trained individuals typically employ a structured cognitive framework, such as the PDCA cycle [ 16 ] , to develop a comprehensive plan based on a clear understanding of the problem’s intent, using a top-down approach. They then execute and verify each step in accordance with the plan, always with the goal of generating a final answer. The human cognitive process for complex problem-solving consists of the following critical phases: 1. Global Planning Phase : From a holistic perspective, complex problems are decomposed into logically interconnected yet independently solvable subproblems. This structural decomposition constructs coherent reasoning pathways, effectively preventing deviations in the reasoning path. 2. Plan Execution and Multi-granularity Verification Phase : During plan execution, when encountering insufficient local information, coarse-grained data with semantic similarity to the current subproblem are first retrieved to generate a preliminary answer, thereby enhancing the understanding of the problem. Subsequently, guided by the interim answer, fine-grained data with rich semantic correlations are further retrieved. A cross-verification mechanism then facilitates the refinement of provisional solutions, constructing an error-constrained reasoning trajectory. This granular retrieval-verification mechanism significantly mitigates the risk of error accumulation in intermediate results. 3. Synthesis and Reasoning Phase : Upon resolving the subproblems, humans synthesize the complex problem with the entire reasoning trajectory to derive the final answer. The PDCA cycle, as a systematic improvement method, is grounded in the core principle of continuously enhancing result quality through an iterative closed-loop process comprising the Plan-Do-Check-Act stages. This method aligns with the requirements of multi-hop question answering, where the complexity of multi-hop QA necessitates a system capable of solving problems in distinct, verifiable stages. Specifically, the planning phase corresponds to problem decomposition, the execution phase involves evidence retrieval and reasoning, the checking phase entails result verification, and the action phase focuses on refining results based on the verification outcomes. Inspired by this, we propose an innovative RAG framework, PAR RAG, which integrates the P lan-then- A ct-and- R eview methodology from the PDCA (Plan-Do-Check-Act) model with the reading comprehension and reasoning capabilities of large language models (LLMs) [ 17 , 18 , 19 , 20 , 21 , 22 ] . In the planning phase, PAR RAG systematically decomposes complex problems, ensuring a comprehensive understanding of the query’s intent. This approach converts the problem into a structured plan with multiple reasoning steps, effectively mitigating the tendency of iterative reasoning to converge on local optima, which can lead to deviations in the reasoning path. In the act (plan execution) phase, the framework retrieves coarse-grained data to generate an initial answer based on the query. Subsequently, the interim answer is used to retrieve finer-grained, relevant data, while a verification mechanism employs both the original question and the retrieved information to refine and adjust the answer. This process aligns evidence and results to ensure coherence [ 23 , 24 , 25 ] , thereby reducing errors induced by hallucinations in LLMs [ 26 , 27 ] , and ensuring the accuracy and reliability of intermediate results. By implementing a top-down planning and execution process with iterative verification and refinement, PAR RAG effectively mitigates error propagation and amplification caused by reasoning path deviations and process errors in existing RAG methods for multi-hop reasoning tasks. Extensive testing was carried out on the PAR RAG system across multiple multi-hop datasets. Experimental results demonstrate that, in comparison to existing state-of-the-art methods, PAR RAG achieves superior results in terms of EM and F1 scores. Moreover, experiments on single-hop datasets further confirm that PAR RAG exhibits strong adaptability, making it applicable beyond multi-hop question-answering tasks. Our contributions can be summarized as follows: • We propose a trustworthy, plan-driven RAG framework, PAR RAG. This framework begins with the formulation of a top-down plan and, during plan execution, verifies and refines the results of each step using a multi-granularity verification mechanism. It offers an interpretable, incremental reasoning paradigm for solving multi-hop question-answering tasks, effectively mitigating error propagation through comprehensive reasoning path planning and multi-granularity verification. • Through extensive experimentation across multiple multi-hop datasets, we demonstrate that the PAR RAG method significantly improves the accuracy of multi-hop question answering, surpassing current state-of-the-art methods. These results highlight the considerable potential of the integrated planning and verification

Read the complete text