InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning

Zheng Wang, Shu Xian Teo, Jun Jie Chew, Wei Shi · 2025

Read InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning by Zheng Wang, Shu Xian Teo, Jun Jie Chew, Wei Shi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13032.txt.

Introduction

Zheng Wang, Shu Xian Teo, Jun Jie Chew, and Wei Shi present InstructRAG, a planner that lets language model agents look up how similar tasks were solved before. Model agents plan step by step, but their built-in knowledge of complex tasks is thin. Retrieval can ground them, if the retrieved know-how can grow and transfer.

Their system stores past successful action sequences in an instruction graph. A reinforcement learning agent expands that graph's coverage, while a second mechanism carries lessons across tasks. The design sits inside a multi-agent meta-learning framework. Planning improves when yesterday's solutions are indexed like a manual. The authors name enlargability and transferability as the twin tests.

Catalog record

CREATORZheng Wang, Shu Xian Teo, Jun Jie Chew, Wei Shi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-13032

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning Zheng Wang, Shu Xian Teo, Jun Jie Chew, Wei Shi Abstract . Recent advancements in large language models (LLMs) have enabled their use as agents for planning complex tasks. Existing methods typically rely on a thought-action-observation (TAO) process to enhance LLM performance, but these approaches are often constrained by the LLMs’ limited knowledge of complex tasks. Retrieval-augmented generation (RAG) offers new opportunities by leveraging external databases to ground generation in retrieved information. In this paper, we identify two key challenges (enlargability and transferability) in applying RAG to task planning. We propose InstructRAG , a novel solution within a multi-agent meta-reinforcement learning framework, to address these challenges. InstructRAG includes a graph to organize past instruction paths (sequences of correct actions), an RL-Agent with R einforcement L earning to expand graph coverage for enlargability, and an ML-Agent with M eta- L earning to improve task generalization for transferability. The two agents are trained end-to-end to optimize overall planning performance. Our experiments on four widely used task planning datasets demonstrate that InstructRAG significantly enhances performance and adapts efficiently to new tasks, achieving up to a 19.2% improvement over the best existing approach. Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730009 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Information retrieval † † cc-license: by 1. INTRODUCTION With the significant advancement of large language models (LLMs), a recent trend has emerged in employing LLMs as intelligent agents to tackle diverse real-world planning tasks. These tasks include multi-hop reasoning ( Yang et al., 2018 ) , embodied tasks ( Shridhar et al., 2020b ; Wang et al., 2024d ; Shridhar et al., 2020a ) , web shopping ( Yao et al., 2022a ) , and scientific reasoning ( Wang et al., 2022a ) , etc. Many recent solutions to the planning problem, such as ReAct ( Yao et al., 2022b ) , KnowAgent ( Zhu et al., 2024 ) , WKM ( Qiao et al., 2024 ) , Reflexion ( Shinn et al., 2024 ) , FireAct ( Chen et al., 2023 ) , NAT ( Wang et al., 2024a ) , and ETO ( Song et al., 2024 ) , follow a thought-action-observation (TAO) process. In the thought phase, the LLM leverages its reasoning ability to create a plan by breaking down a task into a series of subtasks. In the action phase, the LLM determines the specific actions required, such as selecting which tool to use. In the observation phase, it captures the results of executing the action and provides feedback from the external environment to the LLM, facilitating the planning of subsequent TAO steps. Within this process, the thought and action are generated by the LLM, while the observation is implemented by the environment. Existing solutions adopt diverse strategies, including prompting ( Yao et al., 2022b ; Shinn et al., 2024 ) or fine-tuning ( Zhu et al., 2024 ; Qiao et al., 2024 ; Chen et al., 2023 ; Wang et al., 2024a ; Song et al., 2024 ) , to improve LLM-generated thoughts and actions for more effective planning. In particular, KnowAgent ( Zhu et al., 2024 ) integrates pre-defined rules into prompts to ensure that generated thoughts exhibit logical action transitions. For example, it prevents looking up an entity without first performing a search operation on the topic, as seen in HotPotQA ( Yang et al., 2018 ) . Reflexion ( Shinn et al., 2024 ) incorporates self-reflection summaries into the TAO process to guide subsequent trials. WKM ( Qiao et al., 2024 ) trains a world knowledge model to generate thoughts based on knowledge acquired from human task-solving experiences. While these existing methods aim to enhance LLM planning, they are often constrained by the inherent limitations of the LLMs themselves, such as their limited knowledge on complex tasks. The rapid development of retrieval-augmented generation (RAG) provides new opportunities to address these limitations by leveraging external databases. By anchoring LLM generation in retrieved information, RAG improves performance through the integration of relevant data during the planning process. In this context, we recognize that task-specific nature of information retrieval plays a crucial role for effective planning generation. For example, consider the question “Were Scott Derrickson and Ed Wood of the same nationality?” from HotPotQA, as shown in Figure 1 (a). A potential retrieved plan for this question involves a sequence of actions (referred to as instruction paths in this paper): first, use Google Search to find information about Scott Derrickson (denoted by Search[Scott Derrickson] ), then look up a sentence containing the keyword “nationality” ( Lookup[nationality] ), followed by Search[Ed Wood] and Lookup[nationality] . These instructions are specific to the question at hand and may vary depending on the topics or entities involved. A similar phenomenon can also be observed in ALFWorld embodied tasks, where instructions might include Goto[shelf 6] , then Take[vase 2 from shelf 6] . In this paper, we discuss the task-specific nature in two aspects, with the goal of bridging the gap between task-specific questions and instructions derived from past experiences stored in a database through RAG. (1) Enlargeability : This refers to a task where the question falls within the scope of those covered by the external database. Specifically, we pre-store successful instruction paths in the database, with each path tailored to a specific task. To address questions related to these tasks, we explore a paradigm for combining instructions to expand the database’s coverage. As illustrated in Figure 1 (a), there are two successful instruction paths, P_{1}^{1} and P_{2}^{1} , for solving questions Q_{1}^{1} and Q_{2}^{1} , respectively. These paths consist of five instructions: searching for (a) Scott Derrickson, (b) Ed Wood, and (c) Christopher Nolan, and looking up entities related to (d) nationality and (e) birthplace. By combining these instructions in sequences such as (a)\rightarrow(e)\rightarrow(c)\rightarrow(e) , we can generate a new instruction path for solving a novel question (i.e., \hat{Q} ) that was not covered by the original paths (but shares the same task of querying a location). The enlargeability is proposed to enhance the database’s ability to address a wider range of questions. (2) Transferability : This refers to a task where the question is outside the scope of tasks covered by the external database. We note that LLM-based task planning is provided as a capability to support a wide range of tasks in practice. Transferability is essential for bridging the gaps between different tasks (i.e., those in the pre-built database and the questions at hand) within the RAG system. To achieve this, it is necessary to expand the database to incorporate new instructions required for different tasks, such as updating it with instructions relevant to the new tasks based on a development set. Additionally, certain trainable modules associated with the RAG can be rapidly adapted to accommodate the new tasks. New Solution. Although recent research efforts ( Lee et al., 2024 ; Kagaya et al., 2024 ; Shi et al., 2024 ) have attempted to apply RAG

Read the complete text