Graph-Anchored Knowledge Indexing for Retrieval-Augmented Generation

Zhenghao Liu, Mingyan Wu, Xinze Li, Yukun Yan, Shuo Wang, Cheng Yang, Minghe Yu, Zheni Zeng, Maosong Sun · 2026

Read Graph-Anchored Knowledge Indexing for Retrieval-Augmented Generation by Zhenghao Liu, Mingyan Wu, Xinze Li, Yukun Yan, Shuo Wang, Cheng Yang, Minghe Yu, Zheni Zeng, Maosong Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16462.txt.

Introduction

Zhenghao Liu and coauthors build a graph index to ground large models in facts. Their 2026 preprint deals with search backed text builds, called RAG. Such builds cut false claims by use of known facts. Proof is oft spread in bits through noisy texts. Bits that sit far apart are hard to join in one build.

A graph index ties proof bits by their links ere search. Tied bits can then be pulled as one firm set. Firm sets should cut false claims in the end text. They should also make the claim source clear to check. The work seeks RAG builds with less noise.

Catalog record

CREATORZhenghao Liu, Mingyan Wu, Xinze Li, Yukun Yan, Shuo Wang, Cheng Yang, Minghe Yu, Zheni Zeng, Maosong Sun
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16462

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Graph-Anchored Knowledge Indexing for Retrieval-Augmented Generation Zhenghao Liu, Mingyan Wu, Xinze Li, Yukun Yan, Shuo Wang, Cheng Yang, Minghe Yu, Zheni Zeng, Maosong Sun Abstract Retrieval-Augmented Generation (RAG) has emerged as a dominant paradigm for mitigating hallucinations in Large Language Models (LLMs) by incorporating external knowledge. Nevertheless, effectively integrating and interpreting key evidence scattered across noisy documents remains a critical challenge for existing RAG systems. In this paper, we propose GraphAnchor, a novel Graph-Anchored Knowledge Indexing approach that reconceptualizes graph structures from static knowledge representations into active, evolving knowledge indices. GraphAnchor incrementally updates a graph during iterative retrieval to anchor salient entities and relations, yielding a structured index that guides the LLM in evaluating knowledge sufficiency and formulating subsequent subqueries. The final answer is generated by jointly leveraging all retrieved documents and the final evolved graph. Experiments on four multi-hop question answering benchmarks demonstrate the effectiveness of GraphAnchor, and reveal that GraphAnchor modulates the LLM’s attention to more effectively associate key information distributed in retrieved documents. All code and data are available at https://github.com/NEUIR/GraphAnchor . 1 Introduction Retrieval-Augmented Generation (RAG) 13 has emerged as a promising paradigm for mitigating hallucinations in Large Language Models (LLMs) 2 . For a given query, typical RAG models retrieve relevant documents, which are incorporated as contextual evidence to guide LLMs in generating accurate answers 23 ; 25 ; 18 . Although RAG models have demonstrated effectiveness across many knowledge-intensive tasks 1 , the retrieved contexts often contain substantial noise 33 , and supporting evidence is frequently distributed across multiple documents 19 . The dispersed nature of key information across retrieved documents poses a challenge for RAG models in effectively integrating and leveraging the retrieved knowledge 12 . Figure 1: Illustration of GraphAnchor. The graph is progressively updated for knowledge indexing. To address this issue, numerous studies have explored structuring retrieved knowledge to better aggregate supporting evidence distributed across multiple passages 19 ; 17 . These approaches typically investigate the effectiveness of different representation formats for extracting and representing salient information from retrieved evidence 32 ; 19 ; 38 . Although these methods enhance the coherence of knowledge representations and help reduce noise in the retrieved contexts, they primarily leverage these structures to filter out irrelevant information and assist LLMs in better utilizing the retrieved knowledge. As a result, they often fail to fully exploit crucial evidence that cannot be captured in these formats and overlook the role of structured data in facilitating deeper interpretation and reasoning over the retrieved knowledge 21 . As shown in Figure 1 , graphs provide a well-defined structural representation for explicitly modeling entities and their relations. Such graphs can be incrementally updated by introducing newly identified entities, for example, adding “Carbon County” and linking it with “Red Lodge” through the “county seat” relation. This graph updating process progressively interprets the retrieved documents, which motivates us to treat the graph as a knowledge index for RAG systems. This paper presents GraphAnchor, a Graph-Anchored Knowledge Indexing approach that iteratively updates a graph to enhance the interpretation of retrieved knowledge during the iterative retrieval process. Unlike previous methods 32 ; 19 ; 38 , GraphAnchor redefines the role of the graph, shifting it from a mere knowledge representation to an active indexing tool. At each retrieval step, GraphAnchor first retrieves documents based on the current query and updates the graph based on these retrieved documents. Then GraphAnchor assesses whether the current knowledge, informed by the graph, is sufficient to answer the query. If the knowledge is lacking, a new query is generated. Once the available knowledge is sufficient, the final response is generated according to the documents retrieved at each step with the final version of the graph. Our experiments on four multi-hop QA datasets with different backbone LLMs demonstrate the effectiveness of GraphAnchor. Rather than refining knowledge for question answering, GraphAnchor focuses on knowledge anchoring and improves both the retrieval and QA modules within an iterative RAG framework. Specifically, GraphAnchor facilitates the retrieval of more relevant and diverse documents across multiple retrieval steps, and enables LLMs to better interpret and utilize the retrieved knowledge. Further analysis shows that, as the retrieval process progresses, the graph anchors an increasing number of related entities and relations into the index, and subsequently modulates the attention mechanism of LLMs to aggregate distributed evidence from the retrieved documents. 2 Related Work Retrieval-Augmented Generation (RAG) models typically retrieve relevant passages and incorporate them as contextual inputs, thereby helping Large Language Models (LLMs) to generate accurate and grounded responses 23 ; 13 ; 25 ; 26 . However, conventional RAG approaches often suffer from insufficient knowledge acquisition, particularly in multi-hop question answering scenarios where reasoning requires the integration of evidence across multiple passages. To mitigate this limitation, several studies have proposed iterative retrieval strategies that progressively seek and accumulate relevant knowledge to support answer generation ( 28 ; 9 ; 30 ) . In addition, some methods perform deep retrieval by interacting with external retrieval tools during the reasoning process, enabling more flexible and adaptive information seeking 9 ; 16 . Nevertheless, as the number of retrieval steps increases, these RAG systems inevitably introduce more noisy or irrelevant information, which can substantially degrade the quality of the generated outputs 3 . To improve knowledge utilization in LLMs, prior work has explored reranking retrieved passages 1 or summarizing salient information to filter out noise 29 ; 33 . However, these approaches typically process retrieved passages in isolation, with limited modeling of relationships among different documents. As a result, cross-document evidence may be overlooked or insufficiently captured, particularly in settings that require aggregating complementary information across multiple sources 19 . To better aggregate diverse knowledge from multiple retrieved passages, a growing body of work leverages structured representations, including schemas, tables, and graphs, to explicitly extract key information scattered across these retrieved documents 5 ; 17 ; 19 ; 6 ; 38 . 20 further explores multiple reasoning paths over the constructed knowledge graph, identifying conflicting paths as input context, thereby alleviating knowledge conflicts between internal and external information. Instead of replacing retrieved passages with structured representations, some works 14 ; 31 construct graphs to identify related information across multiple passages or chunks. Specifically, they extract key entities and facts from different chunks or passages and connect them via a graph structure. Such graphs enable LLMs to better capture long-range dependencies and multi-hop relationships within long texts or across documents, thereby facilitating more effective localization of relevant evidence. In contrast, GraphAnchor maintains an evolved graph through iterative retrieval, which anchors key concepts for knowledge interpretation and subquery generation. Figure 2: Illustration of GraphAnchor. 3 Methodology As illustrated in Figure 2 , this section presents the proposed GraphAnchor, which introduces an evolving graph to anchor knowledge within an iterative RAG framework. We first describe the overall architecture of GraphAnchor, highlighting how the evolving graph is integrated to support reasoning, query generation, and question answering (Sec. 3.1 ). We then elaborate on the construction and dynamic updating of the graph, which functions as a context-aware knowledge index to enhance the knowledge association and comprehension capabilities of LLMs (Sec. 3.2 ). 3.1 Graph-based Knowledge Anchoring for Iterative RAG Given an initial query q_{0} that

Read the complete text