Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models

Keshu Wu, Chenchen Kuai, Zihao Li, Jiwan Jiang, Shiyu Shen, Shian Wang, Chan-Wei Hu, Zhengzhong Tu, Yang Zhou · 2026

Read Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models by Keshu Wu, Chenchen Kuai, Zihao Li, Jiwan Jiang, Shiyu Shen, Shian Wang, Chan-Wei Hu, Zhengzhong Tu, Yang Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12185.txt.

Introduction

Wu and eight colleagues note most retrieval systems treat evidence as an unordered set, though real reasoning often turns on the order in which steps unfold. Their OKH-RAG stores knowledge as higher-order links in a hypergraph with precedence structure and recasts retrieval as finding paths over hyperedges. A learned transition model infers precedence from data alone. On order-sensitive tasks it beats baselines that ignore order, with ablations tracing the gains to order.

Catalog record

CREATORKeshu Wu, Chenchen Kuai, Zihao Li, Jiwan Jiang, Shiyu Shen, Shian Wang, Chan-Wei Hu, Zhengzhong Tu, Yang Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50891
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12185

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models Keshu Wu, Chenchen Kuai, Zihao Li, Jiwan Jiang, Shiyu Shen, Shian Wang, Chan-Wei Hu, Zhengzhong Tu, Yang Zhou Abstract Retrieval-augmented generation (RAG) enhances large language models by grounding outputs in retrieved knowledge. However, existing RAG methods including graph- and hypergraph-based approaches treat retrieved evidence as an unordered set, implicitly assuming permutation invariance. This assumption is misaligned with many real-world reasoning tasks, where outcomes depend not only on which interactions occur, but also on the order in which they unfold. We propose Order-Aware Knowledge Hypergraph RAG (OKH-RAG) , which treats order as a first-class structural property. OKH-RAG represents knowledge as higher-order interactions within a hypergraph augmented with precedence structure, and reformulates retrieval as sequence inference over hyperedges. Instead of selecting independent facts, it recovers coherent interaction trajectories that reflect underlying reasoning processes. A learned transition model infers precedence directly from data without requiring explicit temporal supervision. We evaluate OKH-RAG on order-sensitive question answering and explanation tasks, including tropical cyclone and port operation scenarios. OKH-RAG consistently outperforms permutation-invariant baselines, and ablations show that these gains arise specifically from modeling interaction order. These results highlight a key limitation of set-based retrieval: effective reasoning requires not only retrieving relevant evidence, but organizing it into structured sequences. K eywords Knowledge Graph \cdot Retrieval-Augmented Generation \cdot Hypergraph Representation \cdot Graph-based RAG 1 Introduction Large language models (LLMs) have demonstrated impressive capabilities in reasoning and generation, yet their performance on knowledge-intensive tasks remains constrained by the lack of explicit mechanisms for structured inference ( 55 ; 3 ; 36 ; 48 ) . Retrieval-augmented generation (RAG) mitigates this limitation by conditioning generation on externally retrieved knowledge, thereby separating knowledge access from language modeling ( 27 ; 13 ; 52 ; 24 ) . While early RAG approaches rely on unstructured text retrieval, recent work has shown that incorporating structured knowledge representations can significantly improve reasoning fidelity and factual consistency ( 57 ; 8 ) . Graph-based RAG methods represent an important step in this direction by organizing knowledge as a graph of entities and relations ( 6 ; 17 ; 47 ) . However, conventional graphs encode only binary relationships, restricting their ability to model interactions that inherently involve more than two entities. Many phenomena in real-world systems exhibit such higher-order dependencies, where outcomes emerge from the joint configuration of multiple factors rather than pairwise interactions alone. Hypergraphs naturally generalize graphs by allowing edges to connect arbitrary-sized sets of nodes, and therefore provide a principled representation for capturing higher-order graph dynamics ( 9 ; 2 ; 50 ; 49 ; 23 ) . Recent work on hypergraph-based RAG demonstrates that representing knowledge with n-ary relations can reduce information loss and improve retrieval effectiveness compared to binary graph formulations ( 10 ; 29 ; 43 ; 18 ) . Despite this advance, existing hypergraph-based RAG methods remain fundamentally static : hyperedges encode timeless relational facts, and retrieval is performed without accounting for how interactions evolve over time. This static abstraction limits the ability of retrieval mechanisms to support reasoning about processes, causality, and delayed effects. Figure 1: Knowledge graphs fragment multi-entity interactions into pairwise edges; knowledge hypergraphs preserve higher-order relations but retrieve them as unordered sets; our order-aware knowledge hypergraph augments hyperedges with learned precedence, enabling retrieval as ordered trajectories that reflect how interactions unfold. Though many interaction-aware RAG methods have been proposed, sequential structure is fundamental to knowledge-intensive reasoning, where system-level outcomes typically emerge from ordered interactions rather than simple co-occurrences. Whether representing a chain of subsystem failures or the propagation of causal dependencies, modeling these phenomena as static relations obscures the inherent ordering, persistence, and dynamics of the system, preventing retrieval from capturing underlying dynamics. For example, in a tropical cyclone scenario, port disruption may depend jointly on forecast storm intensity, local infrastructure exposure, evolving port conditions, and operational response. This requires a hypergraph representation, since the relevant dependency is higher-order rather than pairwise. It also requires order-awareness, since the reasoning depends on how these factors evolve over time, from forecast updates to warning escalation, port restrictions, and ultimately operational disruption ( 25 ; 26 ) . In this work, we argue that order is a fundamental dimension of knowledge structure in retrieval-augmented generation. We introduce Order-Aware Knowledge Hypergraph RAG (OKH-RAG) , as illustrated in Figure 1 , a framework that models knowledge as higher-order interactions with explicit precedence and reformulates retrieval as trajectory inference . Instead of retrieving a set of independent facts, OKH-RAG retrieves ordered hyperedge trajectories that capture how interactions evolve and connect to form coherent reasoning chains. This shifts retrieval from selecting relevant evidence to recovering structured sequences that reflect underlying processes. Our formulation unifies two complementary advances: hypergraph representations for modeling higher-order dependencies, and order-aware inference for capturing interaction dynamics. By integrating these into a single framework, OKH-RAG enables LLMs to reason over processes rather than static snapshots. Our contributions are summarized as follows: • We introduce an order-aware knowledge hypergraph representation that captures higher-order interactions together with their precedence structure. • We reformulate retrieval as trajectory inference over hyperedges, moving from permutation-invariant set retrieval to order-aware sequence retrieval. • We integrate order-aware hypergraph retrieval into the RAG framework, enabling generation grounded in structured, temporally coherent evidence. By integrating higher-order structure with interaction order, OKH-RAG establishes a retrieval framework where the sequence of evidence, not merely its content, is explicitly modeled, optimized, and empirically validated as a key driver of generation quality. 2 Related Work Retrieval-Augmented Generation. Retrieval-augmented generation (RAG) enhances parametric language models by conditioning generation on externally retrieved knowledge, mitigating issues such as hallucination, outdated information, and lack of traceable reasoning ( 28 ; 15 ) . By introducing a retriever that accesses a non-parametric memory, RAG enables models to incorporate domain-specific and up-to-date information at inference time. This framework has been widely adopted in knowledge-intensive applications, particularly in domain-specific settings. Representative applications include biomedical question answering ( 44 ) , legal document analysis ( 38 ) , and enterprise knowledge retrieval ( 33 ) . Early RAG systems primarily rely on retrieving unstructured text passages using dense retrieval methods such as Dense Passage Retrieval (DPR) ( 21 ; 20 ) . Subsequent surveys and benchmarks ( 12 ; 35 ) identify bottlenecks in both retrieval quality and scalability, particularly with the emergence of large language models (LLMs). Recent work has explored several directions to address these challenges, including knowledge-grounded dialogue systems ( 45 ) , adaptive retrieval strategies ( 1 ; 54 ; 51 ) , and multi-hop reasoning frameworks ( 7 ; 42 ) . These developments collectively motivate the incorporation of structured representations into the retrieval process. Graph and Hypergraph-Based RAG. Graph-based RAG methods ( 37 ; 19 ; 58 ) , which organize knowledge as entities and binary relations, represent one of the most effective extensions beyond unstructured retrieval in RAG. These approaches enable graph traversal ( 31 ; 16 ) , path-based reasoning ( 46 ; 4 ) , and community-level summarization ( 6 ) , thereby improving evidence aggregation and interpretability compared to purely similarity-based retrieval. However, they remain fundamentally limited by binary representations that decompose higher-order dependencies into pairwise edges, potentially leading

Read the complete text