Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs

Hussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis, Essam Mansour · 2026

Read Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs by Hussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis, Essam Mansour (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13979.txt.

Introduction

Abdallah and colleagues present GLOW, a hybrid system for open-world question answering over incomplete knowledge graphs. A graph neural network proposes top candidate answers from graph structure, and a language model reasons over those candidates plus serialized facts in a structured prompt, with no retrieval or fine-tuning. On a new 1,000-question benchmark and standard sets, improvements reach up to 53.3 percent, averaging 38 percent.

Catalog record

CREATORHussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis, Essam Mansour
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52190
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13979

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs Hussein Abdallah, Ibrahim Abdelaziz, Panos Kalnis, Essam Mansour Abstract Open-world Question Answering (OW-QA) over knowledge graphs (KGs) aims to answer questions over incomplete or evolving KGs. Traditional KGQA assumes a closed world where answers must exist in the KG, limiting real-world applicability. In contrast, open-world QA requires inferring missing knowledge based on graph structure and context. Large language models (LLMs) excel at language understanding but lack structured reasoning. Graph neural networks (GNNs) model graph topology but struggle with semantic interpretation. Existing systems integrate LLMs with GNNs or graph retrievers. Some support open-world QA but rely on structural embeddings without semantic grounding. Most assume observed paths or complete graphs, making them unreliable under missing links or multi-hop reasoning. We present GLOW, a hybrid system that combines a pre-trained GNN and an LLM for open-world KGQA. The GNN predicts top- k k candidate answers from the graph structure. These, along with relevant KG facts, are serialized into a structured prompt (e.g., triples and candidates) to guide the LLM’s reasoning. This enables joint reasoning over symbolic and semantic signals, without relying on retrieval or fine-tuning. To evaluate generalization, we introduce GLOW-Bench , a 1,000-question benchmark over incomplete KGs across diverse domains. GLOW outperforms existing LLM–GNN systems on standard benchmarks and GLOW-Bench , achieving up to 53.3% and an average 38% improvement. GitHub code and data are available here . 1 Introduction Open-World Question Answering (OW-QA) over knowledge graphs (KGs) aims to answer questions when relevant facts are missing or the KG is incomplete. This challenge arises in real-world domains like biomedicine, scientific research, and finance, where knowledge is often evolving, implicit, or incomplete 13 ; 30 . For example, systems may need to infer undocumented drug interactions, latent collaborations, or causal relationships. In such cases, answers are not explicitly stored in the KG and must be predicted based on graph structure, entity semantics, and cues in the question. Traditional KGQA methods assume a closed-world setting, where all facts are known and retrievable. This limits their use in dynamic or incomplete environments. Open-world QA instead requires reasoning over both observed and missing information. Unlike standard retrieval or link prediction, it must integrate symbolic language with structural graph signals. This motivates hybrid approaches that combine the semantic flexibility of Large Language Models (LLMs) with the relational reasoning of Graph Neural Networks (GNNs). Most existing KGQA systems rely on structured or dense retrieval to find explicit paths between question and answer entities. Methods like G-Retriever 10 , GNN-RAG 16 , RoG 14 , and ToG 24 retrieve graph fragments using semantic similarity. GCR 15 improves scalability but still assumes a complete graph. These methods often fail under open-world conditions, where answer paths are missing or poorly aligned with the question. This results in noisy retrieval and low accuracy. OW-QA instead requires predictive reasoning to infer plausible answers beyond observed facts. AskGNN 11 addresses this by using GNNs to guide LLM inference, but relies heavily on structural embeddings and lacks semantic grounding. This limits its ability to handle complex or multi-hop questions. Table 1: Average accuracy (%) on OW-QA benchmarks by reasoning depth. GLOW-GN leads on both 1- and 2-hop questions, while GCR drops sharply under OWA. All use Qwen3-8B; scores for existing datasets are averaged over arxiv2023, ogbn-arxiv, and ogbn-products. Method Existing Datasets GLOW-Bench (ours) 1-Hop 1-Hop 2-Hop LLM {}_{\text{Only}} 25.7 30.0 18.9 GCR 7.8 34.1 15.3 GoG 44.6 48.3 29.5 AskGNN 53.7 79.4 34.3 GLOW-GN 71.7 83.3 42.4 To address this challenge, we propose GLOW, a novel hybrid approach that synergistically combines the strengths of both LLMs and GNNs. Our method uses a pre-trained GNN to predict the top-k possible answers based on the graph topology. These answers, along with a serialized subgraph of KG facts relevant to the question, are then incorporated into the LLM prompt using a controlled and structured format (e.g., relational triples, top-ranked candidate answers), enabling the LLM to reason jointly over language and structure. Rather than relying on retrieval or pretraining alone, our model dynamically bridges gaps in the KG by augmenting the LLM with on-the-fly, GNN-driven prompts. We evaluate GLOW on the AskGNN 11 open-world QA benchmark, which is based on ogbn-arxiv, ogbn-products, and arxiv2023 datasets. AskGNN used these datasets to reflect realistic KG incompleteness and avoid the closed-world assumption in existing benchmarks, such as WebQSP or CWQ 15 . To test generalization, we introduce GLOW-Bench , a new benchmark of 1,000 natural language questions across diverse domains. Each question requires reasoning over incomplete KGs, with the correct answer deliberately removed. Unlike AskGNN’s dataset, which covers only single-hop questions in less diverse domains, GLOW-Bench includes 1- and 2-hop questions across multiple KGs. We compare our method against state-of-the-art baselines, including AskGNN 11 , GCR 15 , a KG-grounded QA pipeline, GoG 33 ,KGQA over incomplete KG, and LLMs such as GPT-4o-mini 17 and Qwen3-8B 34 . As shown in Table 1 , LLMs perform poorly on deeper reasoning tasks, often failing to retrieve or organize relevant knowledge. AskGNN performs better but is limited by its structural focus and lack of semantic flexibility. GCR struggles under incomplete KGs, often hallucinating answer paths or returning incorrect answers. GoG hallucinates answer path generation as it overlooks the underlying KG schema and structural constraints, resulting in semantically inconsistent or invalid paths. Our benchmark highlights these issues and shows the need for models that can reason jointly over symbolic and structural signals. In summary, our contributions are: • We propose GLOW, a novel OW-KGQA system that combines a GNN and an LLM to jointly reason over structured and unstructured knowledge. • We present GLOW-Bench , a 1,000-question benchmark for open-world KGQA with multi-hop reasoning over incomplete, cross-domain KGs. • We demonstrate that GLOW outperforms state-of-the-art LLM–GNN QA and KGQA systems across standard benchmarks and GLOW-Bench , with up to 53.3% and an average of 38% improvement. 2 Related Work Our work connects to research in KG completion, LLM-based QA, GNN–LLM hybrid models, and open-world KGQA. Knowledge Graph Completion. Embedding-based methods (TransE, RotatE, ComplEx) 19 and recent semantic models (StructurE, HopfE, DensE) 7 infer missing links via latent representations. Rule-based systems (AnyBURL 20 , SAFRAN 18 ) generalize KG patterns with logical rules. While useful for KG augmentation, these approaches operate independently of QA. CBR-iKB applies case-based reasoning with KGEs but is computationally expensive and limited to transductive settings. In contrast, GLOW integrates KG completion into QA through GNN-guided prompting. LLMs for KGQA. Fine-tuning LLMs on KG triples can improve domain adaptation 29 ; 23 ; 12 , but requires extensive training and risks catastrophic forgetting 32 ; 40 . Our approach avoids fine-tuning by injecting both graph-structured data and GNN outputs into LLM prompts, enabling semantic and structural reasoning without modifying LLM weights. GNN–LLM Hybrid Models. GNN-based QA systems 35 ; 43 ; 1 support multi-hop reasoning, but often ignore language semantics. Retrieval-augmented methods, such as GNN-RAG 16 , G-Retriever 10 , STaRK 31 , and RoG 14 , embed graph fragments for LLMs to reason over. However, they assume complete graphs and rely on retrieving full answer paths, which breaks under missing links 42 . Our method differs by using a GNN to

Read the complete text