ClaimPKG: Enhancing Claim Verification via Pseudo-Subgraph Generation with Lightweight Specialized LLM

Hoang Pham, Thanh-Do Nguyen, Khac-Hoai Nam Bui · 2025

Read ClaimPKG: Enhancing Claim Verification via Pseudo-Subgraph Generation with Lightweight Specialized LLM by Hoang Pham, Thanh-Do Nguyen, Khac-Hoai Nam Bui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22552.txt.

Introduction

Hoang Pham, Thanh-Do Nguyen, and Khac-Hoai Nam Bui strengthen automated fact checking with knowledge graphs in this 2025 paper. Checking a claim needs reasoning over facts, and graphs store facts as clean linked structures. Language models reason well but stumble over graph data without help.

Their framework, ClaimPKG, first drafts a small pseudo-subgraph tailored to the claim. A lightweight, specialized model does that drafting, keeping costs low. The main model then reasons over the tidy subgraph instead of raw text piles. The pipeline runs end to end. Cleaner evidence should mean fewer confident errors in verdicts on public claims.

Catalog record

CREATORHoang Pham, Thanh-Do Nguyen, Khac-Hoai Nam Bui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22552

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ClaimPKG: Enhancing Claim Verification via Pseudo-Subgraph Generation with Lightweight Specialized LLM Hoang Pham, Thanh-Do Nguyen, Khac-Hoai Nam Bui Abstract Integrating knowledge graphs (KGs) to enhance the reasoning capabilities of large language models (LLMs) is an emerging research challenge in claim verification. While KGs provide structured, semantically rich representations well-suited for reasoning, most existing verification methods rely on unstructured text corpora, limiting their ability to effectively leverage KGs. Additionally, despite possessing strong reasoning abilities, modern LLMs struggle with multi-step modular pipelines and reasoning over KGs without adaptation. To address these challenges, we propose ClaimPKG 1 1 1 https://github.com/HoangHoang1408/ClaimPKG , an end-to-end framework that seamlessly integrates LLM reasoning with structured knowledge from KGs. Specifically, the main idea of ClaimPKG is to employ a lightweight, specialized LLM to represent the input claim as pseudo-subgraphs, guiding a dedicated subgraph retrieval module to identify relevant KG subgraphs. These retrieved subgraphs are then processed by a general-purpose LLM to produce the final verdict and justification. Extensive experiments on the FactKG dataset demonstrate that ClaimPKG achieves state-of-the-art performance, outperforming strong baselines in this research field by 9%-12% accuracy points across multiple categories. Furthermore, ClaimPKG exhibits zero-shot generalizability to unstructured datasets such as HoVer and FEVEROUS, effectively combining structured knowledge from KGs with LLM reasoning across various LLM backbones. 1 Introduction Figure 1: Different claim verification paradigms: (a) Unstructured Text-based methods focusing on claim decomposition and sequential reasoning over text, (b) KG-based methods facing challenges in entity resolution and structured reasoning, and (c) ClaimPKG’s unified framework with specialized modules for pseudo-subgraph generation, retrieval, and general reasoning. In today’s rapidly evolving information landscape, distinguishing fact from misinformation is becoming more challenging, especially with the rise of AI-generated content. Robust claim verification systems, leveraging NLP methods to automatically assess the veracity of claims Glockner et al. (2022a) ; Glockner et al. (2022b) ; Thorne and Vlachos (2018) , are essential to ensure information reliability. Effective methods require not only accuracy but also transparency, necessitating strong reasoning to identify evidence and provide clear justifications Pan et al. (2023) . Most existing verification approaches focus on unstructured text corpora, using techniques like chain-of-thought (CoT) reasoning Wei et al. (2022) to break down claims for verification. Approaches like ProgramFC Pan et al. (2023) and FOLK Wang and Shu (2023) employ modular pipelines to verify claims against text-based knowledge bases (Figure 1 (a)). However, the inherent limitations of text representation pose challenges. Specifically, ambiguous entity references and complex multi-hop relationships make it difficult to perform rigorous verification against unstructured text. In contrast, Knowledge Graphs (KGs) provide structured relationships for effective reasoning Luo et al. (2024) ; Sun et al. (2024) , yet their use in claim verification remains limited. Existing KG-based approaches (Figure 1 (b)) Kim et al. (2023b) ; Zhou et al. (2019) ; Kim et al. (2023a) lack end-to-end solutions, often requiring pre-extracted entities via modules like entity or relation extraction. Meanwhile, despite excelling at general reasoning, LLMs struggle with KG-specific tasks like entity resolution and multi-hop reasoning Cao et al. (2021) ; Aly et al. (2021) , suggesting the need for a system combining LLM capabilities with KG-based inference. Overall, solving claim verification problems is hindered by following major limitations: (1) Entity Ambiguity: Systems must accurately disambiguate entities within claims to identify relevant evidence Aly et al. (2021) ; (2) Multihop Reasoning: Complex claims often require reasoning across multiple evidence from different sources Pan et al. (2023) ; Wang and Shu (2023) ; and (3) Limited integration of KGs and LLMs: Current approaches are underexploring the potential of combining the application of structured representation with strong inference capabilities of LLMs Kim et al. (2023a) . To address these challenges, we propose ClaimPKG (Claim Verification using Pseudo-Subgraph in Knowledge Graphs), a novel end-to-end framework that synergizes the adaptability and generalization strengths of LLMs with the structured and rigorous representation of KGs to enable robust and transparent claim verification. As specified in Figure 1 (c), ClaimPKG operates through three phases: (1) Pseudo-Subgraphs Generation : A KG-specialized lightweight LLM generates pseudo subgraphs as the representations of input claims under a Trie-based KG-Entity Constraint, ensuring the correctness of extracted entities; (2) Subgraphs Retrieval : A retrieval algorithm considers generated pseudo subgraphs as queries to identify actual relevant KG subgraphs as evidence; and (3) General Reasoning : A general-purpose LLM reasons over the retrieved KG subgraphs to produce the verdict and human-readable justifications. Through extensive experiments on the FactKG dataset, ClaimPKG achieves state-of-the-art performance, demonstrating its effectiveness over various claim types with a small number of training samples. Furthermore, its zero-shot generalizability to unstructured datasets (HoVer, FEVEROUS) highlights its robustness. Our contributions can be summarized as follows: (1) We introduce ClaimPKG, a holistic framework that integrates LLMs and KGs for accurate and interpretable claim verification, handling various types of claims in a unified manner; (2) We develop a lightweight specialized LLM with its according decoding algorithm for pseudo-subgraph generation and pair it with general-purpose LLMs to achieve robust reasoning; and (3) We validate the effectiveness of ClaimPKG through extensive experiments, achieving state-of-the-art performance on structure-based datasets and generalizing to unstructure-based datasets. 2 Related Work Claim Verification Approaches. Claim verification systems utilize knowledge bases that can be categorized into unstructured and structured formats. In the unstructured domain, text-based verification methods predominate, with systems designed to verify claims against textual evidence, as demonstrated in the FEVER dataset Thorne et al. (2018) . Recent advances have focused on handling specialized verification scenarios, including ambiguous question-answer pairs Park et al. (2022) , detecting factual changes Schuster et al. (2021) , and processing multiple documents concurrently Jiang et al. (2020) . For structured verification, research has primarily focused on tables and graphs, with early work developing specialized architectures: graph neural networks for knowledge graph processing Zhou et al. (2020) , table-specific transformers Herzig et al. (2020) , and tree-structured decoders for hierarchical data Wang et al. (2020) . Claim Verification over Knowledge Graphs (KGs). The emergence of Large Language Models (LLMs) has simplified direct reasoning over textual corpora for claim verification, as demonstrated by ProgramFC Pan et al. (2023) and FOLK Wang and Shu (2023) . However, structured data sources like tables and graphs can provide more grounded and robust verification results Kim et al. (2023b) . Knowledge graphs are particularly advantageous as they enable explicit representation of reasoning processes through logical rules over nodes and edges. FactKG Kim et al. (2023b) established a foundation in this direction by introducing a comprehensive dataset for evaluating modern verification methods. KG-GPT Kim et al. (2023a) followed this work by demonstrating performance gains through a pipeline that performs sentence decomposition, subgraph retrieval, and logical inference. Additionally, while not directly addressing claim verification, StructGPT Jiang et al. (2023) and RoG Luo et al. (2024) achieved promising results in related tasks (e.g., Knowledge Base Question Answering) by collecting relevant evidence, such as subgraphs in KGs, then leveraging LLMs for complex reasoning in particular scenarios. 3 Preliminary Knowledge Graph: Knowledge Graph (KG) \mathcal{G} represents facts as triplets of format t=(e,r,e^{\prime}) , where entities e,e^{\prime}\in\mathcal{E} are connected by a relation r\in\mathcal{R} ; r can also be referred as r(e,e^{\prime}) . Claim Verification: Given a claim c , a verification model \mathcal{F} determines its veracity as Supported or Refuted

Read the complete text