A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages

Ivan Vykopal, Martin Hyben, Robert Moro, Michal Gregor, Jakub Simko · 2025

Read A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages by Ivan Vykopal, Martin Hyben, Robert Moro, Michal Gregor, Jakub Simko (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20668.txt.

Introduction

Ivan Vykopal and four co-authors built a system to fetch already-checked claims for fact-checkers in a 2025 paper. False claims online spread fast, and checkers waste time redoing claims that were checked before. Their system works across social platforms and in many languages.

Given a new claim, the system pulls past fact-checks that might match, then uses large language models to drop the poor matches. It also writes short summaries that say why a past check fits. Tests by machine scores and by people using the tool showed the filter cut much dead weight. That lets checkers judge faster whether a claim is truly new. The authors frame it as a workload cut, not a verdict machine.

Catalog record

CREATORIvan Vykopal, Martin Hyben, Robert Moro, Michal Gregor, Jakub Simko
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20668

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languages Ivan Vykopal, Martin Hyben, Robert Moro, Michal Gregor, Jakub Simko Abstract Online disinformation poses a global challenge, placing significant demands on fact-checkers who must verify claims efficiently to prevent the spread of false information. A major issue in this process is the redundant verification of already fact-checked claims, which increases workload and delays responses to newly emerging claims. This research introduces an approach that retrieves previously fact-checked claims, evaluates their relevance to a given input, and provides supplementary information to support fact-checkers. Our method employs large language models (LLMs) to filter irrelevant fact-checks and generate concise summaries and explanations, enabling fact-checkers to faster assess whether a claim has been verified before. In addition, we evaluate our approach through both automatic and human assessments, where humans interact with the developed tool to review its effectiveness. Our results demonstrate that LLMs are able to filter out many irrelevant fact-checks and, therefore, reduce effort and streamline the fact-checking process. 1 Introduction Figure 1: An example of a post with two fact-checked claims retrieved by the embedding model. The LLM selects the relevant claim, explains its choice, summarizes the fact-check article, and predicts the post’s veracity. The rise of social media has accelerated the spread of false information, posing significant societal, economic and public health risks Zubiaga et al. (2018) . This challenge is further compounded by the multilingual nature of false information, making fact-checking a complex and resource-intensive task for fact-checkers. Fact-checkers often struggle to verify claims across languages, particularly in low-resource settings where limited fact-checking support exists Hrckova et al. (2024) . To address this issue, it is crucial to develop multilingual fact-checking approaches that can assist fact-checkers to identify and verify misinformation efficiently. One of the key tasks in fact-checking is claim retrieval, also known as previously fact-checked claim retrieval Pikuliak et al. (2023) , where the goal is to identify fact-checks from a database that are the most similar to a given input. This task is crucial, as many claims are not entirely new but rather rephrased or repeated versions of previously debunked misinformation. Efficient retrieval enables fact-checkers to quickly detect repeated claims, reduce redundant efforts, and prioritize emerging or complex claims Hrckova et al. (2024) . However, retrieved results may include fact-checks that are only loosely related or irrelevant, increasing the workload. To mitigate this, LLMs can be leveraged to assess the relevance of retrieved fact-checks, thereby streamlining the review process Vykopal et al. (2025) . In this paper, we propose a novel pipeline for retrieving previously fact-checked claims and assisting fact-checkers in assessing their relevance to a given query. Our experiments cover more than 10 languages from diverse linguistic families and scripts, including low- and high-resource languages. We analyze the ability of language models to retrieve relevant fact-checks while incorporating summarization and explanation. An example is shown in Figure 1 . In addition, we evaluate LLMs’ performance to determine the veracity based on retrieved fact-checks and supporting information. 1 1 1 The data are available at Zenodo upon request for research purposes only : https://zenodo.org/records/15267292 . The source code is available at: https://github.com/kinit-sk/claim-retrieval . Our contributions are as follows: • We provide a novel AFP-Sum dataset consisting of around 19K fact-checking articles along with their summaries across 23 languages. Additionally, we created a subset of 2300 fact-checks in 23 languages along with the summaries in the original language and translated summaries in English. • We evaluated multiple text embedding models (TEMs) for retrieving previously fact-checked claims across 20 languages and the capabilities of TEMs for filtering fact-checks based on instructions in the natural language. • We proposed a novel pipeline for incorporating LLMs into the verification process by employing LLMs for identifying relevant previously fact-checked claims, providing fact-check summaries and predicting the veracity of a given claim based on the previously retrieved fact-checks. 2 Related Work Previously Fact-Checked Claim Retrieval. Previously fact-checked claim retrieval, also known as verified claim retrieval Barrón-Cedeño et al. (2020) or claim-matching Kazemi et al. (2021) , aims to reduce fact-checkers’ workload by retrieving similar, already verified claims. While most research focused on monolingual settings Shaar et al. (2020) ; Shaar et al. (2022) ; Hardalov et al. (2022) , multilingual retrieval remains underexplored Vykopal et al. (2024) . Recent work, such as Pikuliak et al. (2023) , introduces the MultiClaim dataset for multilingual claim retrieval, evaluating various TEMs for ranking fact-checked claims in monolingual and cross-lingual contexts. Recent advancements in LLMs present new opportunities for enhancing claim retrieval. Existing approaches primarily rely on two strategies. The first involves textual entailment , where models classify the entailment between an input claim and a fact-check into three categories Choi and Ferrara (2024a) ; Choi and Ferrara (2024b) . In contrast, the second strategy employs generative re-ranking to rank the previously fact-checked claims based on the conditional probabilities generated by LLMs, which are used to prioritize more relevant claims Shliselberg and Dori-Hacohen (2022) ; Neumann et al. (2023) . Fact-Checking Pipelines & Tools. With the growing importance of online fact-checking, numerous pipelines have been developed to combat misinformation. Many of these systems rely on retrieving the evidence based on a given claim and leveraging LLMs to asses veracity and provide justifications. However, most research has predominantly focused on English Hassan et al. (2017) ; Shu et al. (2019) ; Li et al. (2024) or Arabic Jaradat et al. (2018) ; Althabiti et al. (2024) ; Sheikh Ali et al. (2023) , with fewer studies dedicated to other languages. Several online tools have been developed to address false information. WeVerify 2 2 2 https://weverify.eu/ provides a suite of tools for identifying false information, including image analysis for detecting manipulated content. In addition, BRENDA Botnevik et al. (2020) assesses the credibility of claims, helping users evaluate online information. Furthermore, the models trained by Pikuliak et al. (2023) for retrieving previously fact-checked claims have been integrated into the Fact-Check Finder 3 3 3 https://fact-check-finder.kinit.sk/ , a tool designed to assist fact-checkers in identifying relevant fact-checked claims across multiple languages. Multilingual Summarization. Multilingual summarization has been propelled by the development of extensive datasets and the application of LLMs Scialom et al. (2020) ; Hasan et al. (2021) ; Bhattacharjee et al. (2023) . These resources have enabled the fine-tuning of multilingual models like mT5 Xue et al. (2021) , which demonstrate competitive performance in both multilingual and low-resource summarization tasks. Furthermore, studies have explored the zero-shot and few-shot capabilities of LLMs such as GPT-3.5 and GPT-4 in cross-lingual summarization, highlighting their potential to handle diverse language pairs without extensive fine-tuning Wang et al. (2023) . Efforts to enhance factual consistency in multilingual summarization have also been made, exemplified by the use of multilingual models to improve the reliability of machine-generated summaries across languages Aharoni et al. (2023) . 3 Methodology Our experiments aim to evaluate the capabilities of TEMs and LLMs in assisting fact-checkers by providing additional information. This includes retrieving the most similar previously fact-checked claims, summarizing fact-checking articles along with their ratings, and potentially predicting the veracity of a given input based on the retrieved information. Much of this process

Read the complete text