Large Language Models for Multilingual Previously Fact-Checked Claim Detection
Ivan Vykopal, Matúš Pikuliak, Simon Ostermann, Tatiana Anikina, Michal Gregor, Marián Šimko · 2025
Read Large Language Models for Multilingual Previously Fact-Checked Claim Detection by Ivan Vykopal, Matúš Pikuliak, Simon Ostermann, Tatiana Anikina, Michal Gregor, Marián Šimko (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02737.txt.
Introduction
Vykopal and co-authors run in 2025 the first broad test of large language models at spotting claims that fact checkers have already judged, across many languages. Checkers waste effort when the same false claim resurfaces abroad in a new tongue. A tool that matches a fresh claim to an old check would cut that waste.
Seven models faced the task in twenty languages, both inside one language and across pairs. The models did well where training data is rich and stumbled where it is thin. Translating claims into English first lifted the weak languages notably. The results seed a practical path for checker support tools.
Catalog record
| CREATOR | Ivan Vykopal, Matúš Pikuliak, Simon Ostermann, Tatiana Anikina, Michal Gregor, Marián Šimko |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-02737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large Language Models for Multilingual Previously Fact-Checked Claim Detection Ivan Vykopal, Matúš Pikuliak, Simon Ostermann, Tatiana Anikina, Michal Gregor, Marián Šimko Abstract In our era of widespread false information, human fact-checkers often face the challenge of duplicating efforts when verifying claims that may have already been addressed in other countries or languages. As false information transcends linguistic boundaries, the ability to automatically detect previously fact-checked claims across languages has become an increasingly important task. This paper presents the first comprehensive evaluation of large language models (LLMs) for multilingual previously fact-checked claim detection. We assess seven LLMs across 20 languages in both monolingual and cross-lingual settings. Our results show that while LLMs perform well for high-resource languages, they struggle with low-resource languages. Moreover, translating original texts into English proved to be beneficial for low-resource languages. These findings highlight the potential of LLMs for multilingual previously fact-checked claim detection and provide a foundation for further research on this promising application of LLMs. 1 Introduction The proliferation of false information is a global issue affecting societies across diverse linguistic and cultural boundaries. With the growing spread of false information across digital platforms, fact-checking has become a crucial mechanism for verifying claims and debunking false narratives. However, the process is resource-intensive, especially when conducted manually. Thus, the automation of fact-checking is an important field of research, enabling effective countermeasures against the growing scale of false information Vosoughi et al. 2018 ; Aïmeur et al. 2023 . Figure 1: An example of a Facebook post with four previously fact-checked claims retrieved by the multilingual E5 embedding model, annotated by human annotators for relevance. Two claims are relevant to the post, while two are irrelevant. Previously fact-checked claim detection (PFCD) is an important sub-task of fact-checking that focuses on identifying claims that have already been verified. Here, AI systems determine for a given claim whether it has been previously fact-checked by a human, usually by comparing the claim at hand against thousands of already checked claims in a database. An example is shown in Figure 1 . Fact-checking organizations are seeking ways to reduce duplicative efforts and efficiently address false claims that may circulate across languages, which is hard to check for humans that are not capable of speaking all such languages Barrón-Cedeño et al. 2020 ; Nakov et al. 2021 ; Micallef et al. 2022 ; Hrckova et al. 2024 . LLMs have shown promise in automating various fact-checking tasks like claim detection, evidence retrieval or claim verification Vykopal et al. 2024 . For detecting previously fact-checked claims, LLMs could enhance the identification of relevant fact-checks even across languages, addressing challenges like linguistic diversity and the limited capabilities of existing embedding-based methods, which often underperform for low-resource languages Kazemi et al. 2021 ; Pikuliak et al. 2023 . We believe that multilingual LLMs are beneficial because of their cross-lingual capabilities and ability to leverage resources from other, higher-resourced languages. In this study, we evaluate the capabilities of seven LLMs to identify relevant previously fact-checked claims in multilingual scenarios. We include 20 languages from different language families and scripts, considering high and low-resource languages 1 1 1 Code and data are available at: https://github.com/kinit-sk/llms-claim-matching . Our research fills an important gap in detecting previously fact-checked claims in a multilingual context. Our contributions are as follows: • We provide a comprehensive evaluation of LLMs for detecting previously fact-checked claims in monolingual and cross-lingual settings, analyzing the effectiveness of different prompting techniques. Our study offers insights into multilingual PFCD using LLMs and outlines the best strategies to select appropriate LLMs and techniques tailored to language-specific contexts. • We introduce a multilingual dataset for PFCD, comprising 16K manually annotated pairs that assess the relevance between social media posts and fact-checked claims. 2 Related Work Previously Fact-Checked Claim Detection. Detecting previously fact-checked claims, also known as claim-matching, aims to identify relevant claims for a given input Shaar et al. 2020 , reducing the need to revisit previously fact-checked information. Research in this field primarily focuses on utilizing information retriever (IR) systems that measure the similarity between input claims and fact-checked claims. Most studies have evaluated these systems in monolingual settings, mostly in English Shaar et al. 2020 ; Shaar et al. 2022 ; Hardalov et al. 2022 . Recent efforts have expanded the PFCD task to a multilingual context by developing multilingual datasets and exploring the performance of existing IR systems Kazemi et al. 2021 . Pikuliak et al. 2023 created the MultiClaim dataset with over 27 languages, demonstrating that English embedding models with translated data achieved superior results compared to multilingual models with input in the original language. LLMs for Detecting Previously Fact-Checked Claims. The recent advent of highly capable LLMs has opened new possibilities for PFCD in both monolingual and multilingual settings. While most research focuses primarily on text embedding models, there are marginal efforts to investigate the performance of various LLMs. A recent survey Vykopal et al. 2024 identified only five studies utilizing LLMs for PFCD, all focused on a single language. This highlights a research gap that our study aims to address by evaluating the performance of multilingual LLMs across languages. Current approaches for PFCD with LLMs primarily employ two strategies: textual entailment Choi and Ferrara 2024a ; Choi and Ferrara 2024b and generative re-ranking Shliselberg and Dori-Hacohen 2022 ; Neumann et al. 2023 . The textual entailment classifies relationships between input claims and fact-checked claims as entailment , contradiction or neutral . Existing studies evaluated LLM’s performance using synthetic data generated from fact-checked claims for fine-tuning Choi and Ferrara 2024a . In contrast, generative re-ranking employs LLMs to compute conditional probabilities to re-rank claims based on their relevance to a given input. This method combines input and fact-checked claims, using the probabilities to prioritize more relevant claims Neumann et al. 2023 . This research demonstrates LLMs’ potential to improve PFCD but underscores the need for further exploration in multilingual contexts. 3 Methodology Figure 2: Our PFCD pipeline, consisting of (1) a retrieval of the top N most similar previously fact-checked claims (left-hand side) and (2) a classification of the relevance between social media posts and fact-checked claims using LLMs (right-hand side). We assess the ability of LLMs to determine the relevance between social media posts and previously fact-checked claims by instructing them to classify each post-claim pair as either relevant or irrelevant. Our experiments consider both monolingual settings, where the post and fact-checked claim are in the same language, and cross-lingual settings, where they are in different languages. We proposed a pipeline, illustrated in Figure 2 , to facilitate the evaluation by identifying fact-checked claims relevant to a given social media post. The pipeline consists of two main steps. First, the retriever component retrieves the N most similar previously fact-checked claims from a database (Section 3.1 ) using an embedding-based similarity. In the second step, an LLM determines the relevance of the retrieved claims to the social media post. The role of LLM is, therefore, to filter out false positives from the first stage. We consider a fact-checked claim to be relevant when it is implied by the statements in the post and when it is potentially useful for verifying the post’s veracity. To validate