Combining Evidence and Reasoning for Biomedical Fact-Checking Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Develop

Mariano Barone, Antonio Romano, Giuseppe Riccio, Marco Postiglione, Vincenzo Moscato · 2025

Read Combining Evidence and Reasoning for Biomedical Fact-Checking Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Develop by Mariano Barone, Antonio Romano, Giuseppe Riccio, Marco Postiglione, Vincenzo Moscato (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-13879.txt.

Introduction

Mariano Barone and four co-authors build a biomedical fact-checker in a 2025 paper. The target is health misinformation, from vaccine hesitancy to unproven cures. Checking medical claims is harder than checking ordinary news. The terminology is technical and the stakes are high. Verdicts must rest on real scientific evidence, not on a model's memory.

Their framework, CER, combines three stages. It retrieves high-quality biomedical evidence. It reasons over that evidence with a large language model. And it predicts veracity with a supervised classifier. Grounding the model's output in retrieved sources curbs hallucination. On the expert-annotated HealthFC, BioASQ-7b, and SciFact datasets, CER reached the best reported performance. It also generalized promisingly across datasets. Code and data are released openly.

Catalog record

CREATORMariano Barone, Antonio Romano, Giuseppe Riccio, Marco Postiglione, Vincenzo Moscato
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-13879

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Combining Evidence and Reasoning for Biomedical Fact-Checking Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3729931 ISBN: 979-8-4007-1592-1/2025/07 CCS: Computing methodologies Natural language generation CCS: Applied computing Health care information systems Mariano Barone, Antonio Romano, Giuseppe Riccio, Marco Postiglione, Vincenzo Moscato Abstract . Misinformation in healthcare, from vaccine hesitancy to unproven treatments, poses risks to public health and trust in medical systems. While machine learning and natural language processing have advanced automated fact-checking, validating biomedical claims remains uniquely challenging due to complex terminology, the need for domain expertise, and the critical importance of grounding in scientific evidence. We introduce CER (Combining Evidence and Reasoning), a novel framework for biomedical fact-checking that integrates scientific evidence retrieval, reasoning via large language models, and supervised veracity prediction. By integrating the text-generation capabilities of large language models with advanced retrieval techniques for high-quality biomedical scientific evidence, CER effectively mitigates the risk of hallucinations, ensuring that generated outputs are grounded in verifiable, evidence-based sources. Evaluations on expert-annotated datasets (HealthFC, BioASQ-7b, SciFact) demonstrate state-of-the-art performance and promising cross-dataset generalization. Code and data are released for transparency and reproducibility: https://github.com/PRAISELab-PicusLab/CER . † † cc-license: by-nc 1. Introduction Distinguishing reliable information from misinformation represents a global challenge that threatens the foundations of informed decision-making. Misinformation not only shapes public opinion and policy outcomes but also systematically erodes trust in authoritative information sources, thereby creating a recursive cycle that further complicates the pursuit of accurate knowledge ( Battineni et al. 2020 ) . The healthcare domain exemplifies the devastating impact of misinformation on public health outcomes. Misinformation surrounding evidence-based medical interventions — including cancer treatments ( Johnson et al. 2022 ) , birth control 1 1 1 https://www.washingtonpost.com/health/2024/03/21/stopping-birth-control-misinformation/ , and other forms of medical care — has led to devastating public health consequences, including treatment delays, increased mortality rates, and erosion of trust in healthcare systems ( Gisondi et al. 2022 ) . For example, during the COVID-19 pandemic, misinformation about vaccines led to hesitancy, resulting in lower vaccination rates and increased spread of the virus. This misinformation was often propagated through social media platforms, where false claims about vaccine safety and efficacy spread rapidly ( Apuke and Omar 2021 ) . Additionally, misinformation about unproven treatments, such as the use of hydroxychloroquine, caused confusion and sometimes harmful self-medication practices ( Saag 2020 ) . CER running example Figure 1 . Combining Evidence and Reasoning ( CER ) for biomedical fact-checking. (1) Evidence supporting or opposing the biomedical claim is retrieved from scientific literature. (2) A large language model (LLM) is leveraged to justify the claim in light of the scientific evidence. (3) A classifier is trained to assess the veracity of the claim by leveraging the information collected in steps (1) and (2). CER running example Machine learning (ML) and natural language processing (NLP) techniques have emerged as powerful tools for automated fact-checking, demonstrating promising results across various domains ( Schuetz et al. 2021 ; Sharifani et al. 2022 ) . While these computational approaches have been extensively studied and deployed in political fact-checking ( Rashkin et al. 2017a ; Vo and Lee 2019 ; Rashkin et al. 2017b ) , their effectiveness in validating biomedical claims remains limited. Biomedical fact-checking presents distinct challenges that transcend the capabilities of current automated systems. Unlike many other domains, medical claims often incorporate domain-specific terminology, rely on complex causal relationships, and require sophisticated interpretation of empirical evidence ( Zhong 2023 ) . Furthermore, health-related misinformation demands particularly rigorous verification due to its potential impact on public health outcomes, necessitating both specialized domain knowledge and advanced reasoning capabilities that can contextualize claims within the broader framework of scientific evidence. Large language models (LLMs) have recently emerged as a promising approach to address these challenges in biomedical fact-checking. Their ability to process and reason about complex textual information, combined with their broad knowledge acquisition during pre-training, makes them potentially valuable tools for analyzing biomedical claims ( Lui et al. 2024 ) . LLMs have demonstrated capabilities in understanding technical medical terminology, identifying relevant scientific evidence, and providing nuanced interpretations of research findings ( Singhal et al. 2023 ) . Retrieval-augmented generation (RAG) frameworks further enhance these capabilities by grounding LLM outputs in verified scientific literature, potentially mitigating the risk of hallucinations through direct access to peer-reviewed evidence ( Kim and Yoon 2024 ; Jeong et al. 2024 ) . By dynamically retrieving and incorporating relevant medical research during the analysis process, RAG systems could provide more reliable and verifiable fact-checking results . However, these approaches still face important limitations that must be carefully considered: (1) LLMs, even when augmented with retrieval, can still generate plausible-sounding but factually incorrect information. (2) The quality of fact-checking remains heavily dependent on the relevance and reliability of the retrieved evidence, and challenges persist in efficiently identifying and incorporating the most pertinent scientific literature. (3) Additionally, their knowledge cutoff dates and potential biases in training data can lead to outdated or incomplete medical information. In this work, we propose a novel approach to biomedical fact-checking that integrates three key components: systematic scientific evidence retrieval, LLM reasoning capabilities, and supervised veracity prediction. Our system, CER (Combining Evidence and Reasoning), addresses the challenges in biomedical claim verification through a multi-stage architecture (illustrated in Figure 1 ) that synthesizes (1) the input claim, (2) relevant scientific evidence from peer-reviewed literature, and (3) LLM-generated justifications. This architecture leverages the advanced reasoning capabilities of LLMs while maintaining factual grounding through explicit evidence linkage, thereby mitigating the risk of hallucinations. The proposed architecture follows an open-domain fact-checking paradigm, which eliminates reliance on a predefined set of evidence ( Abdelnabi et al. 2022 ) . Rather than being restricted to a fixed corpus of golden evidence, this approach dynamically retrieves relevant information from an extensive external knowledge base, enabling a content-driven evaluation of claims. We evaluate CER on three benchmarks annotated by domain experts: HealthFC ( Vladika et al. 2024a ) , BioASQ-7b ( Nentidis et al. 2020 ) , and SciFact ( Wadden et al. 2020 ) . The results show that CER consistently outperforms state-of-the-art methods, achieving up to a +3.43% improvement in F1 scores. To provide deeper insights, we analyze the performance impact of different retrieval strategies (sparse vs. dense) and examine the contributions of scientific evidence integration and LLM-based reasoning. Additionally, cross-dataset evaluations highlight the generalizability and robustness of our framework across diverse settings. While these results underscore its effectiveness in healthcare, the modular design of CER makes it adaptable to other domains requiring domain-specific evidence and advanced reasoning. To promote transparency and reproducibility, we release the code, evidence, and outputs generated during our experiments 2 2 2 Code is shared anonymously; data will be available upon publication. . 2. Combining Evidence and Reasoning ( CER ) In this section, we present our proposed methodology to classify biomedical claims 3 3 3 In this work, a claim refers to any statement or question whose truthfulness or factual accuracy can be evaluated based on external evidence.

Read the complete text