ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims
2026
Read ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26449.txt.
Introduction
Abu Ahmad and co-authors report ClimateCheck 2026, a shared task on checking climate claims against science. The task pairs fact-checking of claims with a new job: naming the disinformation narrative behind a claim. This second run triples the training data of the 2025 edition. Twenty teams signed up and eight filed leaderboard entries, mixing dense retrieval, cross-encoders, and language models. The report also shows how standard scores can misrank systems when labels are incomplete, and that some narratives resist checking more than others.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49726 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26449
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims Abstract Automatically verifying climate-related claims against scientific literature is a challenging task, complicated by the specialised nature of scholarly evidence and the diversity of rhetorical strategies underlying climate disinformation. ClimateCheck 2026 is the second iteration of a shared task addressing this challenge, expanding on the 2025 edition with tripled training data and a new disinformation narrative classification task. Running from January to February 2026 on the CodaBench platform, the competition attracted 20 registered participants and 8 leaderboard submissions, with systems combining dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning. In addition to standard evaluation metrics (Recall@K and Binary Preference), we adapt an automated framework to assess retrieval quality under incomplete annotations, exposing systematic biases in how conventional metrics rank systems. A cross-task analysis further reveals that not all climate disinformation is equally verifiable, potentially implicating how future fact-checking systems should be designed. Keywords: scientific fact-checking, disinformation narrative classification, climate change, shared task ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims Raia Abu Ahmad 1, 2 Max Upravitelev 1, 2 Aida Usmanova 3 Veronika Solopova 1, 2 Georg Rehm 1, 4 1 Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Germany 2 Technische Universität Berlin, Germany 3 Leuphana Universität Lüneburg, Germany 4 Humboldt-Universität zu Berlin, Germany Corresponding author: [email protected] Abstract content 1. Introduction Public understanding of climate change is increasingly shaped by online discourse, the scientific validity of which is often difficult to assess at scale 17 . While automatic fact-checking (AFC) has made substantial progress in recent years 19 , most existing approaches verify claims against general sources such as Wikipedia or web search 42 ; 7 ; 37 . However, recent studies emphasise the importance of using trustworthy evidence sources ( 38 ) , which, for scientific topics, means direct engagement with scholarly literature. Although scientific publications constitute one of the most authoritative forms of evidence for climate-related claims, they remain challenging for AFC systems. This stems from several difficulties when dealing with scholarly document processing, such as the use of in-domain terminology, document length, complex reasoning requirements, connection to other documents through citations, and temporal changes of evidence veracity 44 ; 14 . To tackle this, we introduced the ClimateCheck shared task 3 , focusing on the verification of climate-related social media claims using scientific abstracts as evidence. The task consists of retrieving relevant abstracts for a given claim, and classifying whether the text supports , refutes , or does not have enough information (NEI) about the claim, treating the problem at the level of individual claim–abstract pairs (CAPs) instead of producing one final verdict. The competition demonstrated the feasibility of grounding AFC in scholarly knowledge, but also exposed several limitations, including restricted training data, evaluation challenges caused by incomplete annotations, and a growing reliance on computationally expensive large language models (LLMs). This paper presents the 2026 iteration of ClimateCheck, in which we address these challenges and substantially expand the scope of our dataset. We release approx. three times more training data and introduce a new task, Disinformation Narrative Classification , which moves beyond claim-level verification to capture the broader rhetorical structures underlying climate disinformation. Crucially, all three tasks: retrieval, verification, and disinformation narrative classification, are annotated over the same set of claims, providing a unified dataset for evidence-grounded narrative classification, as demonstrated in Figure 1 . 1 1 1 Our dataset is publicly available: https://huggingface.co/datasets/rabuahmad/climatecheck To the best of our knowledge, this is the first work that enables systematic investigation of how rhetorical narrative structure interacts with evidence retrieval and veracity classification. Our contributions are threefold: 1. A unified benchmark connecting scholarly evidence retrieval, claim verification, and narrative classification. 2. An adapted evaluation framework for incomplete annotations in multi-evidence scientific fact-checking, based on the work proposed by 6 . 3. An empirical analysis of system behaviour across retrieval, verification, and narrative tasks, highlighting systematic verification biases and persistent challenges in fine-grained narrative classification. ClimateCheck 2026 ran from January 15 until February 18 on the CodaBench platform 51 , with registration from 20 participants and 8 leaderboard submissions across all tasks. Four teams submitted system descriptions, three of which outperformed our baselines. In this paper, we report our dataset development process (§ 4 ), evaluation framework (§ 5 ), baselines design (§ 6 ), participant systems (§ 7 ), and error analyses across tasks (§ 8 ), which we discuss and conclude in § 9 and § 10 , respectively. Figure 1: Instance from ClimateCheck 2026. Task 1: Given a claim, systems must retrieve relevant abstracts and use them for verification. Task 2: Given a claim, systems must predict all disinformation narratives associated with it. 2. Related Work Shared tasks have played a central role in advancing AFC research by establishing common benchmarks and evaluation frameworks. Early efforts such as FEVER 42 and FEVEROUS 7 focused on verifying claims against Wikipedia, while subsequent work expanded into more specialised retrieval. SciFact 45 ; 46 , for example, introduced expert-written scientific claims paired with biomedical abstracts, paving the way for the SCIVER shared task 47 , which demonstrated the particular challenges of reasoning over scholarly publications. More recently, AVeriTeC 37 ; 5 shifted the focus to real-world claims verified using open-web evidence, highlighting the importance of trustworthy and traceable sources. As these tasks have grown in complexity, LLM-based systems have come to dominate leaderboards 5 ; 37 ; 3 . However, recent work suggests that this trend does not consistently yield superior performance, with smaller or task-specialised models remaining competitive in climate-related settings 10 ; 43 . Claim-level verification, however, addresses only one dimension of the problem. Climate disinformation rarely operates through isolated false claims, as it tends to cluster around recurring narrative patterns that span many individual statements. Several datasets have been developed to capture these patterns, grouping climate denial claims by their underlying narrative messages 12 ; 35 ; 31 . However, these datasets annotate narrative labels at the claim level without linking them to scientific evidence. To the best of our knowledge, our dataset is the first to connect narrative classification and evidence-based claim verification, enabling the exploration of strategies jointly targeting these tasks. 3. Task Definitions The 2026 iteration of ClimateCheck consisted of two tasks: • Task 1: Abstract retrieval and claim verification : given a social media claim about climate change, retrieve the top 5 most relevant abstracts from the given publications corpus (task 1.1) and classify each claim-abstract pair as supports , refutes , or NEI (task 1.2). • Task 2: Disinformation narrative classification : given a social media claim about climate change, predict which disinformation narrative(s) are present using the schema proposed by 12 . Both tasks were evaluated independently, and participants could choose to submit to either or both. Participants were allowed to use external datasets and apply data augmentation methods. 4. Dataset Development Both tasks build on the ClimateCheck dataset introduced in the 2025 iteration 2 . For task 1, we extend the training split by adding 523 unique English claims, drawn from the original claims pool, and 1897 CAPs, where each claim is linked to 1-5 abstracts. The testing split remains unchanged.