Notebook for the CheckThat! Lab at CLEF 2025
Manan Sharma, Arya Suneesh, Manish Jain, Pawan Kumar Rajpoot, Prasanna Devadiga, Bharatdeep Hazarika, Ashish Shrivastava, Kishan Gurumurthy, Anshuman B Suresh, Aditya U Baliga · 2025
Read Notebook for the CheckThat! Lab at CLEF 2025 by Manan Sharma, Arya Suneesh, Manish Jain, Pawan Kumar Rajpoot, Prasanna Devadiga, Bharatdeep Hazarika, Ashish Shrivastava, Kishan Gurumurthy, Anshuman B Suresh, Aditya U Baliga (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05078.txt.
Introduction
Manan Sharma and co-authors tackle claim normalization for social media posts in many languages, as part of the CheckThat! work. The task turns noisy posts into clear claims that a checker can verify, here across 20 languages.
Their method breaks each post down with who, what, where, when, why, and how questions, which aids transfer across languages though training used English only. They fine-tuned a Qwen model with added ranked examples at run time. Scores were strongest for English and fell for some other languages, which shows the gap that cross-language work still faces.
Catalog record
| CREATOR | Manan Sharma, Arya Suneesh, Manish Jain, Pawan Kumar Rajpoot, Prasanna Devadiga, Bharatdeep Hazarika, Ashish Shrivastava, Kishan Gurumurthy, Anshuman B Suresh, Aditya U Baliga |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05078
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Notebook for the CheckThat! Lab at CLEF 2025 Manan Sharma, Arya Suneesh, Manish Jain, Pawan Kumar Rajpoot, Prasanna Devadiga, Bharatdeep Hazarika, Ashish Shrivastava, Kishan Gurumurthy, Anshuman B Suresh, Aditya U Baliga Abstract We address claim normalization for multilingual misinformation detection - transforming noisy social media posts into clear, verifiable statements across 20 languages. The key contribution demonstrates how systematic decomposition of posts using Who, What, Where, When, Why and How questions enables robust cross-lingual transfer despite training exclusively on English data. Our methodology incorporates finetuning Qwen3-14B using LoRA with the provided dataset after intra-post deduplication, token-level recall filtering for semantic alignment and retrieval-augmented few-shot learning with contextual examples during inference. Our system achieves METEOR scores ranging from 41.16 (English) to 15.21 (Marathi), securing third rank on the English leaderboard and fourth rank for Dutch and Punjabi. The approach shows 41.3% relative improvement in METEOR over baseline configurations and substantial gains over existing methods. Results demonstrate effective cross-lingual generalization for Romance and Germanic languages while maintaining semantic coherence across diverse linguistic structures. [[email protected] ] *1 1 [[email protected] ] *1 1 [[email protected] ] *1 1 [[email protected] ] [[email protected] ] [[email protected] ] [[email protected] ] [[email protected] ] [[email protected] ] [[email protected] ] keywords claim normalization ,misinformation detection ,multilingual NLP ,social media analysis † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: CLEF 2025 Working Notes, 9 – 12 September 2025, Madrid, Spain † † address: TIFIN † † corresp: Corresponding author. † † These authors contributed equally. 1 Introduction Misinformation represents the foremost global threat for 2025, according to the World Economic Forum’s Global Risks Report 8 , while false news spreads up to 10 times faster than accurate reporting on social media platforms 6 . Social media giants have recently abandoned traditional fact-checking programs in favor of community-driven approaches 4 , creating new gaps in verification systems precisely when misinformation campaigns target everything from elections to disaster response. The noisy nature of social media posts makes it challenging to identify important claims that require manual fact-checking, forcing researchers to develop automated solutions for processing the overwhelming volume of misleading content. Our work addresses this critical challenge through CheckThat! Lab CLEF 2025 Task 2: Claim Normalization, which focuses on transforming chaotic social media posts into clear, verifiable statements across 20 languages. This text generation task requires systems to extract core assertions from noisy posts and present them in normalized forms suitable for fact-checking pipelines, representing a fundamental step toward scaling verification efforts to match the speed and volume of misinformation spread. 1.1 Task Overview CheckThat! Lab CLEF 2025 1 2 Task 2 21 introduces the problem of simplifying noisy social media posts into normalized claims that fact-checkers process efficiently. The task operates across 20 languages including English, Arabic, German, French, Spanish, Hindi and 14 others, requiring systems to handle diverse linguistic structures and cultural contexts. Participants face two distinct settings: monolingual, where training, development and test data exist for the same language and zero-shot, where only test data exists for the target language. The monolingual setting covers 13 languages with full datasets, while the zero-shot setting evaluates generalization across 7 languages including Dutch, Romanian, Bengali, Telugu, Korean, Greek and Czech. Posts originate from various social media platforms including Twitter, Reddit and Facebook, sourced from Google Fact-check Explorer to ensure real-world relevance. Systems generate normalized claims evaluated using METEOR score, measuring the quality of simplified text against human-annotated ground truth. This research addresses the practical challenge faced by fact-checkers who must process thousands of posts daily, extracting verifiable claims from content laden with hashtags, mentions, emojis and informal language that obscures the core assertions requiring verification. 2 Related Work At its core, claim normalization is an abstractive generation task closely related to summarization, but with key differences. Sequence-to-sequence models like BART 16 or T5 19 have advanced general-purpose summarization. Controlled summarization techniques allow setting summary length or focus 20 14 10 . However, generic summaries may omit critical facts or introduce hallucinations, making them unreliable for fact-checking. For example, Kryściński et al. (2020) 15 showed that abstractive models often add contradictory information. Utama et al. (2022) 23 and Durmus et al. (2020) 7 developed QA-based checks for factual consistency. Claim normalization instead prioritizes factual precision and context-independence: the generated claim must be fully verifiable on its own. Sundriyal et al. 22 note that unlike typical summaries, normalized claims “must be self-contained and verifiable”. This means, for example, resolving entities or adding minimal context so that the claim cannot be misunderstood when isolated (e.g. clarifying that “Bird” refers to the scooter company, rather than the animal). In practice, many systems treat normalization as a specialized summarization. For instance, Reddy et al. (2024) 11 recast document-level claim extraction as extractive summarization followed by decontextualization: they extract central sentences and then use a QA-based model to expand them into stand-alone claims. This approach yielded higher relevance (precision@1) and fact consistency in their test cases. Similarly, models trained for text summarization (T5/BART/PEGASUS) have been applied directly as baselines for normalization. However, the unique goal of preserving a single factual assertion often calls for tailored strategies. 3 Methodology Figure 1: System workflow for multilingual claim normalization using fine-tuned Qwen3-14B with 5W1H reasoning framework and retrieval-augmented few-shot prompting. 3.1 Model Architecture We employ Qwen3-14B 25 as our base model due to its strong multilingual capabilities and efficient architecture for fine-tuning across diverse languages. Qwen3-14B demonstrates robust performance on multilingual tasks, achieving 79.69 on the MMMLU benchmark, while maintaining computational efficiency, making it well-suited for our cross-lingual claim normalization objectives. The model’s strong multilingual foundation provides an ideal starting point for fine-tuning across diverse languages without sacrificing performance on cross-lingual understanding tasks. We fine-tune the model using Low-Rank Adaptation (LoRA) 12 with 4-bit quantization for memory efficiency. Our training configuration includes: • LoRA rank r=16 , scaling factor \alpha=32 , dropout rate 0.05 • Target modules: attention and projection layers • Training epochs: 3 • Per-device batch size: 6, gradient accumulation steps: 4 (effective batch size: 24) • Optimizer: paged AdamW 8-bit with learning rate 3\times 10^{-4} • Precision: bfloat16 with gradient checkpointing enabled • Hardware: Single NVIDIA A100 GPU (40GB VRAM) 3.2 Data Preprocessing The CheckThat! Lab CLEF 2025 Task 2 dataset (Table 1 ) encompasses 26,399 instances across 20 languages, representing one of the most comprehensive multilingual collections for claim normalization research. The dataset exhibits significant linguistic diversity, with English comprising the largest subset (13,830 instances), followed by Spanish (4,336), Portuguese (2,183) and French (1,469). Thirteen languages provide complete training, development and test splits for monolingual evaluation, ranging from high-resource languages like English to lower-resource languages such as Tamil (252 instances) and Polish (304 instances). Seven additional languages—Bengali, Czech, Greek, Korean, Dutch, Romanian and Telugu—are included exclusively for zero-shot evaluation with 1,068 test instances total. Post lengths vary dramatically across languages and cultural contexts, from concise Tamil posts averaging 26 words to verbose Czech posts averaging 332 words, while normalized claims maintain relative consistency (8.87-19.85 words) across all languages. Table 1: Dataset overview for CheckThat! Lab CLEF 2025 Task 2: Claim Normalization across 20 languages. Language Train Dev Test Total Avg.