ViClaim: A Multilingual Multilabel Dataset for Automatic Claim Detection in Videos

Patrick Giedemann, Pius von Däniken, Jan Deriu, Alvaro Rodrigo, Anselmo Peñas, Mark Cieliebak · 2025

Read ViClaim: A Multilingual Multilabel Dataset for Automatic Claim Detection in Videos by Patrick Giedemann, Pius von Däniken, Jan Deriu, Alvaro Rodrigo, Anselmo Peñas, Mark Cieliebak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12882.txt.

Introduction

Patrick Giedemann and five co-authors introduce ViClaim in a 2025 paper on fact-checking video content. Videos spread claims fast, yet most detection tools work on written text. Spoken claims in transcripts, across languages and topics, remain hard to sort at scale.

ViClaim holds 1,798 annotated video transcripts in English, German, and Spanish across six topics. Each sentence is labeled as a claim worth checking, a claim not worth checking, or an opinion. The team built a custom tool to handle the complex labeling. Tests with multilingual language models scored well on known topics. Performance dropped on new topics, which shows the limits of current methods.

Catalog record

CREATORPatrick Giedemann, Pius von Däniken, Jan Deriu, Alvaro Rodrigo, Anselmo Peñas, Mark Cieliebak
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12882

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ViClaim: A Multilingual Multilabel Dataset for Automatic Claim Detection in Videos Patrick Giedemann, Pius von Däniken, Jan Deriu, Alvaro Rodrigo, Anselmo Peñas, Mark Cieliebak Abstract The growing influence of video content as a medium for communication and misinformation underscores the urgent need for effective tools to analyze claims in multilingual and multi-topic settings. Existing efforts in misinformation detection largely focus on written text, leaving a significant gap in addressing the complexity of spoken text in video transcripts. We introduce ViClaim, a dataset of 1,798 annotated video transcripts across three languages (English, German, Spanish) and six topics. Each sentence in the transcripts is labeled with three claim-related categories: fact-check-worthy, fact-non-check-worthy, or opinion. We developed a custom annotation tool to facilitate the highly complex annotation process. Experiments with state-of-the-art multilingual language models demonstrate strong performance in cross-validation (macro F1 up to 0.899) but reveal challenges in generalization to unseen topics, particularly for distinct domains. Our findings highlight the complexity of claim detection in video transcripts. ViClaim offers a robust foundation for advancing misinformation detection in video-based communication, addressing a critical gap in multimodal analysis. 1 Introduction Video content is increasing in popularity worldwide. Alone in the US, the average adult spends around 47 minutes on YouTube and 55 Minutes on TikTok 1 1 1 https://www.statista.com/statistics/1359403/us-time-spent-per-day-netflix-tiktok-youtube . Especially short-format videos (i.e., videos of at most 90 seconds) are gaining in popularity 2 2 2 https://www.yaguara.co/short-form-video-statistics/ . While platforms like YouTube serve educational and informational purposes Srinivasacharlu (2020) ; Wafa’A and Khasawneh (2024) , they are also used to disseminate narratives aimed at influencing viewer’s opinions. Despite the recognized necessity of extending misinformation detection technologies to video modalities Da San Martino et al. (2021) , research efforts remain limited, with a predominant focus on analyzing textual data from platforms like X (formerly Twitter) Arslan et al. (2020) ; Alam et al. (2023) . Furthermore, most existing research on video-based misinformation primarily targets visual features, such as detecting manipulated content Papadopoulou et al. (2018) ; Palod et al. (2019) ; Bu et al. (2023) , leaving the semantic analysis of video transcripts largely unexplored, although features found in transcripts of spoken language differ from written language in structure and delivery Dingemanse and Liesenfeld (2022) . Studies that do analyze transcripts tend to treat them as a global, binary classification task, aiming to determine whether a video contains misinformation as a whole Hou et al. (2019) ; Hussein et al. (2020) ; Papadamou et al. (2022) ; Christodoulou et al. (2023) . However, misinformation detection is a multilayered task, with claim detection as the first step Panchendrarajan and Zubiaga (2024) . In this work, we present the first step towards misinformation detection in videos by introducing ViClaim, a novel dataset where transcripts of YouTube Short videos are annotated using a custom annotation tool on a sentence level with the claim taxonomy introduced in Panchendrarajan and Zubiaga (2024) . Each sentence is annotated to indicate whether it contains a claim, an opinion, or both, and if a claim is present, whether it is check-worthy. Since sentences may fall into multiple categories simultaneously, the task is framed as a multi-label classification problem. The dataset encompasses 1,798 videos in three languages (English, German, and Spanish) across six topics—five political and one entertainment—allowing for investigating domain transfer capabilities. Each sentence was annotated by four independent annotators, resulting in a total of 17,116 annotated sentences. Thus, ViClaim offers a rich database for future research on multimodal misinformation detection. As a first step to showcase the utility of ViClaim, we trained several baseline models on the transcripts (leaving multimodal models for future work). Our best-performing model achieved an F1 score of 89.9 for checkworthiness classification, 77.6 for detecting non-check-worthy claims, and 83.6 for opinion detection. These results demonstrate both the effectiveness of the dataset and the challenges posed by the nuanced task of analyzing short-form video transcripts. To summarize, our contributions are as follows: (i) the ViClaim dataset, a multilingual, multi-topic resource for claim detection in video transcripts; (ii) a custom annotation tool and comprehensive guidelines; (iii) baseline models to showcase the potential of ViClaim. Release. We release the corpus in form of the video IDs used, the time stamps of the annotated sentences, the corresponding labels, and the code pipeline to reconstruct the corpus, which can be found under the following GitHub repository 3 3 3 https://github.com/pgied/viclaim_stt . We release the experimental pipeline and the trained models, which can be found under the following GitHub repository 4 4 4 https://github.com/pgied/viclaim_training . 2 Related Work Claim detection constitutes the first step in misinformation detection Panchendrarajan and Zubiaga (2024) where we mainly refer to claims that, due to their content, have a significant impact on public opinion or pose a risk of being widely disseminated due to their controversial nature Alam et al. (2023) . There has been vast research on claim detection, mainly on texts Zhang and Ghorbani (2020) , where most systems receive the exact statement to be checked. These claims are usually sentences extracted from a document or social media posts (e.g., a Tweet) Barrón-Cedeno et al. (2020) . On the other hand, other datasets aimed at detecting the exact span containing the claim in tweets, which is a more challenging task, but it allows systems to retrieve more accurate evidence Sundriyal et al. (2022) . The available datasets commonly focused on a single and controversial topic like environment Stammbach et al. (2023) , politics Dutta et al. (2023) or COVID-19 Salek Faramarzi et al. (2023) . Some datasets contain claims about several topics like elections or COVID-19 Kazemi et al. (2021) . The most common language for these datasets is English. Still, there have also been several efforts for creating datasets in multiple languages, like the dataset from the NLP4IF 2021 shared task, with tweets in English, Arabic, and Bulgarian Shaar et al. (2021) , extended in the CLEF2022 CheckThat! lab with Dutch tweets Alam et al. (2021) . Although most of the work on claim detection has been devoted to text content, there have also been some efforts towards detecting claims in multimedia content. These works consider that multimedia content in social networks is an effective way for spreading misinformation compared with textual content Jin et al. (2017) ; Dhawan et al. (2022) . One of the first works in the multimedia setting extended previous text-based collections with images to evaluate multimodal detection approaches Cheema et al. (2021) . However, the labels were only based on the textual content. This is why Cheema et al. (2022) created the MM-Claims dataset, which includes tweets and corresponding images and where the annotations are based on both text and images. The CheckThat! Lab 23 followed their methodology Alam et al. (2023) , where the organizers show the tweet’s image alongside the tweet itself, and the image could be a piece of evidence or contain a text containing a claim. The inclusion of multimodal data yielded better scores von Däniken et al. (2023) . On the other hand, to the best of our knowledge, there is almost no work on claim detection using videos. The most relevant work is the ClaimBuster dataset Arslan et al. (2020) , which provides 23,533 statements extracted

Read the complete text