TSVer: A Benchmark for Fact Verification Against Time-Series Evidence

Marek Strong, Andreas Vlachos · 2025

Read TSVer: A Benchmark for Fact Verification Against Time-Series Evidence by Marek Strong, Andreas Vlachos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01101.txt.

Introduction

Marek Strong and Andreas Vlachos introduce TSVer, a benchmark for checking factual claims against time series evidence. Fact checking often turns on temporal and numerical data, such as trends in official statistics, but existing data sets lack structured evidence, give thin justifications, or invent synthetic claims.

TSVer holds just over three hundred real world claims gathered from forty one fact checking organizations, matched against a curated database of four hundred time series from varied domains. Each claim is annotated with the relevant time frames, a verdict, and justifications showing how the evidence supports it, with strong agreement between annotators. Even a leading reasoning model scores modestly on verdicts and worse on justifications, showing how hard time series reasoning remains.

Catalog record

CREATORMarek Strong, Andreas Vlachos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01101

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TSVer: A Benchmark for Fact Verification Against Time-Series Evidence Marek Strong, Andreas Vlachos Abstract Reasoning over temporal and numerical data, such as time series, is a crucial aspect of fact-checking. While many systems have recently been developed to handle this form of evidence, their evaluation remains limited by existing datasets, which often lack structured evidence, provide insufficient justifications for verdicts, or rely on synthetic claims. In this paper, we introduce TSVer , a new benchmark dataset for fact verification focusing on temporal and numerical reasoning with time-series evidence. TSVer contains 287 287 real-world claims sourced from 38 38 fact-checking organizations and a curated database of 400 400 time series covering diverse domains. Each claim is annotated with time frames across all pertinent time series, along with a verdict and justifications reflecting how the evidence is used to reach the verdict. Using an LLM-assisted multi-step annotation process, we improve the quality of our annotations and achieve an inter-annotator agreement of κ = 0.745 \kappa=0.745 on verdicts. We also develop a baseline for verifying claims against time-series evidence and show that even the state-of-the-art reasoning models like Gemini-2.5-Pro are challenged by time series, achieving a 63.37 63.37 accuracy score on verdicts and an Ev 2 ​ R \mathrm{Ev}^{2}\mathrm{R} score of 48.63 48.63 on verdict justifications. 1 Introduction Figure 1: Example claim from TSVer. Our dataset includes real-world claims paired with historical time-series evidence. All claims are annotated with time ranges (blue boxes), verdicts, and justifications emphasizing numerical and temporal reasoning. With the growing use of social media and generative AI, there has been an unprecedented increase in the amount of inaccurate and misleading information Adams et al. 2023 ; Arnold 2020 . In response, automated fact-checking systems have advanced substantially with the application of new large language models (LLMs) and the development of comprehensive datasets Vykopal et al. 2024 . These systems have shown promising results in identifying and verifying claims across diverse domains and languages Strong et al. 2024 . However, they continue to face challenges when assessing claims that rely on external evidence Fontana et al. 2025 or when evaluating claims requires deeper reasoning beyond surface-level textual cues Choi and Ferrara 2023 ; Dziri et al. 2023 . One of the areas where the reasoning limitations of LLMs are particularly prominent is numerical and temporal reasoning Akhtar et al. 2023a ; Bubeck et al. 2023 . This is particularly problematic in the context of fact-checking, where numerical and temporal expressions are prevalent. Studies have shown that over one-third of check-worthy claims involve numerical data V et al. 2024 ; Hassan et al. 2017 , and many claims demand complex numerical reasoning to be properly evaluated Aly et al. 2021 . Furthermore, as facts can evolve and change over time, fact-checking systems must be capable of accurately interpreting and reasoning over temporal aspects of both claims and supporting evidence Allein et al. 2023 ; Allein et al. 2020 . Therefore, it is increasingly important to effectively evaluate these capabilities and ensure that fact-checking systems can reliably reason over numerical and temporal data. In this work, we introduce a new benchmark for evaluating fact verification systems using time-series evidence. Time series is a modality shown to be challenging for language models ( Merrill et al. 2024 ; Fons et al. 2024 ) and is frequently used by human fact-checkers for fact verification Akhtar et al. 2023b ; Alam et al. 2021 . To address this gap, we present TSVer —the first benchmark dataset for explainable fact verification grounded in time-series evidence. TSVer pairs real-world claims with historical time-series evidence sourced from fact-checking organizations and includes textual justifications for verdicts, allowing for the evaluation of reasoning about evidence. To construct TSVer , we collected 287 claims from 38 fact-checking organizations, focusing on those involving numerical and temporal expressions resolvable via time series data. These claims were then aligned with our curated database of 400 time series, extracted from Our World in Data 1 1 1 https://ourworldindata.org/ . We avoided claims solvable by simple look-ups or simple arithmetic operations (common in numerical datasets Lu et al. 2023 ) and instead targeted claims requiring reasoning across multiple countries, time series, and claims containing temporal and numerical ambiguities. While time series can be seen as tabular data, their temporal structure and scale add complexity. Compared to prior datasets, TSVer features much larger time series, averaging around 20,000 records per instance, with some over 217,000 records, posing new challenges for fact verification on high-volume, real-world data. Figure 1 illustrates an example claim from TSVer with annotations for the evidence, verdict, and justification. To fact-check this claim, a system must identify the relevant time series from our dataset (i.e., Greenhouse Gas Emissions, Gross Domestic Product (GDP), and Annual GDP Growth), determine the relevant time frames (i.e., 1990–2020), reason over all data points within these time frames for relevant countries (i.e., the United Kingdom), and generate a verdict accompanied by a justification. Identifying relevant time frames is particularly challenging in our benchmark, as selecting different date ranges often leads to different verdicts. Politicians frequently exploit this by choosing selective dates to support their claims, a practice known as cherry-picking Asudeh et al. 2020 . Additionally, a time series may contain both supporting and contradicting periods. For example, while the Gemini-2.5 Pro reasoning model correctly selects 1990 as the starting year (a common baseline for climate-related claims), it uses the period 1990–2019 rather than 1990–2020 to provide supporting evidence. This contradicts the reasoning of our annotators and the original fact-checking article, which notes that Boris Johnson, who made the claim in 2021, relied on outdated figures that ignore the extraordinary impact of the COVID-19 pandemic. We also propose a fact-checking pipeline as a baseline to demonstrate the feasibility of the task and to benchmark the performance of the state-of-the-art open-weight and proprietary language models. The gemini-2.5-pro-preview-03-25 reasoning model Anil et al. 2023 , currently ranked first on ChatBot Arena Chiang et al. 2024 , achieves an accuracy of 63.37 on verdict prediction. Additionally, to evaluate models’ reasoning in comparison to human annotators, we use the \mathrm{Ev}^{2}\mathrm{R} scorer Akhtar et al. 2024 , originally developed for evidence retrieval, and demonstrate its effectiveness in this new context. Furthermore, to specifically evaluate evidence retrieval performance with time series data, we introduce a novel metric— TSCS , which jointly measures the accuracy of both time series selection and temporal coverage. Our dataset is available under a CC BY-SA 4.0 license at https://github.com/marekstrong/TSVer . 2 Related Work We summarize key characteristics of existing fact verification datasets in Table 1 . In the following , we compare these datasets to TSVer along three key dimensions: evidence modalities, numerical and temporal focus, and the inclusion of human-written justifications. Dataset Domain #Labels Real-world Claims Numerical Focus Temporal Focus Evidence Modality Justifications FEVER Thorne et al. 2018 Multi 3 ✗ ✗ ✗ Text ✗ TabFact Chen et al. 2019 Multi 2 ✗ ✓ ✗ Tables ✗ FEVEROUS Aly et al. 2021 Multi 3 ✗ ✗ ✗ Text + Tables ✗ AVERITEC Schlichtkrull et al. 2023 Multi 4 ✓ ✗ ✗ Text ✓ SciTab Lu et al. 2023 Science 3 ✓ ✓ ✗ Tables ✗ Liar++ Russo et al. 2023

Read the complete text