Think Right, Not More: Test-Time Scaling for Numerical Claim Verification
Primakov, Chungkham TU Delft [email protected] Venktesh VStockholm University [email protected] Vinay SettyUniversity of Stavanger [email protected] Avishek AnandTU Delft [email protected] · 2025
Read Think Right, Not More: Test-Time Scaling for Numerical Claim Verification by Primakov, Chungkham TU Delft [email protected] Venktesh VStockholm University [email protected] Vinay SettyUniversity of Stavanger [email protected] Avishek AnandTU Delft [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22101.txt.
Introduction
Checking numerical claims, like figures in a news report, takes several steps of reasoning and arithmetic at once. Primakov Chungkham and three co-authors study how to spend extra thinking time well on that task in this 2025 paper, accepted at EMNLP. Language models, even reasoning ones, often misread the nuance of numbers. They also suffer reasoning drift, losing the thread of a claim's parts and backtracking.
The team's approach samples many reasoning paths at test time, then uses a trained verifier model to pick the path most likely to reach the right verdict. That search reduced the drift problem and brought solid gains in verification accuracy. To save computing, an adaptive mechanism applies the heavy search only to claims that look complex. The selective version ran much more efficiently than always searching, while still beating single-shot verification by a clear margin. Code and data are shared.
Catalog record
| CREATOR | Primakov, Chungkham TU Delft [email protected] Venktesh VStockholm University [email protected] Vinay SettyUniversity of Stavanger [email protected] Avishek AnandTU Delft [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22101
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Think Right, Not More: Test-Time Scaling for Numerical Claim Verification Primakov, Chungkham TU Delft [email protected] Venktesh VStockholm University [email protected] Vinay SettyUniversity of Stavanger [email protected] Avishek AnandTU Delft [email protected] Abstract Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model ( VerifierFC ) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8 × \times higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at https://github.com/VenkteshV/VerifierFC Mitigating Reasoning drift for Numerical FC Claim : Diaspora remittances “now stand at an all-time high of over KSh400 billion in 2021”. [ Evidence 1 ]: diaspora remittances stand at an all-time-high at over ksh. 400 billion per annum as of 2021. [ Evidence2 ]: Diaspora remittances were robust in 2021, with total remittances in 2021 reaching a record USD 3,718 million . of daily users [ Verdict ]: True LLM reasoning without VerifierFC [Reasoning]: The evidence provides conflicting information about the exact amount of diaspora remittances in 2021, and some sources mention figures that are not directly comparable to KSh400 billion. [Prediction]: Conflicting LLM reasoning with VerifierFC [Reasoning]: The evidence supports the claim that diaspora remittances reached an all-time high of over KSh400 billion in 2021. However, there are minor variations in the wording and units used across different sources. [ Prediction ]: True Figure 1: Example numerical claim demonstrating reasoning drift issue and mitigation offered by VerifierFC 1 Introduction Automated verification of claims is critical to mitigate misinformation and disinformation. While many approaches focus on synthetically generated claims from Wikipedia Jiang et al. 2020 ; Thorne et al. 2018 ; Aly et al. 2021 , recent works have focused on fact-checking real-world claims Augenstein et al. 2019 ; Schlichtkrull et al. 2023 ; Chen et al. 2022 ; V et al. 2024 . Real-world claims contain numerical (quantitative and temporal) elements V et al. 2024 ; Alam et al. 2025 and verifying such claims necessitates systematically comparing numerical aspects within the claim against corresponding evidence provided. Consider the example in Figure 1 , where verifying the claim about diaspora remittances requires ensuring that the claimed amount (KSh400 billion) and the specified year (2021) align correctly with the evidence, even when the evidence presents the amount in a different currency (USD). Unlike binary verification tasks, real-world claims often contain contradictory or partially correct components, making verification more complex. Standard automated fact-checking approaches employ a multistage pipeline involving fine-tuned NLI models. Recently, Large Language Models (LLMs) have emerged as promising alternatives to NLI-based methods for claim verification Li et al. 2024 . However, these approaches have been evaluated mainly in synthetic claims, and recent studies highlight significant shortcomings when applied to complex claims in the real world that involve numerical data V et al. 2024 ; Setty 2024 ; Schlichtkrull et al. 2023 ; Augenstein et al. 2024 . Although prior work has acknowledged these limitations, they have not thoroughly analyzed why LLMs fail in realistic scenarios. To address this gap, we analyze failure modes in both open-source and proprietary models of different sizes. Beyond noise in retrieval, we identify a critical reasoning challenge even in the presence of relevant evidence introduced specifically due to multi-aspect or inability to contextualize diverse information from evidence sources, termed reasoning drift due to issues in its ability for accurate analysis of claims and evidence. This differs fundamentally from the “overthinking” phenomenon observed in simpler Q&A contexts, where models unnecessarily elaborate on obvious details Sui et al. 2025 . While it can impact simple claims, reasoning drift is especially pronounced in complex cases with multiple aspects and diverse evidences, leading to unfocused reasoning that strays from the core verification task. Our analysis reveals that approximately 34% of claims in the QuanTemp dataset suffer from this reasoning problem. In the context of fact-checking, Reasoning drift stems from LLMs focusing on unimportant aspects of the claim or misinterpretation of information in diverse evidences as part of its reasoning process. For instance, in the example in Figure 1 the claim mentions an amount of 400 billion Ksh (kenyan shillings). While one of the evidences confirm this figure, the other evidence shown in the example quotes an amount of 3718 mil. USD. Though this is equivalent to 400 billion ksh, the LLM interprets this quantity as not being equivalent to the one mentioned in the claim and falls short by billions, predicting its verdict as “Conflicting". To our knowledge, our work is the first to study the problem reasoning drift occuring due to overanalysis and explore test-time scaling as a solution to the same in the context of fact checking of real-world claims. More formally reasoning drift due to overanalysis can be defined as the phenomenon in which: “The LLM backtracks from it’s current accurate reasoning trajectory due to misinterpretation of diverse information which may also stem from multi-aspect nature of claims and inability to contextualize information." We introduce and investigate test-time scaling (TTS) Zhang et al. 2025 , an approach that allocates additional computational resources at inference time, enabling LLMs to explore multiple reasoning pathways to mitigate reasoning drift. To our knowledge, this is the first study to examine test-time scaling specifically for verifying complex factual claims. To effectively navigate and select from diverse reasoning paths, we train a dedicated verifier model ( VerifierFC ). Our experiments demonstrate that employing this verifier with a best-of-N (BoN) strategy significantly outperforms standard majority-voting Wang et al. 2023 methods. Since real-world claims are diverse, applying a fixed uniform computational effort across all claims is inefficient: simpler claims may consume unnecessary resources, and complex claims may not receive sufficient computational attention, resulting in suboptimal verification. To address this issue, we propose a novel adaptive TTS strategy that dynamically adjusts computational resources based on claim complexity, determined by analyzing layerwise latent representations within the LLMs. This adaptive method boosts verification performance by 18.8% on complex numerical claims (QuanTemp) and 21.27% on compositional claims (ClaimDecomp) compared to single-shot techniques, while also delivering 1.8x greater computational efficiency than the standard TTS approach. In this paper, we answer the following research questions through empirical validation: RQ1 : To what extent LLMs suffer from reasoning drift issue for complex numerical fact-checking