DeepSciVerify : Verifying Scientific Claim–Citation Alignment via LLM-Driven Evidence Escalation

Shaghayegh Sadeghi*, Khashayar Khajavi*, Rise Adhikari, Alexander Tessier · 2026

Read DeepSciVerify : Verifying Scientific Claim–Citation Alignment via LLM-Driven Evidence Escalation by Shaghayegh Sadeghi*, Khashayar Khajavi*, Rise Adhikari, Alexander Tessier (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27710.txt.

Introduction

Shaghayegh Sadeghi, Khashayar Khajavi, Rise Adhikari, and Alexander Tessier present DeepSciVerify, a two-stage checker for claim-citation alignment in scientific text. The system reasons at the abstract level and escalates only unsure cases to full-text passage evidence, using complementary model behaviors under uncertainty. On the SCitance benchmark it reaches about 87 Micro-F1, beating abstract-only rivals by about 5 points while settling 67 percent of cases without full-text retrieval.

Catalog record

CREATORShaghayegh Sadeghi*, Khashayar Khajavi*, Rise Adhikari, Alexander Tessier
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55274
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27710

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DeepSciVerify : Verifying Scientific Claim–Citation Alignment via LLM-Driven Evidence Escalation Shaghayegh Sadeghi*, Khashayar Khajavi*, Rise Adhikari, Alexander Tessier Abstract Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. We present DeepSciVerify , a two-stage pipeline for scientific claim–citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence. The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary. This design leverages complementary behaviors across LLMs, as some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1, outperforming strong abstract-only baselines by +4.5 points while resolving 67% of instances without full-text retrieval. These results suggest that selective evidence escalation improves both accuracy and efficiency in claim–citation verification. † † affiliationnotice: * Equal contribution. 1 Introduction Scientific claim verification ( 17 ; 2 ; 1 ; 19 ) is the task of determining whether a scientific claim is supported, contradicted, or not sufficiently resolved by evidence from the research literature. In generated scientific reports, scientific claim verification can be framed as claim–citation alignment: given a claim and its associated citation, the goal is to verify whether the cited work actually provides evidence for the claim. This distinction is important because a citation may be bibliographically valid while still being semantically misaligned with the statement it is used to support. As the volume of scientific publications continues to grow, and as AI-generated text becomes increasingly common in research workflows ( 9 ; 10 ; 5 ; 7 ) , automatic verification of claim–citation alignment has become an important requirement for maintaining the reliability and trustworthiness of scientific writing. Figure 1 : Overview of the proposed two-stage claim verification pipeline. The system first performs abstract-level verification with an early-exit mechanism and escalates to passage-level retrieval and verification only when abstract evidence is insufficient. Existing work in scientific claim verification has largely been developed around benchmark settings where claims are verified against abstracts from a fixed scientific corpus. Systems such as SciFact ( 17 ) and SCIVER ( 19 ) provide a controlled setting for evaluating evidence retrieval and veracity prediction, but they simplify several challenges that arise in practical claim–citation verification. In particular, abstracts provide only a compressed summary of a paper and often omit methodological details, experimental conditions, quantitative results, ablation findings, or limitations that are necessary to assess whether a cited work truly supports a claim. As a result, an abstract may be insufficient even when the cited paper itself contains relevant evidence. To address this limitation, in this work, we propose DeepSciVerify , a two-stage claim–citation verification pipeline (Figure 1 ). Given a claim and its associated citation, the system first parses the citation into structured retrieval signals, retrieves the cited paper’s abstract through a multi-source retrieval cascade, and performs abstract-level verification with an LLM. If the abstract provides sufficient evidence to support or contradict the claim, the system returns the verdict through an early-exit mechanism. Otherwise, when the abstract evidence is insufficient, DeepSciVerify escalates to full-text verification: it retrieves the complete paper when available, extracts relevant passages using RAG-based passage retrieval, and produces a final verdict from passage-level evidence. We evaluate DeepSciVerify on SCitance ( 1 ) , a scientific claim verification dataset derived from citation sentences and evidence abstracts. In the three-class setting, DeepSciVerify achieves 86.7 Micro-F1 and 81.5 Macro-F1, outperforming the strongest abstract-only baseline by 4.5 Micro-F1 and improving over the best prior result on SCitance by 6.6 Micro-F1 (Section 6.3 ). Beyond overall performance, we analyze how different LLMs behave under evidence uncertainty. Our calibration analysis reveals distinct model-level biases: GPT-5.4 is overly cautious and frequently defaults to nei , while Claude Sonnet 4.6 is more overconfident and tends to commit to supports or contradicts even when the available evidence is incomplete (Section 6.2 ). Finally, our escalation analysis shows that DeepSciVerify resolves 67.0% of instances at the abstract level and escalates only the remaining 33.0% to full-text verification, yielding a net gain on it’s correct predictions(Section D ). Our work makes the following contributions: • We propose DeepSciVerify , a two-stage claim–citation verification pipeline that combines abstract-level early exit with selective escalation to passage-level evidence. • We develop multi-source retrieval cascades for citation parsing, abstract retrieval, and full-text retrieval, enabling verification under incomplete or noisy citation metadata. • We analyze model-level calibration in abstract-only verification and show that different LLMs exhibit distinct biases under evidence uncertainty, motivating our use of complementary models across pipeline stages. • We show that selective evidence escalation improves over strong abstract-only baselines on SCitance , and provide ablations demonstrating the importance of RAG-based passage extraction 1 1 1 Code and data are available upon request. . 2 Related Work Scientific claim verification is a subarea of fact verification focused on determining whether scientific evidence supports, contradicts, or is insufficient to verify a claim. FEVER ( 15 ) established the general evidence-based verification formulation, while SciFact ( 16 ; 17 ) adapted it to the scientific domain by pairing expert-written claims with evidence abstracts. Recent benchmarks further extend this setting: SCitance ( 1 ) uses citation sentences as claims, and SCIVER ( 19 ) evaluates verification in multimodal scientific contexts. Our work is also related to retrieval-augmented generation and dense retrieval. RAG ( 8 ) , dense passage retrieval ( 4 ) , and sentence embedding methods ( 13 ) show that retrieval can improve reasoning on knowledge-intensive tasks and enable scalable search over long documents. These methods are especially relevant for scientific verification, where necessary evidence may appear outside the abstract. Despite these advances, most scientific claim verification systems remain centered on abstract-level evidence. In practical claim–citation verification, the cited paper may need to be resolved from multiple sources, and the relevant evidence may appear only in the full text. DeepSciVerify addresses this gap by combining abstract-level verification with selective escalation to passage-level evidence. 3 Problem Definition Let c denote a scientific claim and r a cited reference. The goal is to determine a veracity label y\in\{\texttt{SUPPORTS},\texttt{CONTRADICTS},\texttt{NEI}\} for the pair (c,r) based on evidence from the cited work. Here, NEI denotes not enough information , following the label schema used in prior work ( 16 ; 15 ) . This formulation captures the task of claim–citation alignment, where the objective is to verify whether the cited reference provides sufficient evidence to support or contradict the claim, or whether the available evidence is inconclusive. 4 Methodology We present DeepSciVerify , a two-stage pipeline for claim–citation verification (Figure 1 ). The system first attempts to verify claims using abstract-level evidence and escalates to passage-level analysis only when necessary. Each stage consists of a retrieval step followed by an evidence comparison step. The pipeline operates as a staged decision process. In the first stage, an abstract-level verifier f_{a} produces a prediction \hat{y}_{a} based on abstract evidence e_{a} . If \hat{y}_{a}\neq\texttt{NEI} , the system returns this prediction. Otherwise, the system retrieves passage-level evidence e_{p}=\{p_{1},\ldots,p_{k}\} from the full text and applies a second verifier f_{p} to produce the final label: \hat{y}=\begin{cases}f_{a}(c,e_{a}),&\text{if }f_{a}(c,e_{a})\neq\texttt{NEI},\\[4.0pt] f_{p}(c,e_{p}),&\text{otherwise.}\end{cases} (1) 4.1 Phase

Read the complete text