CiteCheck : Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text

Khashayar Khajavi*, Shaghayegh Sadeghi*, Rise Adhikari, Alexander Tessier · 2026

Read CiteCheck : Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text by Khashayar Khajavi*, Shaghayegh Sadeghi*, Rise Adhikari, Alexander Tessier (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27700.txt.

Introduction

Khashayar Khajavi, Shaghayegh Sadeghi, Rise Adhikari, and Alexander Tessier introduce CiteCheck, a hybrid tool that spots bad citations in scientific text. The system pulls likely papers, checks each citation against its likely paper with a set verifier, and grades results Exact, Minor, or Major. On a 982-citation physics benchmark, CiteCheck reaches macro-F1 near 89 and accuracy about 89 percent, beating GPT, Claude, and Gemini rivals.

Catalog record

CREATORKhashayar Khajavi*, Shaghayegh Sadeghi*, Rise Adhikari, Alexander Tessier
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60933
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27700

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CiteCheck : Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text Khashayar Khajavi*, Shaghayegh Sadeghi*, Rise Adhikari, Alexander Tessier Abstract Large language models (LLMs) are increasingly used to generate scientific reports, but they can produce references that appear plausible while containing corrupted metadata or pointing to papers that do not exist. We introduce CiteCheck , a hybrid framework for citation hallucination detection that verifies whether a citation corresponds to a real scholarly work and whether its metadata is faithful to that work. CiteCheck retrieves candidate publications from external scholarly sources, compares the citation against the retrieved candidate using a structured LLM verifier, and maps verifier scores into three labels: Exact , Minor , and Major . We also construct a 982-citation physics benchmark with controlled corruptions that capture both subtle metadata drift and fully fabricated references. On the held-out test set, CiteCheck achieves 88.7 macro-F1 and 88.9% accuracy, outperforming GPT, Claude, and Gemini baselines, including web-search and few-shot variants. These results show that reliable citation verification benefits from combining scholarly retrieval, structured LLM-based comparison, and calibrated decision rules. † † affiliationnotice: * Equal contribution. 1 Introduction Large language models (LLMs) are rapidly becoming embedded in scientific workflows, from literature discovery and summarization to hypothesis generation, manuscript drafting, and report generation ( 40 ; 23 ) . Furthermore, recent studies have documented their growing footprint in scientific writing itself, including measurable linguistic shifts in published papers and broader use across the research lifecycle ( 19 ; 17 ) . This shift creates new opportunities for accelerating research, but it also changes the reliability requirements of generated scientific text. In scholarly writing, factuality is not only a matter of producing fluent and plausible claims: each claim must be connected to a traceable literature, and each cited source must be recoverable and correctly described. Citations are a core mechanism through which science maintains continuity, attribution, and accountability. They situate new work within prior knowledge, guide readers to supporting evidence, acknowledge intellectual contributions, and increasingly influence research evaluation and academic reputation ( 11 ; 6 ; 25 ) . Citation errors therefore do more than introduce formatting noise: they can mislead readers, obscure provenance, distort credit, and weaken the reproducibility of scientific communication. This problem becomes especially acute for LLM-generated content, where models can produce references that appear stylistically legitimate while changing authors, years, titles, DOIs, or URLs, or fabricating entire papers that do not exist ( 34 ; 24 ; 20 ; 27 ; 1 ; 28 ) . In this work, we introduce CiteCheck , a hybrid framework for detecting hallucinated references in AI-generated scientific reports. Rather than relying on the surface plausibility of a citation, CiteCheck grounds each reference against external scholarly sources and uses a structured LLM verifier to judge whether the retrieved publication actually matches the cited metadata. The system distinguishes between three outcomes: exact matches , where the citation correctly describes a real paper; minor hallucinations , where the intended paper exists but some bibliographic fields are corrupted; and major hallucinations , where the reference points to no matching scholarly work. To evaluate this setting, we construct a benchmark of 982 real physics citations and controlled corrupted variants. The benchmark covers a diverse set of physics subfields and includes both subtle metadata perturbations, such as incorrect authors, years, titles, or identifiers, and fully fabricated references that remain topically plausible. On the test set, CiteCheck achieves 88.7 macro-F1 and 88.9% accuracy, outperforming GPT, Claude, and Gemini baselines, including variants with web search and few-shot examples. Notably, CiteCheck remains zero-shot, yet surpasses the strongest few-shot LLM baseline by 5.8 macro-F1 points and 5.7 accuracy points, showing that grounded retrieval and structured comparison are more reliable than prompting LLMs to judge citation validity directly. The rest of the paper is organized as follows. Section 2 reviews work on LLM-assisted scientific writing, citation hallucination, citation integrity, and claim-level verification. Section 3 presents the CiteCheck pipeline, including candidate retrieval, LLM-based verification, threshold-based labeling, and the reviewer pass. Section 4 describes the construction of our citation-hallucination benchmark. Section 5 reports the main comparison against LLM baselines, verifier-model sensitivity, and component ablations. Section 6 discusses the implications and remaining limitations of the approach, and Section 7 concludes the paper 1 1 1 Code and data are available upon request. . Figure 1 : The CiteCheck pipeline. A raw citation string is first parsed into structured metadata, then matched against candidates from a retrieval cascade. An LLM verifier scores the citation against the retrieved candidate and the score is mapped to one of three labels (Exact / Minor / Major). When the candidate’s identifier match is suspicious, a Reviewer LLM performs a second-pass check before the final label is emitted. 2 Related Work LLMs in scientific writing and workflows. LLMs are increasingly used across scientific workflows, including literature review, summarization, hypothesis generation, manuscript drafting, and research assistance ( 40 ; 23 ) . Their use is also becoming visible in the scientific literature itself: recent work has measured linguistic signals of LLM-assisted writing in published papers and documented the growing footprint of LLMs in scientific text production ( 19 ; 17 ) . This trend motivates tools that can verify not only the fluency of generated scientific text, but also the integrity of the references on which such text depends. Citation hallucination. A growing body of work shows that LLMs can fabricate or distort bibliographic references, producing citations that appear plausible but do not correspond to real publications or contain incorrect metadata ( 34 ; 24 ; 20 ; 27 ; 1 ; 28 ) . Existing studies have documented errors such as invented titles, non-existent DOIs, incorrect authors, and fabricated papers. Closest to our setting, 1 introduce a tool for checking reference existence against scholarly metadata sources. CiteCheck differs by framing citation hallucination as a three-class severity task, distinguishing exact matches, minor metadata corruptions, and major fabricated references, and by combining scholarly retrieval, structured LLM scoring, threshold-based classification, and automatic correction in a single evaluation-driven pipeline. Citation integrity and citation ethics. Beyond the LLM setting, citations are central to scholarly communication: they connect new work to prior evidence, attribute intellectual contributions, and support the traceability and reproducibility of research. Prior work on citation integrity and citation ethics emphasizes that inaccurate or manipulated references can mislead readers, distort credit, and weaken the reliability of the scholarly record ( 11 ; 6 ; 25 ) . CiteCheck addresses this concern in the context of AI-generated scientific reports, where reference errors can be generated at scale and may be difficult to detect manually. Attribution, factuality, and claim verification. A parallel line of work studies whether generated claims are properly supported by their cited sources ( 10 ; 30 ; 14 ; 36 ; 39 ; 38 ) . Long-form factuality metrics such as FActScore ( 26 ) and self-reflective retrieval methods ( 5 ; 4 ) evaluate atomic claim support, but generally assume that references resolve to real documents. Scientific claim verification benchmarks make the same assumption by focusing on whether evidence supports or refutes a claim ( 32 ; 33 ; 21 ; 2 ; 9 ; 35 ; 13 ) . CiteCheck sits one level earlier:

Read the complete text