Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents

Delip Rao, Eric Wong, Chris Callison-Burch · 2026

Read Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents by Delip Rao, Eric Wong, Chris Callison-Burch (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03173.txt.

Introduction

Rao, Wong, and Callison-Burch measure how often language models and research agents invent citation addresses that never existed. Across two large collections, they find 3 to 13 percent of citations are hallucinated and up to 18 percent fail to resolve, with research agents hallucinating more than search-aided models. Their open tool, urlhealth, checks links against the Wayback Machine and cuts unresolved citations below 1 percent in self-correction tests.

Catalog record

CREATORDelip Rao, Eric Wong, Chris Callison-Burch
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68951
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03173

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents Delip Rao, Eric Wong, Chris Callison-Burch Abstract Large language models and deep research agents supply citation URLs to support their claims, yet the reliability of these citations has not been systematically measured. We address six research questions about citation URL validity using 10 models and agents on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs across 32 academic fields). We find that 3–13% of citation URLs are hallucinated—they have no record in the Wayback Machine and likely never existed—while 5–18% are non-resolving overall. Deep research agents generate substantially more citations per query than search-augmented LLMs but hallucinate URLs at higher rates. Domain effects are pronounced: non-resolving rates range from 5.4% (Business) to 11.4% (Theology), with per-model effects even larger. Decomposing failures reveals that some models fabricate every non-resolving URL, while others show substantial link-rot fractions indicating genuine retrieval. As a solution, we release urlhealth , an open-source tool for URL liveness checking and stale-vs-hallucinated classification using the Wayback Machine. In agentic self-correction experiments, models equipped with urlhealth reduce non-resolving citation URLs by 6 – 79 × 6\textrm{--}79\times to under 1%, though effectiveness depends on the model’s tool-use competence. The tool and all data are publicly available. Our characterization findings, failure taxonomy, and open-source tooling establish that citation URL validity is both measurable at scale and correctable in practice. 1 Introduction Retrieval-augmented generation (RAG) and web search integration are now standard across all major LLM providers, and every provider offers a ‘‘deep research’’ mode that generates long-form reports with inline citations. Users treat these citations as evidence. However, citation hallucination by large language models has already caused real-world harm in several notable incidents ranging from legal sanctions to academic retractions to medical misinformation 1 1 1 In Mata v. Avianca (2023) , a New York attorney was sanctioned after submitting a brief that contained fabricated case citations generated by ChatGPT . A paper co-authored by Tsinghua University researchers was withdrawn from ICLR after reviewers identified AI-generated references. Perplexity AI fabricated reviews attributed to named medical doctors. . Yet no systematic study has characterized the reliability of LLM-generated citation URLs across models, providers, and knowledge domains, and no reusable tools exist for detecting and mitigating broken and hallucinated URLs. We organize this study around six research questions: RQ1 (Prevalence) How prevalent are citation URL failures across commercial LLMs and deep research agents? RQ2 (System type) How do deep research agents compare with search-augmented LLMs? RQ3 (Domain variation) How strongly does citation reliability vary across academic fields? RQ4 (Failure composition) What fraction of citation failures is fabrication versus link rot? RQ5 (Quantity vs. quality) Does producing more citations improve per-citation reliability? RQ6 (Mitigation) Can post-hoc URL verification reduce citation hallucination? Our main findings are: 1) RQ1: Hallucinated URL rates range from 3% to 13%, with 5–18% non-resolving overall. 2) RQ2: Deep research agents exhibit the highest hallucination rates despite generating far more citations; retrieval architecture matters more than output volume. 3) RQ3: Non-resolving rates vary from 5.4% (Business) to 11.4% (Theology), with per-model effects reaching 4.3 \times . 4) RQ4: Some models fabricate every non-resolving URL, while others show substantial stale fractions indicating genuine retrieval. 5) RQ5: More citations per query do not mean fewer errors per citation. 6) RQ6: In agentic self-correction experiments, urlhealth reduces non-resolving URLs by 6\textrm{--}79\times (all p<10^{-35} ), with post-mitigation rates below 1% across all three models, though effectiveness depends on tool-use competence. 2 Definitions and Scope A citation in the context of LLM-generated text refers to a URL, bibliographic reference, or attributed snippet intended to support a claim. This work focuses on URL-based citations, which we classify as follows. A non-resolving URL is one that returns an HTTP error (4xx or 5xx status code) or fails to connect. This is the broadest class of unreliable citations. A hallucinated URL is a non-resolving URL for which no archived snapshot exists in the Wayback Machine 2 2 2 https://web.archive.org/ at any point in time. That is, there is no historical evidence the URL ever existed. 3 3 3 This is an operational definition. Because Wayback coverage is substantial but incomplete and non-uniform ( 3 ; 9 ; 8 ) , our estimates of hallucinated URL rates should be interpreted as conservative lower bounds. Three factors contribute: (1) URLs that never existed cannot appear in the Wayback Machine, but URLs that did exist may also be absent if they were never crawled, so absent Wayback snapshots under-count stale URLs and over-count hallucinated ones; (2) our pipeline excludes HTTP 403 responses (bot-blocking), which a headless-browser audit confirms are 99.7% live (Appendix A ); (3) the residual UNKNOWN category (non-200, non-404 responses) is 89% live or blocked when probed with a real browser. The only plausible over-counting direction—Wayback archiving soft-404 or wildcard-redirect pages—is small relative to these under-counting factors. Hallucinated URLs form a strict subset of non-resolving URLs. A stale URL is a non-resolving URL that does have a Wayback Machine snapshot—it existed at some point but is now unreachable. Stale URLs represent natural link rot rather than fabrication. The relationship among these categories can be formalized as \text{Hallucinated URLs}=\text{Non-resolving URLs}-\text{Stale URLs} Scope. This study focuses on URL-based citation hallucinations. Two additional failure modes—fabricated snippets (quoted text not appearing at the cited page) and invented bibliographic entries (plausible but incorrect metadata such as wrong authors or non-existent DOIs)—are documented qualitatively but require separate systematic study. 3 Experimental setup 3.1 Datasets We use two complementary datasets. DRBench ( 17 ) comprises 100 multilingual research queries (Chinese and English) covering finance, science, and technology, with pre-collected outputs from 23 models. ExpertQA ( 27 ) consists of 2,177 expert-curated questions across 32 academic and professional fields. DRBench enables cross-model comparison of citation URL validity; ExpertQA enables domain-stratified analysis of fabrication rates. A dataset summary table is provided in Appendix A . 3.2 Models Table 1 summarizes the models we evaluate. From the 23 models in DRBench, we analyze 10 from three providers (Google, OpenAI, Anthropic) for which reliable URL liveness data is available; exclusion criteria for the remaining 13 are detailed in Appendix A . We distinguish two model types. Deep research agents ( gemini-2.5-pro-deepresearch , openai-deepresearch ) execute multi-step retrieval and synthesis, producing long-form reports. Search-augmented LLMs perform a single query with search integration. For ExpertQA, we evaluate three search-augmented models— claude-sonnet-4-5 , gemini-2.5-pro , and gpt-5.1 —each prompted to return structured JSON with a markdown answer and numbered citations. Provider-specific API configurations are listed in Appendix A . Model Provider Type Study URLs gemini-2.5-pro-deepresearch Google Deep research DRBench 11,309 openai-deepresearch OpenAI Deep research DRBench 4,121 gemini-2.5-pro-with-search Google Search-aug. DRBench 1,609 gemini-2.5-flash-with-search Google Search-aug. DRBench 2,433 gpt-4o-search-preview OpenAI Search-aug. DRBench 387 gpt-4o-mini-search-preview OpenAI Search-aug. DRBench 402 gpt-4.1 OpenAI Search-aug. DRBench 336 gpt-4.1-mini OpenAI Search-aug. DRBench 296 claude-3-7-sonnet-with-search Anthropic Search-aug. DRBench 1,735 claude-3-5-sonnet-with-search Anthropic Search-aug. DRBench 641 claude-sonnet-4-5 Anthropic Search-aug. ExpertQA 61,407 gemini-2.5-pro Google Search-aug. ExpertQA 22,878 gpt-5.1 OpenAI Search-aug. ExpertQA 83,736 Table 1: Model inventory. “Search-aug.” denotes search-augmented LLMs. URL counts reflect total unique URLs extracted from model outputs. 3.3 URL extraction and classification For DRBench, URLs are extracted from model-generated text via regex matching of https?:// patterns. For ExpertQA,

Read the complete text