The Attribution Crisis in LLM Search Results Estimating Ecosystem Exploitation
Ilan Strauss, Jangho Yang, Tim O’Reilly, Sruly Rosenblat, Isobel Moure · 2025
Read The Attribution Crisis in LLM Search Results Estimating Ecosystem Exploitation by Ilan Strauss, Jangho Yang, Tim O’Reilly, Sruly Rosenblat, Isobel Moure (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00838.txt.
Introduction
Ilan Strauss and co-authors study credit for web sources in search-enabled language models in a 2025 paper. These systems read web pages to answer questions, yet often cite few or none of them. The authors call that gap an attribution crisis. They measure it with about 14,000 real conversation logs from a public model arena.
They found three patterns. Many answers came with no web search at all. One major system gave no clickable source in 92 percent of answers. Another visited about ten useful pages per query but cited only three or four. Models differed widely in how much credit each extra page earned. The authors call for open records of searches and citations, so the web ecosystem gets fair credit.
Catalog record
| CREATOR | Ilan Strauss, Jangho Yang, Tim O’Reilly, Sruly Rosenblat, Isobel Moure |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00838
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Attribution Crisis in LLM Search Results Estimating Ecosystem Exploitation Ilan Strauss, Jangho Yang, Tim O’Reilly, Sruly Rosenblat, Isobel Moure Abstract Web-enabled LLMs frequently answer queries without crediting the web pages they consume, creating an “ attribution gap ” – the difference between relevant URLs read and those actually cited. Drawing on approximately 14,000 real-world LMArena conversation logs with search-enabled LLM systems, we document three exploitation patterns: 1) No Search : 34% of Google Gemini and 24% of OpenAI GPT-4o responses are generated without explicitly fetching any online content; 2) No citation : Gemini provides no clickable citation source in 92% of answers; 3) High-volume, low-credit : Perplexity’s Sonar visits approximately 10 relevant pages per query but cites only three to four. A negative binomial hurdle model shows that the average query answered by Gemini or Sonar leaves about 3 relevant websites uncited, whereas GPT-4o’s tiny uncited gap is best explained by its selective log disclosures rather than by better attribution. Citation efficiency – extra citations provided per additional relevant web page visited – varies widely across models, from 0.19 to 0.45 on identical queries, underscoring that retrieval design, not technical limits, shapes ecosystem impact. We recommend a transparent LLM search architecture based on standardized telemetry and full disclosure of search traces and citation logs. Keywords: LLM Search, citations, content monetization, ecosystem exploitation, large language models . 1 Introduction “The misery of being exploited by capitalists is nothing compared to the misery of not being exploited at all.” – Joan Robinson, Economic Philosophy , 1962. The rapid integration of large language models (LLMs) into information-seeking workflows has fundamentally transformed how users access and interact with content on the world wide web. Search-augmented LLMs, which combine generative capabilities with real-time web retrieval, promise to deliver more accurate, up-to-date, and comprehensive responses than traditional chatbots limited to just their training data [ 13 ] . But questions have been raised about source accuracy [ 25 ] , intellectual property rights, scraping websites without permission – so-called ‘access violations’ [ 4 , 22 ] , and what we term “ecosystem exploitation” – the gap between relevant web content consumed by the LLM when answering a query and those sources cited in the model’s output (its response) . 1 1 1 We use ‘consume’, ‘read’, and ‘search’ interchangeably for a relevant website visit logged by the LLM. This attribution gap has serious implications for the digital ecosystem on which AI’s ongoing utility depends. Content creators, publishers, and knowledge producers rely on appropriate attribution and licensing agreements when their information is used to answer queries. When LLMs systematically consume relevant content without adequate citation or remuneration, they undermine the incentive structures that support high-quality information production and threaten the economic viability of content creation at scale [ 18 ] . 2 2 2 For previous research empirical research on attribution practices across commercial LLM systems see Gao et al. 2023a , Yue et al. 2023 , Li et al. 2023 , Huang et al. 2025 , Fayyazi et al. 2025 , Profound 2025 . This systematic lack of attribution of the content sources consumed by LLMs during web search is a widely known issue [ 2 , 10 , 20 ] . 3 3 3 As recently as June 2025 the BBC threatened Perplexity’s search system with legal action for using its content during search [ 20 ] . Perplexity has set-up a revenue-sharing program with some publishers and increased citation of sources in response to accusations against it [ 2 ] . Yet major LLM vendors reveal little about how their retrieval-augmented generation (RAG) pipelines choose and ingest web content, and how they cite appropriate web sources [ 26 ] . Research questions : To what extent do search-augmented LLMs exploit the web — using content without citing it? And what distinguishes today’s best and worst attribution practices? Method and Data . Using an LMArena dataset of \approx 14,000 real-world answers to around 7,000 multi-turn queries taken between March and April 2025, we analyze the “attribution gap” exhibited by 11 LLM search-enabled models across three provider families: OpenAI (GPT-4o), Perplexity (Sonar), and Google (Gemini). We define the attribution gap as the number of relevant URLs visited by the LLM system when answering the query minus the number of URLs cited in the model’s output, providing a direct measure of ecosystem exploitation. To isolate this quantity, we remove all hallucinated citations (numbered citations provided in-text that do not have a corresponding source in the search log) and all ungrounded citations (URL citations provided that do not appear in the search logs but may link to a valid site). Our statistical model is a negative binomial hurdle model with bootstrapped confidence intervals. This allows us to quantify both when attribution gaps occur and how severe they are, while accounting for differences in query type (e.g. ‘Data Science’, ‘Current Affairs’, etc.). In a second regression we leverage the dataset’s head-to-head design, looking at differences between model citation behavior for the same query. Key findings : 1. Search-enabled LLM systems exploit by, surprisingly, not searching at all , relying instead on their pre-training data or simply not disclosing relevant search logs accurately. Despite the models being in search mode, 15.6% of LLM answers skipped web search entirely. This was highest for Google’s Gemini (34%), followed by OpenAI’s GPT-4o models (24%). 2. LLM search systems exploit by providing no citations (zero attribution) . 30% of answers provided no citations. This is driven less by query topic and more by model-specific behaviors. Gemini provided no citations for a striking 92% of queries, undermining claims that its impact on third-party traffic will be negligible. 3. Our zero-hurdle statistical model shows that, for a typical query, Google’s Gemini models and Perplexity’s Sonar models have sizable attribution gaps : at approximately 3 relevant websites visited but not cited. Perplexity exhibits much higher volume ecosystem exploitation, visiting \approx 10 relevant websites per query, but with a similar overall attribution gap to Gemini. 4. The full extent of ecosystem exploitation may be underestimated because models appear to selectively disclose which websites they visit, especially GPT-4o models. By design, GPT-4o models appear to have a near perfect correspondence between relevant websites drawn on and those cited, leading to a small attribution gap. 5. Refining a search-LLM’s RAG pipeline can almost double the number of citations it provides for each extra webpage it consumes . In “head-to-head” model regressions, comparing citation differences between model pairs for identical queries, we find that citation efficiency – the extra citations shown per additional website visited – ranges from 0.19 to 0.45. This indicates that retrieval design (reasoning modules, search context size, and geolocation), rather than technical limits, determines AI’s relationship with the world wide web’s ecosystem. The classical political economists defined exploitation as a category of production, whereby an owner-producer appropriates the difference between the cost of an input and its realized value contribution to output [ 28 ] . The classical economists focused exclusively on the labor input [ 7 , 11 ] , but we can easily extend this framework to the data inputs consumed by LLM models during inference when producing a relevant response (the output). Policy