Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources
Mowafak Allaham, Nicholas Diakopoulos · 2026
Read Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources by Mowafak Allaham, Nicholas Diakopoulos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23684.txt.
Introduction
Allaham and Diakopoulos audit search citations for AI-generated sources. Generative search engines answer questions by synthesizing web information, but AI content now floods the web, and engines may still cite synthetic sources. Treating synthetic sources as equal to trusted ones puts users at risk. The audit covers ChatGPT, Copilot, Gemini, and Perplexity over 712 queries on politics, health, and environment. About 16 percent of cited sources prove AI-generated across all four engines, with few domains cited repeatedly alongside many rarely cited ones.
Catalog record
| CREATOR | Mowafak Allaham, Nicholas Diakopoulos |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78941 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.23684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources Mowafak Allaham, Nicholas Diakopoulos Abstract The growing accessibility of Large Language Models via conversational interfaces capable of responding to users’ questions by drawing on, synthesizing, and citing information from the web (i.e., Generative Search Engines) has simplified the information-seeking process for users. However, with the proliferation of AI-generated content on the web, it is unclear whether these engines can reliably omit citing synthetic sources (i.e., AI-generated sources). Should these engines be unable to do so, this puts users at risk of harm by treating information from AI-generated sources synthesized in responses of generative search engines as equivalent to information from authoritative or official sources. In a step towards identifying whether AI-generated sources are being cited by these engines, this work presents an audit of four generative search engines (ChatGPT, Copilot, Gemini, Perplexity) using a total of 712 real-world human-generated queries spanning domains of public importance: politics, health, and the environment. Our findings show evidence of AI-generated sources being cited across all four generative search engines ( ∼ \sim 16% of cited sources) and identifies key source web domains these sources belong to that are frequently cited across these engines and topics. In addition, we observed that generative search engines include a somewhat narrow set of repeatedly cited domains while predominantly surfacing a large number of minimally cited domains in responses to users’ queries. These findings contribute to the growing body of work on assessing the risks of generative search engines with the objective of increasing public awareness of their limitations and encouraging appropriate measures to improve information quality and governance of these systems. Introduction People turn to search engines (e.g., Google or Bing) to gather and evaluate information, form opinions, and make decisions on a wide range of topics they find important on the personal and societal levels ( 49 ; 16 ; 20 ; 43 ; 37 ) . However, the growing accessibility of search-enabled large language models (LLMs) via conversational interfaces capable of responding to users’ queries by drawing on and synthesizing information from the web is transforming how people seek information online. In this work we refer to these tools, which include popular search-enabled LLMs such as ChatGPT or Gemini, as generative search engines (GSE). Generative search engines simplify the search process for users by shifting their attention away from clicking and visiting multiple links on a traditional search results page toward consuming a single synthesized response substantiated with in-text citation to various sources – a source in this context refers to a unique web URL that represents an item online (e.g., blog post, an article, image, or video) that is being cited by generative search engines. Experts and journalists have expressed growing concerns about the potential of generative search engines in relying more on AI-generated sources as a means to compensate for gaps in access to authoritative sources of information and to fill information vacuums (i.e., “data voids”) when responding to users’ queries ( 53 ; 4 ; 62 ; 6 ) . While the epistemic quality of AI-generated content could be a contentious point of debate, as not all AI-generated content is necessarily low quality ( 46 ) , the AI-tools used for generating such content (e.g., large language models) have their biases ( 71 ; 19 ) and are susceptible to generating hallucinations and inaccurate information ( 8 ; 14 ; 36 ) . This, in turn, may affect the information quality of the output produced by these tools. In addition, the sourcing mechanisms of generative search engines for information online and the choices these engines make at the system and information levels as part of generating the final answer, such as when to include sources in responses, what sources to include or exclude, and the quality and credibility of these sources, illustrate the opaqueness of these engines, which creates additional opportunities for AI-generated sources that may include false or misleading information to compete with accurate ones ( 22 ; 50 ; 73 ; 44 ) . This ambiguity in information sourcing mechanism exposes responses of generative search engines to potential manipulation for personal or corporate gains (e.g., brands promoting AI-generated content onlie to rank themselves in first place in “best X” lists) ( 3 ) or get exploited by bad actors, through knowledge injection or poisoning attacks ( 57 ; 5 ; 14 ) , to amplify sources with low-quality AI-generated content in the pool of sources on which generative search engines rely on, potentially degrading the quality of generated responses. Therefore, if the source content cited and used by generative search engines as part of the response generation process is AI-generated, given the aforementioned issues associated with this type of content, users may be at risk of unknowingly consuming low-quality information. This is particularly consequential for topics of individual (e.g. health) or societal (e.g. politics, environment) importance, which could misinform beliefs and actions or influence public opinion and upset political processes ( 33 ; 65 ) . To better characterize the nature and extent of this problem, it is imperative as a first step to identify whether or not generative search engines cite sources with AI-generated content. To this end, this research undertakes an audit of the sources cited by four popular generative search engines, which include the search-enabled Chatbot interfaces of ChatGPT, Copilot, Gemini, and Perplexity ( 63 ) . We first identify the range of source web domains (i.e., web domains that sources belong to) these engines rely on to respond to users’ queries, tabulate their frequencies, whether or not the source content corresponding to cited sources from these web domains is AI-generated, and then examine the prominence of such content among cited sources. To this end, we pursue the following two research questions: (1) what are the prevalent source web domains generative search engines cite sources from when responding to users’ queries? and (2) how prevalent is AI-generated sources among those cited sources? We address these questions by auditing answer engines through a socio-technical lens ( 56 ; 40 ; 2 ) that extends beyond assessing the technical artifacts of AI systems to reflect real-world users’ queries spanning three topics of public importance: politics, health, and the environment ( 33 ; 65 ) Using a closely curated sample of 712 English language queries focusing on general knowledge questions spanning the topics of politics, health, and the environment, we audit the cited sources of the aforementioned four generative search engines for evidence of AI-generated sources by first submitting each query in our sample through the user interface (UI) of each engine, then (if present) scraping the cited sources and their respective content in each generated response, and finally using an AI-detection tool to identify whether the source content in these sources is AI-generated or not. To assess whether source content from multiple cited sources is AI-generated, we use Pangram ( 17 ) , an AI-generated-text detection tool, and observe a considerable share of source contents of cited sources (approximately 16% of unique cited sources across all providers) on GSE to be AI-generated, with responses to environment-related queries exhibiting the highest proportion of cited source content that is AI-generated.