Detecting RAG Advertisements Across Advertising Styles CCS: Information systems Content match advertising Conference: ; January 2026; webis.de ArXiv

Sebastian Heineking, Wilhelm Pertsch, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen, Martin Potthast · 2026

Read Detecting RAG Advertisements Across Advertising Styles CCS: Information systems Content match advertising Conference: ; January 2026; webis.de ArXiv by Sebastian Heineking, Wilhelm Pertsch, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen, Martin Potthast (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04925.txt.

Introduction

Sebastian Heineking and team study whether advertisements blended into retrieval-augmented making replies can be detected on its own. They build a taxonomy of advertising styles from explicitness and appeal type, simulate advertisers changing styles to evade finding, and test detector toughness under those shifts. Entity-spotting models that pinpoint ads exactly prove highly strong and largely robust to style changes. Lightweight models like random forests and SVMs, suited to low-resource devices, prove brittle instead, pointing to a need for leanness-oriented research.

Catalog record

CREATORSebastian Heineking, Wilhelm Pertsch, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen, Martin Potthast
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54757
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.04925

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Detecting RAG Advertisements Across Advertising Styles CCS: Information systems Content match advertising Conference: ; January 2026; webis.de ArXiv Sebastian Heineking, Wilhelm Pertsch, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen, Martin Potthast Abstract . Large language models (LLMs) enable a new form of advertising for retrieval-augmented generation (RAG) systems in which organic responses are blended with contextually relevant ads. The prospect of such ‘‘generated native ads’’ has sparked interest in whether they can be detected automatically. Existing datasets, however, do not reflect the diversity of advertising styles discussed in the marketing literature. In this paper, we (1) develop a taxonomy of advertising styles for LLMs, combining the style dimensions of explicitness and type of appeal, (2) simulate that advertisers may attempt to evade detection by changing their advertising style, and (3) evaluate a variety of ad-detection approaches with respect to their robustness under these changes. Expanding previous work on ad detection, we train models that use entity recognition to exactly locate an ad in an LLM response and find them to be both very effective at detecting responses with ads and largely robust to changes in the advertising style. Since ad blocking will be performed on low-resource end-user devices, we include lightweight models like random forests and SVMs in our evaluation. These models, however, are brittle under such changes, highlighting the need for further efficiency-oriented research for a practical approach to blocking of generated ads. 1 1 1 Code and data: https://anonymous.4open.science/r/detecting-rag-advertising-styles/ † † cc-license: by 1. Introduction Ads will soon be added to responses of large language models. Commercial chatbots such as ChatGPT started as free services, but vendors quickly sold subscriptions to Pro features to offset their vast expenses. Yet only 5 % of OpenAI’s users are paying subscribers ( 41 ) and others are likely not better off. All vendors are therefore working on integrating ads into their chatbots, as evidenced by press releases ( 27 ; 33 ; 36 ) , research ( 9 ; 13 ) , and industry news ( 1 ; 12 ) . An open question is what form these ads will take. Figure 1 . Examples of generated native ads in RAG responses using four advertising styles (one per cell). Note the explicitness of the ad snippets in the ”Overt” row, or the appeal to emotions in the “Emotional” column (marked in bold). The figure shows a two-by-two matrix of four RAG responses with advertisements. Above the matrix is a user query that asks ``Are there specific destinations that usually have last minute deals?''. The first dimension of the matrix is the level of overtness, distinguishing between covert advertisements and overt ones. The second dimension is the type of appeal, i.e., does the advertisement make a rational appeal for the advertised product or does it appeal to emotions? In each cell is a response containing an advertisement generated for based on the style that results from the combination of the two dimensions. In this paper, we study a new form of so-called “generated native advertising” (Figure 1 ), in which LLMs blend ads directly into otherwise organically generated responses ( 45 ) . Native advertising predates the digital age and has to date been associated primarily with journalistic media such as newspapers and magazines ( 2 ; 44 ) . Crucially, native ads are deliberately designed by publishers and advertisers to look and read like genuine news articles and to create the impression of editorial content. Even though regulations in many jurisdictions require every form of advertising to be disclosed to consumers ( 29 ) , such disclosures need not be prominent. And since native ads are more effective when readers remain unaware of them ( 34 ) , some publishers render disclosures as inconspicuous as possible, for example, through fine print, low-contrast labels, or unusual formatting. Native advertising has therefore been criticized as a threat to the credibility of editorial content ( 34 ) . As RAG systems and chatbots have become a popular alternative to conventional search engines ( 38 ) , and as users grow accustomed to and become less scrupulous of their ostensibly organic responses ( 18 ) , introducing native ads into them is a worrying prospect. We study the robustness of detectors for generated native ads in RAG responses. As research on generated ad detection is still in its infancy (Section 2 ), we observe several shortcomings in related work in terms of realism and grounding in marketing research. Our contributions are (1) a taxonomy of advertising styles for LLMs derived from marketing literature, which encompasses the explicitness of the promotion and the type of appeal as distinguishing characteristics. This enables (2) a “simulation” of how advertisers may attempt to evade native ad detection by changing its style, engaging in a cat-and-mouse game with developers of ad detectors. We compile a suite of test sets and use them to (3) evaluate how robust various types of ad detectors are under domain shifts caused by switching the ad-generating LLM and/or the advertising style. 2. Related Work Generated advertising and its blocking are new fields of research. Below, we summarize the related work on ad auction mechanisms for LLMs, the blending of ads and organic responses, and their detection and blocking. 2.1. Advertising in LLM Responses Advertising in LLM responses has not been deployed so far. Current research focuses primarily on how to design the auction mechanism for interested advertisers and, related to that, the placement of the ads in the responses. The suggested approaches can be divided into ones in which advertisers bid for a specific position in the response ( 13 ; 8 ) , and others in which the bids influence the distribution from which tokens are sampled ( 9 ; 39 ) . The former treat an ad as a unit placed somewhere in the response. This ensures at least some separation of organic text and advertisements. The latter blur the boundary between organic content and advertising, so that potentially the entire response is sampled from a distribution biased in the interest of advertisers. As an example of the first scenario, 13 propose an auction model for individual segments, e.g., paragraphs, in a RAG response. Similar to the sponsored links in a classic SERP, advertisements are selected based on query relevance, advertiser bid, and click probability, and added to the context of the LLM that generates the segment. In the framework proposed by 8 , advertisers do not bid on a specific segment but for increased prominence. The authors define prominence as an abstract concept that has a monotonically positive relation to user attention. Higher prominence can mean that an ad is represented with a larger number of words or that it is positioned more visibly in the response. An auction module receives bids, quality scores for the ads, and predicted click-through rates. Through its monotonic allocation function (higher bids result in higher prominence), the auction module outputs a “prominence allocation” and prices. The LLM is then instructed to create a text with appropriately prominent ads. 9 present a possible implementation of the second scenario: Advertisers can bid to influence the probability distribution used to sample tokens. Central to their approach are LLM

Read the complete text