Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems

Fatemeh Nazary, Yashar Deldjoo, Tommaso di Noia · 2025

Read Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems by Fatemeh Nazary, Yashar Deldjoo, Tommaso di Noia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11759.txt.

Introduction

Fatemeh Nazary, Yashar Deldjoo, and Tommaso di Noia wrote this paper in two thousand twenty-five. It shows how a poison attack can bend a recommender system that uses retrieval-augmented generation, where a language model draws on stored item data.

Their frame, Poison-RAG, tweaks item metadata such as tags and blurbs to push fringe items up and knock hits down. One plan tailors tags per item, the other stamps the same tags on all. Tests on the MovieLens data set, run blind, show the tailored plan boosts the sway by up to half, while the uniform plan can backfire and boost hits. Hit items prove easier to sway than fringe ones. Sound metadata care is the defense the authors urge.

Catalog record

CREATORFatemeh Nazary, Yashar Deldjoo, Tommaso di Noia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-11759

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems Fatemeh Nazary, Yashar Deldjoo, Tommaso di Noia Abstract This study presents Poison-RAG , a framework for adversarial data poisoning attacks targeting retrieval-augmented generation (RAG)-based recommender systems. Poison-RAG manipulates item metadata, such as tags and descriptions, to influence recommendation outcomes. Using item metadata generated through a large language model (LLM) and embeddings derived via the OpenAI API, we explore the impact of adversarial poisoning attacks on provider-side, where attacks are designed to promote long-tail items and demote popular ones. Two attack strategies are proposed: local modifications , which personalize tags for each item using BERT embeddings, and global modifications , applying uniform tags across the dataset. Experiments conducted on the MovieLens dataset in a black-box setting reveal that local strategies improve manipulation effectiveness by up to 50%, while global strategies risk boosting already popular items. The results indicate that popular items are more susceptible to attacks, whereas long-tail items are harder to manipulate. Approximately 70% of items lack tags, presenting a cold-start challenge; data augmentation and synthesis are proposed as potential defense mechanisms to enhance RAG-based systems’ resilience. The findings emphasize the need for robust metadata management to safeguard recommendation frameworks. Code and data are available at https://github.com/atenanaz/Poison-RAG . 1 Introduction Retrieval-augmented generation (RAG) [ 10 , 9 ] has emerged as a powerful paradigm in information retrieval and recommendation systems, combining the precision of traditional information retrieval (IR) with the generative capabilities of large language models (LLMs). Recent studies indicate that approximately 60% of applications utilizing LLMs now integrate RAG frameworks to address issues such as hallucinations and factual inaccuracies inherent in standalone LLMs [ 1 ] . As depicted in Fig. 1 , RAG systems typically comprise three stages: retrieval , augmentation , and generation . In the retrieval stage, the system accesses an external knowledge base—such as item metadata or user interaction histories—to identify pertinent information. The augmentation stage enriches the user query with the retrieved data, providing enhanced context. Finally, during the generation stage, the LLM produces responses that are contextually relevant and grounded in factual data, thereby mitigating the risk of generating hallucinations or inaccuracies. Figure 1 : Diagram of the RAG-based recommender system pipeline showing Retrieval, Augmentation, and Generation stages. Adversarial attacks poison item descriptions during Retrieval, aiming to promote unpopular items and demote popular ones in the final recommendations. Implementing RAG systems in real-world applications requires careful consideration of each stage to ensure optimal performance. In recommender systems (RS), various RAG applications have been developed based on differing retrieval and augmentation strategies [ 11 , 12 , 14 , 9 ] . For instance, a recommender system may retrieve candidate items using user-item interaction history in a collaborative manner or by leveraging item metadata (e.g., tags, genres, or reviews) to build textual user profiles . Each approach generates an initial ranked list, which is then refined during the augmentation and generation steps. This study focuses on text-based retrieval models within RAG systems for two primary reasons: (i) they create rich, semantic representations of user interests and item attributes, and (ii) they integrate seamlessly with LLMs. Text representations capture nuanced relationships and contextual details that traditional collaborative methods may overlook. Moreover, integrating text-based retrieval with LLMs leverages their advanced language understanding capabilities to extract meaningful embeddings from textual data. For example, if a user frequently engages with science fiction movies, the system can identify themes such as “futuristic technology” or “space exploration” in item descriptions, facilitating recommendations of thematically similar films. Utilizing LLM embeddings in retrieval helps bridge the semantic gap between user preferences and item descriptions. Despite the advantages of text-based retrieval-augmented RS, their reliance on textual data introduces potential security vulnerabilities [ 8 , 5 , 4 , 15 , 16 ] . The open-ended nature of text-based metadata makes it susceptible to data poisoning attacks, where adversaries manipulate descriptions, tags, or other textual attributes to influence recommendations. For example, adversarial attacks can exploit these vulnerabilities to bias the exposure of certain items or manipulate the rankings of user classes [ 3 ] . Overall, the trustworthiness of LLM-based systems in recommendations have raised concerns about provider fairness, biases, and reliable and explainability, which must be considered when deploying these models [ 6 , 7 ] . In the context of RAG, data poisoning attacks could specifically target the retrieval process, exploiting the system’s dependence on language-based representations to achieve undesirable outcomes, such as market manipulation or biased recommendations. For instance, tags—due to their lightweight and often user-generated nature—can be subtly altered in an adversarial manner, presenting opportunities for exploitation. The effectiveness of these manipulations depends on factors such as attack strategy, attacker knowledge (black-box vs. white-box), and system configuration [ 8 ] . In this work, we investigate two types of adversarial poisoning attacks on retrieval-augmented RS (cf. Section 2.2 ): • Local Adversarial Attacks : These attacks personalize tag modifications for individual items by identifying similar items across different popularity classes using BERT embeddings across the entire dataset; • Global Adversarial Attacks : These attacks apply a uniform pool of tags across the entire dataset; Our analysis reveals that only the local tag selection method effectively manipulates recommendation outcomes, likely because its focused approach preserves semantic relevance and enhances adversarial impact. Overall, our work introduces a novel framework for attacking textual RAG systems and examining their vulnerabilities in provider-oriented evaluations (exposure-based and exposure-relevance), offering new insights and paving the way for future research in this area. Contributions. The contributions of this research are as follows: • We investigate the security vulnerabilities of RAG-based recommender systems by exploring adversarial data poisoning attacks that manipulate textual item metadata. • We propose a novel methodology for selecting adversarial tags based on a combination of adversarial impact and semantic relevance, ensuring that the manipulated tags remain contextually appropriate and imperceptible to users. • We conduct extensive experiments using the MovieLens dataset to evaluate the effectiveness of our attack strategies, analyzing their impact on recommendation outcomes and providing insights into the robustness of RAG-based systems. 2 Poison-RAG: A Data Poisoning Attack Framework Against Retrieval Augmented Recommender Systems In this section, we introduce Poison-RAG , a data poisoning attack framework designed to manipulate item popularity in Retrieval-Augmented Generation (RAG)-based recommender systems. This framework focuses on designing attacks targeting the producer side , specifically by modifying item metadata—such as tags—to promote long-tail (unpopular) items and demote popular ones in recommendations. We define the notation and variables used throughout this section in Table 1 . Table 1 : Summary of Notation Symbol Description \mathcal{I} Set of all items \mathcal{U} Set of all users M_{i} Metadata of item i , including tags R(u) Set of items recommended to user u R^{\prime}(u) Recommendations to user u after attack \text{pop}(i) Popularity class of item i ( popular , mid-tail , long-tail ) \mathcal{I}_{L} Set of long-tail items \mathcal{I}_{P} Set of popular items \Gamma Set of adversarial tag modifications \mathbb{I}(\cdot) Indicator function (1 if condition is true, 0 otherwise) A(t) Adversarial impact of tag t s(t,i) Semantic relevance between tag t and item i \mathbf{e}_{t} Embedding vector of tag t \mathbf{e}_{i} Embedding vector of item i \epsilon Small constant to prevent division

Read the complete text