Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation

Zizhong Li, Haopeng Zhang, Jiawei Zhang · 2025

Read Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation by Zizhong Li, Haopeng Zhang, Jiawei Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03110.txt.

Introduction

Zizhong Li, Haopeng Zhang, and Jiawei Zhang show how a search backed answer system can be fooled, in a 2025 paper. Such systems fetch outside text to help a language model answer with fresh facts. That open door cuts both ways: bad text in the store can be fetched and steer the reply.

Their attack works word by word. A small open model writes poisoned passages, then tweaks them step by step so they rank high in search and sway the final answer. It works with full or no access to the target system. In tests it beat past attacks on both the search stage and the end reply. The team frames it as a warning for builders.

Catalog record

CREATORZizhong Li, Haopeng Zhang, Jiawei Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-03110

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation Zizhong Li, Haopeng Zhang, Jiawei Zhang Abstract While large language models (LLMs) have achieved remarkable success in providing trustworthy responses for knowledge-intensive tasks, they still face critical limitations such as hallucinations and outdated knowledge. To address these issues, the retrieval-augmented generation (RAG) framework enhances LLMs with access to external knowledge via a retriever, enabling more accurate and real-time outputs about the latest events. However, this integration brings new security vulnerabilities: the risk that malicious content in the external database can be retrieved and used to manipulate model outputs. Although prior work has explored attacks on RAG systems, existing approaches either rely heavily on access to the retriever or fail to jointly consider both retrieval and generation stages, limiting their effectiveness, particularly in black-box scenarios. To overcome these limitations, we propose Token-level Precise Attack on the RAG (TPARAG), a novel framework that targets both white-box and black-box RAG systems. TPARAG leverages a lightweight white-box LLM as an attacker to generate and iteratively optimize malicious passages at the token level, ensuring both retrievability and high attack success in generation. Extensive experiments on open-domain QA datasets demonstrate that TPARAG consistently outperforms previous approaches in retrieval-stage and end-to-end attack effectiveness. These results further reveal critical vulnerabilities in RAG pipelines and offer new insights into improving their robustness. 1 Introduction The rapid advancement of large language models (LLMs) has led to impressive performance across a broad range of NLP tasks Ouyang et al. 2022 ; Chang et al. 2024 ; Guo et al. 2025 , including but not limited to knowledge-intensive generation Singhal et al. 2025 ; Wang et al. 2023 ; Zhang et al. 2023b ; Zhang et al. 2023a . However, LLM-generated content still faces challenges such as hallucinations and outdated knowledge Liu et al. 2024a ; Ji et al. 2023 . To address these limitations, the Retrieval-Augmented Generation (RAG) framework was introduced and has been widely adopted Lewis et al. 2020 ; Jiang et al. 2023b ; Chen et al. 2024a . RAG combines two core components: (1) a retriever that searches for relevant information from an external knowledge base, and (2) a reader that generates more accurate and informative responses based on the retrieved passages. Figure 1: Comparison between the general RAG system (green background) and the RAG system attack (red background). The attacker replaces key information in the background knowledge to craft malicious passages, tricking the reader into generating an incorrect answer. Although RAG enhances the generation quality of LLMs, it also introduces new potential risks to the reader’s generation process, as the Figure 1 shows. Since the retriever gathers information from external knowledge sources, the system is vulnerable to retrieving harmful or misleading content, which can cause the reader to generate incorrect or unsafe responses Zeng et al. 2024 ; Jiang et al. 2024 . To exploit this weakness, recent studies have proposed attack methods that inject harmful information into the external database to manipulate the reader’s output Zou et al. 2024 ; Cheng et al. 2024 ; Xue et al. 2024 ; Cho et al. 2024 . However, existing RAG attack approaches still face some of the following limitations: 1) Lack of joint consideration for the retrieval and generation. Prior studies Zou et al. 2024 ; Cheng et al. 2024 often rely on teacher LLMs to generate malicious passages targeting the reader, but overlook whether the retriever can effectively retrieve these passages; 2) Overreliance on retriever models. For instance, Xue et al. 2024 depends heavily on access to the retriever to generate query-similar malicious passages, which limits applicability in realistic black-box RAG settings; and 3) Lack of targeted control over reader outputs. The crafted malicious passage by the existing approaches is relatively random and cannot intentionally guide the reader’s responses Cho et al. 2024 ; Chen et al. 2024b . To address the limitations of existing RAG attack methods, we propose Token-level Precise Attack on RAG (TPARAG) , a novel framework designed to target both black-box and white-box RAG systems. TPARAG leverages a lightweight white-box LLM as the attacker to first generate malicious passages and then optimize them at the token level, ensuring that the adversarial content can be retrieved by the retriever and effectively misleads the reader. Specifically, TPARAG operates in two stages: a generation attack stage and an optimization attack stage. In the generation stage, TPARAG records the top- k most probable tokens at each position as potential substitution token candidates. These token candidates are then systematically recombined during optimization to construct a pool of malicious passage candidates. Each passage candidate is further evaluated using two criteria to exploit vulnerabilities in both the retrieval and generation processes of the RAG pipeline: (1) the likelihood that the LLM attacker generates an incorrect answer given the passage, and (2) its textual similarity to the query. The most effective malicious passage will finally be selected and injected into the external knowledge base to execute a targeted attack. We conduct extensive experiments on multiple open-domain QA datasets using various lightweight white-box LLMs as attackers, under both black-box and white-box RAG settings. The results demonstrate that TPARAG significantly outperforms prior approaches in attacking RAG systems, both in retrieval and end-to-end performance. Moreover, we perform a series of quantitative analyses to identify key factors influencing attack success, critical vulnerabilities in RAG architectures and offering insights into improving their robustness. In summary, our contributions are as follows. • We propose TPARAG, a token-level precise attack framework that leverages a lightweight white-box LLM as an attacker to exploit vulnerabilities in both the retrieval and generation stages of RAG systems. • We conduct extensive experiments on multiple open-domain QA datasets, demonstrating the effectiveness of TPARAG in both black-box and white-box RAG settings. • We perform detailed quantitative analyses to identify key factors influencing token-level attacks and explore strategies for maximizing attack effectiveness with minimal computational resources in practical scenarios. 2 Related Work 2.1 Retrieval-augmented Generation Owing to the flexibility and effectiveness of the retriever model, it has become increasingly important to enhance LLM performance on knowledge-intensive tasks, driving the development of the RAG framework Lewis et al. 2020 ; Jiang et al. 2023b ; Gao et al. 2023 . By incorporating information retrieval, RAG equips language models with more accurate and up-to-date background knowledge, improving generation quality and addressing common issues such as hallucination Shuster et al. 2021 ; Béchard and Ayala 2024 . Leveraging these advantages, RAG has been widely adopted across various NLP tasks, including but not limited to open-domain question answering Siriwardhana et al. 2023 ; Setty et al. 2024 ; Wiratunga et al. 2024 , summarization Liu et al. 2024b ; Suresh et al. 2024 ; Edge et al. 2024 , and dialogue systems Vakayil et al. 2024 ; Wang et al. 2024b ; Kulkarni et al. 2024 . Figure 2: The framework of our proposed TPARAG. TPARAG first generates parent malicious passages through the generation attack stage (left). These passages are then recombined and refined during the optimization attack stage (right), producing optimized malicious passages that effectively mislead RAG’s answer. 2.2 Risks in the RAG System While RAG generally produces more reliable and accurate outputs than only

Read the complete text