ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation
Xingyu Lyu, Jianfeng He, Ning Wang, Yidan Hu, Tao Li, Danjue Chen, Shixiong Li, Yimin Chen · 2026
Read ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation by Xingyu Lyu, Jianfeng He, Ning Wang, Yidan Hu, Tao Li, Danjue Chen, Shixiong Li, Yimin Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18762.txt.
Introduction
Xingyu Lyu and seven co-authors introduce ALDEN, a 2026 attack pulling private data from retrieval-augmented generation systems. Foes embed hostile commands in queries to pull private data, but past attacks show low yields and limited practical effect. ALDEN pairs active learning with a decay-based dynamic algorithm estimating the knowledge base topic distribution. Active learning diversifies hostile queries to lift yields, guiding query generation. Together, the two beat state-of-the-art methods across broad evaluations, substantially improving results.
Catalog record
| CREATOR | Xingyu Lyu, Jianfeng He, Ning Wang, Yidan Hu, Tao Li, Danjue Chen, Shixiong Li, Yimin Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67289 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18762
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation Xingyu Lyu, Jianfeng He, Ning Wang, Yidan Hu, Tao Li, Danjue Chen, Shixiong Li, Yimin Chen Abstract Retrieval-Augmented Generation (RAG) is widely used to augment large language models with external knowledge retrieval to improve reliability and generalization. However, recent studies have shown that RAG systems remain vulnerable to data extraction attacks, where adversaries can extract private data by embedding malicious commands into user queries. Despite their feasibility, existing attacks typically suffer from low data extraction rates and limited practical effectiveness. Here, we propose ALDEN, a novel attack that effectively and efficiently extracts private data from RAGs. First, we employ active learning to diversify malicious queries and improve data extraction rates. Second, we observe that the data distribution of the underlying knowledge base provides valuable guidance for query generation and introduce a decay-based dynamic algorithm to estimate the corresponding topic distribution. By combining them together, we demonstrate that ALDEN substantially outperforms state-of-the-art methods through comprehensive evaluations. 2 2 footnotetext: Jianfeng He completed this work while at Virginia Tech. He is now at Amazon. 1 Introduction Large Language Models (LLMs) have shown impressive capabilities in various applications. However, their reliability is often questioned due to issues such as hallucinations in which LLMs generate false or inaccurate information and lacking access to the latest data. Such concerns escalate when deploying LLMs for sensitive or high-stakes sectors like the medical and financial industries 14 ; 25 ; 23 . To address these limitations, Retrieval-Augmented Generation (RAG) has emerged as a practical solution to improve LLM reliability by incorporating externally retrieved knowledge into the input context 39 ; 26 . By keeping private data local rather than embedding it into the model, RAG has been widely adopted in domains such as customer support, healthcare, legal, and finance 32 ; 2 ; 17 ; 46 . The global RAG market was valued at approximately $1.24 billion in 2024 and is projected to reach $67.42 billion by 2034 35 . Despite these benefits, it remains unclear whether RAG provides sufficient privacy protection for the underlying knowledge base. The integration of private data into the LLM generation pipeline introduces new privacy risks. Prior work 45 ; 36 ; 8 shows that attackers can exploit prompt injection to induce LLMs to reveal retrieved private content, leading to automated attacks in 21 ; 11 . However, existing attacks tend to only generate queries of similar topics rather than queries of sufficiently different topics. As a result, they can only retrieve data of a narrow range, thus limiting the attack performance. What’s worse, they only showed feasibility without providing the upper bounds of proposed attacks. In this paper, we propose ALDEN, a query-based attack on RAG systems that efficiently extracts private data from their knowledge bases. To overcome the limited coverage of existing attacks, ALDEN diversifies malicious queries via active learning, thus enabling retrieval from broader topics. Furthermore, broader introduces a distribution-aware strategy that dynamically estimates the topic distribution of the victim knowledge base, which guides query generation and helps us to characterize the upper bound of ALDEN through an oracle setting. Figure 1 illustrates a case in which ALDEN retrieves information like name, city, clinic, disgnosis, etc, which is not supposed to be out there. We evaluate ALDEN under comprehensive settings and confirm that it significantly outperforms state-of-the-art attacks in private data extraction. Our contributions are as follows. • We propose a novel data extraction attack on RAG that is the first to incorporate active learning and distribution estimation together for enhancing attack performance. • We are the first to identify the importance of data distribution on data extraction and design a complete set of algorithms for generating malicious queries so that they are efficient, diverse, adaptive, and conditioned by historic queries. • We comprehensively evaluate ALDEN on a practical RAG system using three real-world datasets, three LLMs, and five baselines under both targeted and untargeted settings, confirming that it significantly outperforms state-of-the-art attacks. We also conduct extensive ablation studies and robustness evaluations against defenses. Notable, we are the first to show that ALDEN’s performance is close to the oracle test. Figure 1: Case study of applying ALDEN on a real-world clinic RAG. 2 System and Adversary Model Figure 2: Workflow of ALDEN. 2.1 System Model We illustrate the workflow of a typical RAG system in Figure 2 . An RAG system comprises a retriever R_{D} , an indexed knowledge base \mathbf{K} , and an LLM for response generation (referred to as LLM-RAG). Mathematically, given query q , the retriever obtains the top- k chunks [c_{1},c_{2},\cdots,c_{k}] from the knowledge base \mathbf{K} , i.e., [c_{1},c_{2},\cdots,c_{k}]=R_{D}(q,\mathbf{K}) . After that, the retriever feeds c_{1}\oplus c_{2}\oplus\cdots\oplus c_{k}\oplus q to LLM-RAG. Finally, LLM-RAG outputs r=\mathcal{G}_{\text{LLM-RAG}}\bigl(c_{1}\oplus c_{2}\oplus\cdots\oplus c_{k}\oplus q) to the user as the response of q . 2.2 Adversary Model We consider a black-box attacker aiming to extract private data from the knowledge base of a RAG system, i.e., to obtain as many chunks from \mathbf{K} as possible. By black-box, we mean that the attacker can only interact with the RAG system through APIs, without access to the internal. Assume that the attacker can use query q to obtain r and apply our proposed ALDEN attack to obtain a set of data chunks \mathbf{C} from r . It follows that when using a query sequence \mathbf{Q}=[q_{1},q_{2},\cdots,q_{n}] , the attacker can obtain \mathbf{R}=[r_{1},r_{2},\cdots,r_{n}] and \mathcal{C}=[\mathbf{C}_{1},\mathbf{C}_{2},\cdots,\mathbf{C}_{n}] , correspondingly. The final set of recovered chunks by the attacker is thus \mathbf{\Omega}=\mathbf{C}_{1}\cup\mathbf{C}_{2}\cup\cdots\cup\mathbf{C}_{n} . Attack goal. The attacker’s goal is to extract as many sensitive chunks from \mathbf{K} as possible, i.e., more sensitive chunks in \mathbf{\Omega} . Following prior attacks 45 ; 11 , we consider the two scenarios. (1) Targeted attack: Here we assume that the attacker has prior knowledge of the data domain of the victim RAG system and aims to extract chunks of targeted types. Similar to 21 ; 45 , we assume that the attacker targets at Personally Identifiable Information (PII) such as phone numbers, emails, etc. (2) Untargeted attack: Here we assume that the attacker has no prior knowledge of the data domain and aims to extract as many unique chunks from \mathbf{K} as possible. 3 Design of ALDEN Figure 3: Ground-truth and estimated distributions of knowledge base \mathbf{K} . 3.1 Intuitions and Overview. Intuitions. As mentioned above, our ALDEN attack is to extract as many (sensitive) chunks from the knowledge base \mathbf{K} of an RAG system as possible. Earlier works mainly demonstrated their feasibility but suffered from low extraction rates and uncertainty about their performance. In comparison, our first intuition is to adopt active learning strategies for generating new attack queries which are more likely to help return new chunks from \mathbf{K} . We explore different active learning strategies and adopt k-center in our design. Our second intuition is based on revisiting the root of such private data extraction attacks on RAG systems. Particularly, we ask the following specific question: what can be the impact of knowing the data distribution of chunks in \mathbf{K} on generating attack queries for private data extraction? The answer is clear; chunk data distribution in \mathbf{K} would certainly guide the generation of proper attack