Distributed Retrieval-Augmented Generation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computing methodologies Natural language processing CCS: Computing method

Chenhao Xu, Longxiang Gao, Yuan Miao, Xi Zheng · 2025

Read Distributed Retrieval-Augmented Generation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computing methodologies Natural language processing CCS: Computing method by Chenhao Xu, Longxiang Gao, Yuan Miao, Xi Zheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00443.txt.

Introduction

Language models answer better when they can look things up, but a central library of knowledge raises privacy alarms. Chenhao Xu and co-authors distribute the library instead. Their DRAG framework spreads retrieval across a peer-to-peer network of devices. No single server holds the data, so owners keep control. A topic-aware random walk guides each query: the model extracts the query's topic, and the walk steers toward peers likely to know it.

Across three datasets and several models, the distributed system nearly matched central retrieval quality. It used half as many messages as naive flooding of the network. The design fits places like smart healthcare, where patient data cannot be pooled but knowledge still needs to travel. Privacy and competence, the work argues, can coexist.

Catalog record

CREATORChenhao Xu, Longxiang Gao, Yuan Miao, Xi Zheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-00443

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distributed Retrieval-Augmented Generation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2025; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Computing methodologies Natural language processing CCS: Computing methodologies Distributed artificial intelligence Chenhao Xu, Longxiang Gao, Yuan Miao, Xi Zheng Abstract . As large language models (LLMs) become increasingly adopted on edge devices, Retrieval-Augmented Generation (RAG) is gaining prominence as a solution to address factual deficiencies and hallucinations by integrating external knowledge. However, centralized RAG architectures face significant challenges in data privacy and scalability. For instance, smart healthcare services often rely on collecting sensitive patient data and building a centralized knowledge base to provide better diagnosis and treatment advice, while privacy concerns significantly impede this process. Besides, maintaining a comprehensive and continuously updated knowledge base is costly, particularly in response to regional epidemics and rapidly mutating viruses. To address these challenges, this paper introduces Distributed Retrieval-Augmented Generation (DRAG), a novel framework that improves data privacy by eliminating the need for a centralized knowledge base and restoring data control to owners. DRAG incorporates a Topic-Aware Random Walk (TARW) algorithm that leverages LLMs to extract query topics and facilitate targeted peer discovery within a peer-to-peer network, enabling efficient knowledge retrieval in decentralized environments. Extensive experiments across three diverse datasets and LLMs demonstrate that DRAG with TARW achieves near-centralized RAG performance by using half as many messages as flooding. The code is available at https://github.com/xuchenhao001/DRAG . 1. Introduction Large Language Models (LLMs) have revolutionized natural language processing, enabling a wide range of applications in edge computing ( zheng2024review ) . From voice assistants like Siri and Google Assistant to personalized recommendation systems, LLMs are integral to enhancing user experience on edge devices ( qu2025mobile ) . In healthcare, LLMs are being explored for symptom checking and medical advice ( wang2023pre ) . In driver assistance, LLMs can improve navigation and real-time traffic updates ( zhou2024vision ) . However, a significant challenge with LLMs is their propensity to generate factually incorrect information, a phenomenon known as hallucination ( ji2023survey ) , which hampers reliability in critical applications. To mitigate this, Retrieval Augmented Generation (RAG) ( gao2023retrieval ) has emerged as a promising approach. RAG combines LLM generative capabilities with external knowledge sources, retrieving relevant information before generating responses to ensure factual grounding. In edge computing, RAG can be particularly beneficial for providing up-to-date and accurate information directly to users on their devices ( qu2025mobile ) . However, centralized RAG architectures present considerable hurdles, particularly concerning data privacy and scalability ( fan2024survey ; zyskind2023don ; koga2024privacy ; muhamed2024cache ; zeng2024mitigating ; purwar2024evaluating ) . Consider a smart healthcare application: while leveraging LLMs to provide personalized treatment recommendations, these systems often require access to sensitive patient data to build and maintain a comprehensive knowledge base. However, privacy concerns and regulations usually restrict the sharing of such protected health information. Similarly, in autonomous driving systems, accessing the locations and traffic plans of other vehicles can improve navigation efficiency and safety. However, aggregating such data into a centralized server raises significant privacy concerns. In addition, the huge amount of data generated by regional epidemics, rapidly mutating viruses, and vehicle movements pose centralized RAG a scalability challenge, requiring efficient management of the computational demands for both retrieval and generation. Beyond these technical challenges, maintaining a continuously updated and comprehensive centralized knowledge base is inherently costly, requiring substantial resources for data acquisition, storage, and processing. To address the aforementioned challenges, this paper introduces Distributed Retrieval-Augmented Generation (DRAG), a novel framework that improves data privacy and scalability of RAG. DRAG eliminates the need for a central knowledge base or direct data sharing by enabling peer-to-peer (P2P) knowledge retrieval and generation across a distributed network. The decentralized design mitigates privacy risks by allowing users to have full control over their data while distributing the computational burden of knowledge base maintenance and retrieval. To facilitate efficient knowledge discovery within the DRAG network, a novel Topic-Aware Random Walk (TARW) algorithm is proposed. TARW leverages LLMs to extract query topics and guide peer discovery while utilizing a local cache to further enhance performance. The key contributions of this paper are summarized as follows: • A decentralized RAG framework tailored for edge computing environments, improving data privacy and scalability by eliminating direct data sharing and distributing the knowledge retrieval burden among peers. • A novel Topic-Aware Random Walk (TARW) algorithm that leverages LLMs to extract query topics and guide efficient peer discovery within the DRAG network through a local cache. • Extensive experiments across diverse datasets and LLMs validate that DRAG with TARW achieves performance comparable to centralized RAG with significantly reduced communication overhead than flooding. The rest of the paper is organized as follows: Section 2 reviews related work on Retrieval-Augmented Generation (RAG) scalability, privacy, and security. Section 3 details the proposed DRAG framework and the TARW algorithm. Section 4 presents the experimental evaluation. Finally, Section 5 concludes the paper. 2. Related Work This section reviews related work on DRAG, focusing on RAG scalability, privacy, and security. 2.1. RAG Scalability Retrieval-Augmented Generation (RAG) has emerged as a pivotal approach to mitigate hallucinations in LLMs ( chen2024benchmarking ) . Lewis et al. ( lewis2020retrieval ) pioneered this field with their seminal work introducing RAG as a framework that combines dense retrieval with sequence-to-sequence models for knowledge-intensive NLP tasks. Despite recent advancements in retrieval mechanisms ( guu2020retrieval ; karpukhin2020dense ; asai2023self ; edge2024local ; hu2024grag ; mavromatis2024gnn ; yu2025rankrag ) , centralized RAG architectures inherently face challenges regarding computational scalability and knowledge base maintenance. Some researchers have proposed query routing ( shnitzer2024large ; mu2025unsupervised ) and hierarchical retrieval ( zhang2024hierarchical ; wang2025archrag ) methods that primarily focus on efficiently assigning queries or aggregating results across multiple LLM candidates. While these approaches mitigate RAG scalability issues to some extent, they still overlook the fundamental scalability challenges associated with centralized knowledge base storage and processing within each LLM candidate. Recent research has also explored federated RAG architectures ( wang2024feb4rag ; addison2024c ) , which enable federated search and collaborative knowledge base development. However, their approaches still rely on a centralized request mediator, which inherently introduces vulnerability points and scalability bottlenecks. 2.2. RAG Privacy and Security Privacy and security concerns within RAG systems have become increasingly critical as these technologies are deployed in sensitive domains ( zeng2024good ) . For instance, studies have demonstrated the feasibility of privacy attacks that can extract confidential data from the RAG knowledge base ( jiang2024rag ) . Research has also revealed vulnerabilities of RAG to membership inference attacks ( anderson2024my ; liu2024mask ; li2024generating ) and poisoning attacks ( xue2024badrag ) . Consequently, various privacy-preserving techniques have been explored in the context of RAG, including secure multi-party computation ( zyskind2023don ) , homomorphic encryption ( zhao2024frag ) , watermarking ( lv2025rag ) , and differential privacy ( grislain2024rag ; koga2024privacy ; zeng2024mitigating ) . While these advancements represent significant progress, they typically rely on centralized knowledge bases or processing nodes. This inherent reliance introduces potential points of vulnerability and can compromise data sovereignty. By contrast, the DRAG framework proposed in this paper addresses these limitations by empowering users with control over their data. DRAG allows individuals to determine

Read the complete text