FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model PubID: pubid:

Md Jueal Mia, M. Hadi Amini · 2025

Read FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model PubID: pubid: by Md Jueal Mia, M. Hadi Amini (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05640.txt.

Introduction

Mia and Amini built a shared word model trainer in twenty twenty-five that guards firm data while many sites fine-tune one model. Shared training keeps data at home, but stray reads can leak it. Plain noise guards cost model skill, so they sought a leaner shield.

Their frame prunes weak links and locks the key tune facts with full code math. Small tune blocks cut both compute and talk loads for small sites. On two model sizes and four data sets it beat past frames on joint skill and speed. Cross-firm use looks in reach, they argue.

Catalog record

CREATORMd Jueal Mia, M. Hadi Amini
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-05640

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model PubID: pubid: Md Jueal Mia, M. Hadi Amini Abstract Federated Learning (FL) offers a decentralized framework for training and fine-tuning Large Language Models (LLMs) by leveraging computational resources across organizations while keeping sensitive data on local devices. It addresses privacy and security concerns while navigating challenges associated with the substantial computational demands of LLMs, which can be prohibitive for small and medium-sized organizations. FL supports the development of task-specific LLMs for cross-silo applications through fine-tuning but remains vulnerable to inference attacks, such as membership inference and gradient inversion, which threaten data privacy. Prior studies have utilized Differential Privacy (DP) in LLM fine-tuning, which, despite being effective at preserving privacy, can degrade model performance. To overcome these challenges, we propose a novel method, FedShield-LLM, that uses pruning with Fully Homomorphic Encryption (FHE) for Low-Rank Adaptation (LoRA) parameters, enabling secure computations on encrypted model updates while mitigating the attack surface by deactivating less important LoRA parameters. Furthermore, optimized federated algorithms for cross-silo environments enhance scalability and efficiency. Parameter-efficient fine-tuning techniques like LoRA substantially reduce computational and communication overhead, making FL feasible for resource-constrained clients. Experimental results show that the proposed method outperforms existing methods while maintaining robust privacy protection, enabling organizations to collaboratively train secure and efficient LLMs. The code and data are available at https://github.com/solidlabnetwork/fedshield-llm . I Introduction Large Language Models (LLMs) such as GPT brown2020language , PaLM chowdhery2023palm , and ChatGPT OpenAI2023 have revolutionized natural language processing by achieving state-of-the-art results across a wide array of language understanding and generation tasks. These foundation models, trained on massive public datasets, exhibit exceptional performance in domains like dialogue generation, summarization, and question answering. However, as LLMs are increasingly fine-tuned for specific domains such as healthcare and finance, they must incorporate sensitive or proprietary data, raising critical privacy and compliance concerns. Regulations like the GDPR and CCPA restrict data sharing and impose legal obligations on organizations handling private information baik2020data ; boyne2018data ; das2024security . Moreover, recent studies show that LLMs can memorize and leak training data carlini2021extracting , which presents significant risks when models are trained or fine-tuned on confidential inputs. This makes it imperative to develop solutions that preserve data locality and ensure privacy during domain adaptation. Training and fine-tuning LLMs also pose substantial computational challenges. Models like GPT-3 require hundreds of GPU-years and millions of dollars in compute resources LambdaLabs2020 , limiting accessibility to large tech firms. Even with the release of open-source LLMs like LLaMA touvron2023llama , resource bottlenecks remain, especially for smaller organizations or in distributed collaborative scenarios. Traditional full-model fine-tuning methods involve transmitting large model updates or datasets, which can be impractical in resource-constrained environments typically found in centralized training scenarios. Federated Learning (FL) mcmahan2017communication addresses these challenges by enabling decentralized training, where clients perform local updates on private data and only share model gradients or parameters with a central server. FL supports privacy by design and allows organizations to jointly train models without centralizing data. FL has been explored in resource-constrained and edge scenarios imteaj2021survey ; li2020federated , and recent works like OpenFedLLM r3_ye2024openfedllm and FederatedScope-LLM r2_kuang2024federatedscope have shown that FL can be adapted for LLM instruction tuning and domain-specific applications. However, standard FL remains vulnerable to efficiency and security issues. Transmitting full-model updates for large models is bandwidth-intensive and often infeasible. Additionally, model updates can leak private information via inference or gradient inversion attacks (GIA) zhu2019deep ; gu2022cs . To address the communication overhead, researchers have proposed parameter-efficient tuning methods such as Low-Rank Adaptation (LoRA) hu2022lora , which inserts low-rank matrices into transformer layers to reduce trainable parameters. LoRA enables clients to only update lightweight adapter layers instead of the full model, drastically reducing computation and communication costs. This makes LoRA highly compatible with federated fine-tuning frameworks wang2024flora . Nonetheless, privacy concerns persist even with reduced update sizes, as model updates can still be exploited by malicious servers or compromised clients. To ensure end-to-end privacy in federated LLM training, several advanced privacy-preserving techniques have been proposed. Differential Privacy (DP) dwork2014algorithmic introduces noise to model updates to provide statistical guarantees but often compromises model accuracy, especially in high-dimensional settings like LLMs liu2023differentially ; xu2024dp . Fully Homomorphic Encryption (FHE) cheon2017homomorphic enables computation over encrypted data, offering stronger confidentiality without requiring noise, though at the cost of increased computation. Secure Multi-Party Computation (MPC) bonawitz2017practical enables joint computation without revealing data but also faces scalability limitations. Meanwhile, model pruning wu2021adversarial can improve efficiency and mitigate attack surfaces by eliminating redundant parameters. Although most prior privacy-preserving LLM frameworks focus on inference-time security in centralized settings, they offer complementary insights relevant to secure model design. For instance, PrivacyAsst zhang2024privacyasst leverages HE to safeguard sensitive user inputs during tool-based LLM inference, illustrating how encrypted computation can be integrated into LLM workflows at runtime. Similarly, InferDPT tong2025inferdpt applies local DP to perturb user prompts submitted to black-box LLMs and reconstructs coherent outputs using a local extraction model. Building on these insights, we propose FedShield-LLM , a novel federated fine-tuning framework that combines FL, LoRA, FHE, and pruning to ensure scalable, secure, and regulation-compliant training of LLMs across decentralized private datasets. In this work, we detail the design of FedShield-LLM and demonstrate its effectiveness in achieving high performance with strong privacy guarantees. The main contributions of our study are listed below. • We propose and implement a novel federated fine-tuning mechanism, FedShield-LLM , for LLMs, which integrates FHE with model pruning. To the best of our knowledge, this is the first exploration of combining FHE with pruning in federated LLM fine-tuning to mitigate vulnerabilities during both the training and inference phases. Our approach enhances security against inference attacks in adversarial settings including an honest-but-curious server. • FedShield-LLM minimizes computational and memory overhead by updating only LoRA adapter layers instead of full model parameters. This design makes secure fine-tuning feasible in resource-constrained environments such as edge devices. • We validate the effectiveness of FedShield-LLM through extensive experiments using the base models meta-llama/Llama-2-7b-hf and meta-llama/Llama-2-13b-hf across four datasets: medalpaca/medical_meadow_medical_flashcards , vicgalle/alpaca-gpt4 , TIGER-Lab/MathInstruct , and FinGPT/fingpt-sentiment-train . Results demonstrate that our framework consistently outperforms existing methods in terms of both model performance and security across various domains. The remainder of this paper is structured as follows: Section II provides a comprehensive review of the related literature. Section III introduces the preliminaries essential for understanding the proposed method. Section IV details the proposed FedShield-LLM methodology, followed by Section V, which presents the experimental setup and results. Section VI offers further discussion, including limitations and implications. Finally, Section VII concludes the paper and outlines directions for future work. II Literature Review Recent advancements in machine learning have sparked significant interest in combining FL with LLMs to enable collaborative model development while preserving data privacy. Researchers have explored various approaches to make this integration practical and secure, including federated and distributed fine-tuning strategies, differential privacy techniques to safeguard individual data contributions, homomorphic encryption to ensure confidentiality during model aggregation, and secure multi-party computation to prevent information leakage during collaborative training. While each of these methods contributes valuable capabilities, they also come with limitations highlighting the need for a comprehensive framework like FedShield-LLM that unifies efficiency, privacy, and robustness.

Read the complete text