Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning
Shanzhi Gu, Zhaoyang Qu, Ruotong Geng, Mingyang Geng, Shangwen Wang, Chuanfu Xu, Haotian Wang, Zhipeng Lin, Dezun Dong · 2025
Read Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning by Shanzhi Gu, Zhaoyang Qu, Ruotong Geng, Mingyang Geng, Shangwen Wang, Chuanfu Xu, Haotian Wang, Zhipeng Lin, Dezun Dong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05739.txt.
Introduction
Shanzhi Gu and co-authors test whether code models can be made to forget private data in a paper from 2025. Models that write code learn from huge code collections. Those collections can hold names, keys, and other secrets that the model may later leak.
The team built a benchmark with a fake forget set full of personal data. A retain set checked that coding skill survived the process. Three unlearning methods were tried on three open code models. Each method trades forgetting against skill in its own way. The study is the first broad check of unlearning for code models. Its benchmark gives others a shared way to test privacy fixes.
Catalog record
| CREATOR | Shanzhi Gu, Zhaoyang Qu, Ruotong Geng, Mingyang Geng, Shangwen Wang, Chuanfu Xu, Haotian Wang, Zhipeng Lin, Dezun Dong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-05739
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning Shanzhi Gu, Zhaoyang Qu, Ruotong Geng, Mingyang Geng, Shangwen Wang, Chuanfu Xu, Haotian Wang, Zhipeng Lin, Dezun Dong Abstract Large Language Models for Code (LLMs4Code) have achieved strong performance in code generation, but recent studies reveal that they may memorize and leak sensitive information contained in training data, posing serious privacy risks. To address this gap, this work presents the first comprehensive empirical study on applying machine unlearning to mitigate sensitive information leakage in LLMs4Code. We first construct a dedicated benchmark that includes: (i) a synthetic forget set containing diverse forms of personal information, and (ii) a retain set designed to evaluate whether code-generation capability is preserved after unlearning. Using this benchmark, we systematically assess three representative unlearning algorithms (GA, GA+GD, GA+KL) across three widely used open-source LLMs4Code models (AIXCoder-7B, CodeLlama-7B, CodeQwen-7B). Experimental results demonstrate that machine unlearning can substantially reduce direct memorization-based leakage: on average, the direct leak rate drops by more than 50% while retaining about over 91% of the original code-generation performance. Moreover, by analyzing post-unlearning outputs, we uncover a consistent shift from direct to indirect leakage, revealing an underexplored vulnerability that persists even when the target data has been successfully forgotten. Our findings show that machine unlearning is a feasible and effective solution for enhancing privacy protection in LLMs4Code, while also highlighting the need for future techniques capable of mitigating both direct and indirect leakage simultaneously. [style=chinese] [style=chinese] [style=chinese] [style=chinese] [style=chinese] [style=chinese] [style=chinese] [style=chinese] [style=chinese] † † highlights: Machine unlearning is a promising way to simultaneously mitigate the privacy concerns of LLMs4Code while maintaining their code generation capabilities at the same time. Specifically, unlearning can decrease the leak rate of AIXCoder by more than 50% while only bringing a negligible side effect to code generation. After unlearning, LLMs4Code learn to adopt diverse forms to prevent the leakage of sensitivities, in which the most popular one is to replace the sensitive fields with variable names and abbreviations. After unlearning, LLMs4Code become more likely to leak the privacy indirectly, which means they tend to leak the information that is not explicitly queried. This suggests that future works should also take into consideration the indirect privacy leakage for a more robust unlearning process. All code and data in this study are publicly available at https://doi.org/10.5281/zenodo.14729266 . keywords large language model ,machine unlearning ,LLMs4Code ,privacy leakage † † corresponding: Chuanfu Xu and Mingyang Geng are the corresponding authors. † † corresponding: Shanzhi Gu, Zhaoyang Qu and Ruotong Geng contribute equally to this article. † † titlenote: [email protected] ( S.Gu ); [email protected]( Z.Qu ) [email protected] ( R.Geng ); [email protected] ( M.Geng ); [email protected] ( S.Wang ); [email protected]( C.Xu ); [email protected] ( H.Wang ); [email protected] ( Z.Lin ); [email protected] ( D.Dong ). 1 Introduction Recently, the remarkable success of Large language models (LLMs) ( Brown, 2020 ; Qiu et al., 2025 ; Xu et al., 2025 ) in diverse natural language processing tasks has been tailored for the realm of software engineering, yielding several language models tailored specifically for code, termed as Large Language Models for Code (LLMs4Code), e.g., CodeLlama and Stable Code ( Xu et al., 2022 ) . With an extensive pre-training process on programming language datasets, such models have demonstrated proficiency with prominent performance on code-related tasks ( Geng et al., 2024 ; Deng et al., 2024 ; Qin et al., 2024 ) . For instance, Geng et al. (2024) has demonstrated the capability of LLMs4Code in producing code summarizations that are not only of superior quality but also cater to the varied requirements of human programmers through in-context learning ( Geng et al., 2024 ) . As a double-edged sword of LLMs4Code, the risk of sensitive information, including Personally Identifiable Information (PII), private data, or confidential secrets, has already been highlighted by recent studies ( Yang et al., 2024b ; Jahanshahi and Mockus, 2025 ) . From the perspective of model attacking ( Yao et al., 2024 ; Dong et al., 2024 ) , empirical evidence reveals that specific prompts can result in the leakage of corresponding sensitive information ( Huang et al., 2024 ; Carlini et al., 2021 ) . More formally, such risk of privacy disclosures during the utilization of LLMs4Code is commonly termed as the memorization problem ( Al-Kaswan and Izadi, 2023 ; Lukas et al., 2023 ) . Since the memorization problem commonly exists in a wide range of code-relevant tasks, e.g., code generation ( Svyatkovskiy et al., 2020 ; Wang et al., 2023a ) , and yields an inevitable risk to developers in their daily development activities, we argue that: In the era of LLMs4Code, it is of significant importance to effectively address the potential leakage of sensitive data for upholding robust user privacy measures and sustaining trust in the deployment. However, to the best of our knowledge, concurrent work rarely provides solutions to such an important but challenging problem, offering only empirical findings on the memorization problem ( Leybzon and Kervadec, 2024 ; Kiyomaru et al., 2024 ) . We also note that one potential approach involves incorporating a dedicated data cleaning phase during pre-processing of the training data, which yields inflexibility and unscalability due to the necessity of substantial engineering efforts to devise appropriate rules and heuristics. Fortunately, Machine Unlearning (MU) has emerged as a technique striving to assist the target model in “forgetting” the data points from the initial training set, offering a lightweight but effective approach to aid in protecting sensitive information from LLMs ( Liu et al., 2024 ; Nguyen et al., 2022 ) . To be specific, MU proposes to resemble an “auxiliary” sanitized dataset (devoid of sensitive information), thereby potentially saving considerable development costs compared to retraining the model from scratch. Following such an intuition, a number of studies have verified the promising effectiveness of MU on making LLMs forget specific contents that they met during training ( Chen and Yang, 2023 ; Chundawat et al., 2023 ) . Nonetheless, we also note that the current literature lacks a comprehensive understanding regarding the strengths and weaknesses of existing MU techniques within the context of LLMs4Code, including their effectiveness on mitigating the privacy leakage during the code generation process, and how the state-of-the-art MU techniques will perform on LLMs4Code. While several prior works (e.g., Chen and Yang (2023) ; Chundawat et al. (2023) ) have explored machine unlearning techniques on general-purpose LLMs, applying such methods directly to LLMs4Code is non-trivial. Code generation models differ significantly from natural language models in that code is more repetitive, memorization-prone, and functionally precise. These properties make unlearning in LLMs4Code both technically more challenging and practically more consequential. Furthermore, the evaluation of unlearning effectiveness in general LLMs often relies on classification accuracy or perplexity, which do not apply to code generation tasks where pass@1 and functional correctness are the primary metrics. Our work addresses this gap by systematically evaluating unlearning strategies on LLMs4Code using both privacy leakage and code generation metrics. To bridge this gap, this paper contributes an extensive empirical study on MU-inspired protection of the sensitive data leakage from LLMs4Code, yielding the correctness of their generated code at the same time. With the aid of the state-of-the-art GPT-4 and well-established code