A Survey of Machine Unlearning in Large Language Models: Methods, Challenges and Future Directions

Qing Li, Jiahui Geng, Herbert Woisetschläger, Zongxiong Chen, Fengyu Cai, Yuxia Wang, Preslav Nakov, Hans Arno Jacobsen, Fakhri Karray · 2025

Read A Survey of Machine Unlearning in Large Language Models: Methods, Challenges and Future Directions by Qing Li, Jiahui Geng, Herbert Woisetschläger, Zongxiong Chen, Fengyu Cai, Yuxia Wang, Preslav Nakov, Hans Arno Jacobsen, Fakhri Karray (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01854.txt.

Introduction

Jiahui Geng and co-authors survey machine unlearning for large language models, the craft of making a model forget. Retraining from scratch to drop tainted data, such as private or illegal text, costs too much, so researchers teach trained models to shed specific influences while keeping their general skill. No full map of this young field existed.

The survey opens with definitions and the main paradigms of unlearning. It then sorts the known methods into a taxonomy, weighing each family's strengths and limits. Metrics and benchmarks for judging whether forgetting worked get their own structured review. The authors close with open challenges and promising directions, from sharper tests of what remains in the model to methods that scale with ever larger systems.

Catalog record

CREATORQing Li, Jiahui Geng, Herbert Woisetschläger, Zongxiong Chen, Fengyu Cai, Yuxia Wang, Preslav Nakov, Hans Arno Jacobsen, Fakhri Karray
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01854

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Survey of Machine Unlearning in Large Language Models: Methods, Challenges and Future Directions Qing Li, Jiahui Geng, Herbert Woisetschläger, Zongxiong Chen, Fengyu Cai, Yuxia Wang, Preslav Nakov, Hans Arno Jacobsen, Fakhri Karray Abstract This study investigates the machine unlearning techniques within the context of large language models (LLMs), referred to as LLM unlearning . LLM unlearning offers a principled approach to removing the influence of undesirable data (e.g., sensitive or illegal information) from LLMs, while preserving their overall utility without requiring full retraining. Despite growing research interest, there is no comprehensive survey that systematically organizes existing work and distills key insights; here, we aim to bridge this gap. We begin by introducing the definition and the paradigms of LLM unlearning, followed by a comprehensive taxonomy of existing unlearning studies. Next, we categorize current unlearning approaches, summarizing their strengths and limitations. Additionally, we review evaluation measures and benchmarks, providing a structured overview of current assessment methodologies. Finally, we outline promising directions for future research, highlighting key challenges and opportunities in the field. 1 Introduction Figure 1: Overview of LLM Unlearning. LLM unlearning focuses on removing specific data (forget set) while minimizing the impact on related knowledge (retain set) and general world knowledge. Related Work Date Taxonomy # Methodologies Modality Adv Evaluation Si et al. 2023 December 2023 Yes 3 Unimodal 0 Xu 2024 April 2024 Yes 3 Unimodal 0 Liu et al. 2025 June 2024 No 6 Unimodal,Multimodal 5 Liu et al. 2024c July 2024 No 5 Unimodal,Multimodal 0 Ours May 2025 Yes 10 Unimodal,Multimodal 7 Table 1: Comparison of different surveys on the LLM Unlearning. Adv Evaluation refers to Adversarial Evaluation. The widespread adoption of large language models (LLMs) has brought significant challenges, particularly concerning user data privacy, copyright protection, and alignment with societal values. During training, these models can inadvertently memorize sensitive information, such as personally identifiable data or copyrighted materials Li et al. 2024b ; Li et al. 2024c ; Zhang et al. 2024b ; Yao et al. 2024 . In addition to privacy and copyright issues, some training data may embed content that conflicts with contemporary social norms, such as discriminatory language based on race, ethnicity, etc Li et al. 2025a . These biases often manifest as harmful stereotypes, undermining the fairness and inclusivity of AI systems. Addressing these concerns is not only a societal imperative but also a regulatory requirement under privacy laws such as the General Data Protection Regulation (GDPR, Council of the European Union 2016 ) and the EU Artificial Intelligence Act (EU AI Act, Council of the European Union 2024 ). These laws mandate the “right to be forgotten" and require mechanisms to delete specific data upon request. To address these challenges, the field of LLM unlearning has emerged, focusing on removing specific information or behaviors from models while preserving their overall performance. However, LLM unlearning faces significant technical challenges. One of the most pressing issues is the prohibitively high cost of retraining. Traditionally, addressing harmful or unwanted data required retraining the model from scratch after excluding problematic data from the training set Jang et al. 2023 . This is impractical for LLMs due to the immense time and computational resources required Li et al. 2024b . Moreover, the frequent unlearning requests that arise in deployed models highlight the need for more efficient unlearning techniques. The complexity of LLMs, with their millions, or even hundreds of billions of parameters, further complicates the task of removing specific information without causing unintended side effects, such as performance degradation or catastrophic forgetting Zhang et al. 2024a . Table 1 provides a comparative overview of existing survey papers on LLM unlearning alongside our work. Most prior surveys were published before July 2024 and therefore do not capture recent advancements in the field. Moreover, these surveys either lack a systematic taxonomy or are limited to unimodal unlearning methods. Here we aim to bridge this gap. In particular, we offer a thorough overview of the field, including various unlearning and evaluation methods, and we make the following contribution • We formalize the LLM unlearning paradigms and propose a comprehensive taxonomy to categorize the existing approaches. This taxonomy not only offers a structured understanding of the research landscape, but also helps researchers identify their areas of interest. • We systematically review the existing methods, analyzing their strengths and weaknesses. We further examine the existing evaluation measures and benchmarks, highlighting the challenges of balancing utility preservation with forgetting quality. • We discuss future research opportunities for LLM unlearning, including extending techniques to multimodal models and addressing complex real-world unlearning requests. These avenues aim to advance the field and address emerging challenges. 2 Preliminaries and Taxonomy 2.1 Problem Definition The objective of LLM unlearning is to selectively remove the influence of specific information while maintaining the model’s overall utility for other tasks. The optimization objective of the model parameters \theta can be expressed as follows: \min_{\theta}\mathcal{L}(\theta)=\min_{\theta}\{-\mathcal{L}_{f}(\theta)+\lambda\mathcal{L}_{r}(\theta)\} (1) Here, the forget loss \mathcal{L}_{f}(\theta) quantifies the model prediction error on the forget set \mathcal{D}_{f} , while the retain loss \mathcal{L}_{r}(\theta) ensures the preservation of the model’s utility on the retain set \mathcal{D}_{r} . The regularization parameter \lambda\geq 0 controls the trade-off between effectively forgetting undesired information and preserving the model’s utility. 2.2 LLM Unlearning Paradigms LLM unlearning follows two main paradigms. The fine-tuning-then-unlearning paradigm focuses on eliminating knowledge introduced during fine-tuning . As illustrated in Figure 1 , this paradigm typically leverages synthetic data (e.g., TOFU Maini et al. 2024 and FIUBench Ma et al. 2024 ) and partitions a task-specific dataset into a forget set \mathcal{D}_{f} and a retain set \mathcal{D}_{r} to highlight the unlearning precision. The original model \theta_{ori} is first obtained by fine-tuning a pretrained model \theta_{pre} on the task-specific data to encode the target knowledge. Unlearning techniques are then applied to reduce the model’s reliance on the forget set, resulting in the unlearned model \theta_{unl} . The direct-unlearning paradigm focuses on eliminating knowledge acquired during the pretraining stage of \theta_{ori} , assuming the target knowledge originates from multiple points within the pretraining dataset. The forget and retain sets are typically sampled from pretraining corpora Yao et al. 2024 or publicly available datasets such as Wiki Jin et al. 2024 . This paradigm is also extensively applied in safety alignment tasks, where it aims to eradicate hazardous knowledge and to mitigate risks such as misuse or jailbreak attacks Li et al. 2024b ; Zhang et al. 2024b . {forest} Figure 2: The taxonomy of machine unlearning in LLMs. 2.3 Taxonomy We present a comprehensive taxonomy of LLM unlearning in Figure 2 , outlining existing research from the perspectives of methods, evaluation measures, benchmarks, and applications. Existing methods can be categorized into four types: direct fine-tuning, localized parameter modification, leveraging auxiliary models, and input/output-based unlearning. Forgetting quality and utility preservation are critical measures for evaluating unlearning algorithms, particularly given recent discussions on whether knowledge is robustly forgotten or remains susceptible to adversarial recovery. This is often assessed through input- or logit-based evaluation, as well as model intervention techniques. Additionally, we review commonly used unimodal and multimodal benchmarks. 3 LLM Unlearning Methods 3.1 Direct Fine-Tuning Relabeling-based fine-tuning first replaces the original responses with generic or neutral substitutes, as present above. The LLM

Read the complete text