Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond

Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, Sijia Liu · 2025

Read Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond by Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, Sijia Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05374.txt.

Introduction

Chongyu Fan and co-authors work to make language model unlearning stick in a 2025 paper. Unlearning removes the influence of unwanted data to meet rules and safety needs. Yet a few examples of the removed data can teach the model the material again, a flaw called a relearning attack. The team connects robust unlearning to a training method that favors smooth, flat solutions.

Through a unified robust training frame, they show smoothness helps resist relearning, much as tough training resists crafted attacks. Tests on benchmark sets found smoothness methods consistently improved resistance. The approach also helped against input level jailbreak attacks. Readers see unlearning treated as a robustness problem, not just a deletion task.

Catalog record

CREATORChongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, Sijia Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05374

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, Sijia Liu Abstract The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model influence. However, state-of-the-art unlearning methods face a critical vulnerability: they are susceptible to “relearning” the removed information from a small number of forget data points, known as relearning attacks. In this paper, we systematically investigate how to make unlearned models robust against such attacks. For the first time, we establish a connection between robust unlearning and sharpness-aware minimization (SAM) through a unified robust optimization framework, in an analogy to adversarial training designed to defend against adversarial attacks. Our analysis for SAM reveals that smoothness optimization plays a pivotal role in mitigating relearning attacks. Thus, we further explore diverse smoothing strategies to enhance unlearning robustness. Extensive experiments on benchmark datasets, including WMDP and MUSE, demonstrate that SAM and other smoothness optimization approaches consistently improve the resistance of LLM unlearning to relearning attacks. Notably, smoothness-enhanced unlearning also helps defend against (input-level) jailbreaking attacks, broadening our proposal’s impact in robustifying LLM unlearning. Codes are available at https://github.com/OPTML-Group/Unlearn-Smooth . † † affiliationnotice: Equal contribution 1 Introduction With the rapid advancement of large language models (LLMs), concerns about their privacy, safety, and trustworthiness, have become increasingly prominent ( Liu et al., 2024d ; Barez et al., 2025 ) . However, retraining these models to eliminate the undesired data-model influence is often infeasible due to the significant computational and time costs involved. To address this challenge, LLM unlearning ( Yao et al., 2024 ; Eldan & Russinovich, 2023 ; Maini et al., 2024 ; Liu et al., 2024b ) has emerged as a post-pretraining strategy, which aims to mitigate the impact of undesirable data ( e.g. , sensitive, biased, unsafe, or illegal information) and suppress associated model capabilities, thereby preventing LLMs from generating harmful content while simultaneously preserving the model’s utility post-unlearning. Despite the increasing importance of LLM unlearning, several recent studies ( Łucki et al., 2024 ; Zhang et al., 2024c ; Lynch et al., 2024 ; Hu et al., 2024 ; Deeb & Roger, 2024 ) have identified a critical issue: LLM unlearning often lacks robustness . Specifically, the susceptibility to quickly recovering ‘already-unlearned’ knowledge post-unlearning is evident through so-called relearning attacks ( Lynch et al., 2024 ; Hu et al., 2024 ) . These attacks can effectively reverse the unlearning process by leveraging lightweight fine-tuning on the unlearned model using only a small number of data from the forget dataset. Although numerous LLM unlearning methods have been proposed in the literature ( Yao et al., 2024 ; Maini et al., 2024 ; Ji et al., 2024b ; Zhang et al., 2024a ; Liu et al., 2024a ; Ji et al., 2024b ; Li et al., 2024 ; Jia et al., 2024a ; Jia et al., 2024b ) , few studies have explored the robust optimization foundation for LLM unlearning. For example, negative preference optimization (NPO) ( Zhang et al., 2024a ) , one of the state-of-the-art (SOTA) LLM unlearning methods, has demonstrated superior unlearning effectiveness compared to other approaches ( Shi et al., 2024 ) . However, as we will motivate in Sec. 3 , NPO still remains vulnerable to relearning attacks. This highlights the need to develop a robust optimization foundation to strengthen LLM unlearning against such attacks. Tracing back to defenses against classic (input-level) prediction-evasion adversarial attacks, adversarial training ( Madry et al., 2018 ) , built upon min-max optimization, has proven to be a generic and effective robust optimization framework. In a similar vein, we ask: (Q) What is the robust optimization foundation for LLM unlearning against relearning attacks? Drawing inspiration from adversarial training ( Madry et al., 2018 ) , we address (Q) through the lens of min-max optimization. Here the minimization step focuses on LLM unlearning, coupled with a maximization step that simulates relearning attacks. The maximization step identifies the worst-case weight perturbations (rather than input perturbations in adversarial training) to the unlearned model, aiming to reverse the unlearning effects. We demonstrate that the robust optimization framework for LLM unlearning naturally aligns with sharpness-aware minimization ( SAM ) ( Foret et al., 2021 ) . SAM was originally developed to enhance model generalization by encouraging a uniformly low loss across the neighborhood of a given model, thereby promoting a smooth loss landscape. We will show that smoothness optimization , such as SAM, is a critical yet underexplored factor for enhancing unlearning robustness against relearning attacks. We summarize our contributions below. \bullet To our best knowledge, this is the first work to reveal that SAM naturally yields a robust optimization framework for LLM unlearning in defending against relearning attacks. \bullet We conduct an in-depth exploration of SAM-integrated LLM unlearning for enhanced robustness and establish its connection to curvature regularization and broader smoothness optimization techniques beyond SAM. \bullet We conduct extensive experiments to demonstrate the critical role of smoothness optimization, particularly SAM, in improving LLM unlearning robustness against various relearning attacks and jailbreaking attacks (that evades unlearned LLMs using input-level adversarial prompts). 2 Related Work Machine unlearning and its applications to LLMs. Machine unlearning modifies models to remove the influence of undesirable data, originally developed to mitigate post-training privacy risks ( Cao & Yang, 2015 ; Ginart et al., 2019 ; Ullah et al., 2021 ) . While retraining from scratch guarantees exact unlearning, it is computationally prohibitive, leading to research on approximate unlearning methods that balance efficiency and effectiveness ( Kurmanji et al., 2024 ; Fan et al., 2024b ; Chen et al., 2023 ) . A rapidly growing subfield is LLM unlearning ( Jang et al., 2022 ; Meng et al., 2022 ; Yao et al., 2023 ; Eldan & Russinovich, 2023 ; Jia et al., 2024b ; Zhang et al., 2024a ; Maini et al., 2024 ; Jia et al., 2024a ; Liu et al., 2024c ; Fan et al., 2024a ; Thaker et al., 2024 ) , which has been shown promise in mitigating the generation of harmful content ( Yao et al., 2023 ; Li et al., 2024 ; Jia et al., 2024b ) and protecting sensitive, copyrighted, or private information ( Eldan & Russinovich, 2023 ; Wu et al., 2023 ; Jang et al., 2022 ) . Existing LLM unlearning approaches include model-based optimization ( Maini et al., 2024 ; Yao et al., 2023 ; Jia et al., 2024a ; Fan et al., 2024a ; Zhang et al., 2024a ; Li et al., 2024 ; Jia et al., 2024a ; Wu et al., 2023 ; Fan et al., 2024a ) and input-based strategies (via prompting or in-context learning) to facilitate unlearning without extensive parameter adjustments ( Liu et al., 2024a ; Thaker et al., 2024 ; Pawelczyk et al., 2023 ) . Furthermore, recent benchmarking efforts provide valuable frameworks for evaluating the effectiveness of LLM unlearning approaches. These include TOFU ( Maini et al., 2024 ) , which focuses on fictitious unlearning using synthetic data, WMDP ( Li et al., 2024 ) , which aims to mitigate sociotechnical harms in model

Read the complete text