Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning

Changsheng Wang, Yihua Zhang, Jinghan Jia, Parikshit Ram, Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalie Baracaldo, Sijia Liu · 2025

Read Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning by Changsheng Wang, Yihua Zhang, Jinghan Jia, Parikshit Ram, Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalie Baracaldo, Sijia Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01339.txt.

Introduction

Changsheng Wang and co-authors make machine unlearning stick, even after someone fine-tunes the model again. Unlearning scrubs targeted knowledge from a large language model for privacy or safety, but ordinary fine-tuning, even on unrelated tasks, can resurrect what was scrubbed. Borrowing invariance from causal learning, the team proposes invariant LLM unlearning.

Their training ties the forgetting to features stable across environments, so later tuning has less to grab onto. Trained on a single data set, the method held up across many surprise fine-tunes, from math to sentiment. On the WMDP and MUSE benchmarks it beat leading rivals while keeping the model's fine-tuned usefulness. A task vector analysis explains why the invariance buys durability.

Catalog record

CREATORChangsheng Wang, Yihua Zhang, Jinghan Jia, Parikshit Ram, Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalie Baracaldo, Sijia Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01339

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning Changsheng Wang, Yihua Zhang, Jinghan Jia, Parikshit Ram, Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalie Baracaldo, Sijia Liu Abstract Machine unlearning offers a promising solution to privacy and safety concerns in large language models (LLMs) by selectively removing targeted knowledge while preserving utility. However, current methods are highly sensitive to downstream fine-tuning, which can quickly recover forgotten information—even from unrelated tasks. To address this, we introduce invariance into unlearning for the first time, inspired by invariant risk minimization (IRM). Building on this principle, we propose invariant LLM unlearning (ILU), a regularization-based framework that enhances robustness. Notably, ILU generalizes well to diverse fine-tuning tasks, even when trained using a single dataset. A task vector analysis is also provided to further elucidate the rationale behind ILU’s effectiveness. Extensive experiments on the WMDP and MUSE benchmark, reveal that ILU significantly outperforms state-of-the-art unlearning methods, including negative preference optimization (NPO) and representation misdirection for unlearning (RMU). Notably, ILU achieves superior unlearning robustness across diverse downstream fine-tuning scenarios ( e.g. , math, paraphrase detection, and sentiment analysis) while preserving the fine-tuning performance. Our experiments and codes are available at https://github.com/OPTML-Group/Unlearn-ILU . 1 Introduction Large language models (LLMs) have revolutionized generative AI ( Touvron et al. 2023 ; Achiam et al. 2023 ; Liu et al. 2024a ) . However, their extensive training on diverse corpora introduces critical ethical and security risks, including privacy violations through memorization of sensitive data ( Huang et al. 2024 ; Shi et al. 2024 ) , amplification of societal biases ( Motoki et al. 2023 ) , and generation of harmful or illegal content ( Wen et al. 2023 ; Li et al. 2024 ) . These challenges necessitate effective mechanisms to eliminate undesirable data-model influences in pre-trained models without compromising their utility—a problem referred to as LLM unlearning ( Liu et al. 2024c ; Maini et al. 2024 ; Yao et al. 2024b ) . Existing LLM unlearning methods often operate under the assumption of being standalone interventions for safe model deployment ( Yao et al. 2024a ; Zhang et al. 2024a ; Li et al. 2024 ; Gao et al. 2024 ; Cooper et al. 2024 ; Liu et al. 2024b ) , meaning they do not account for subsequent operations post-unlearning. However, in practice, unlearning is rarely the final optimization step, as industrial pipelines often apply subsequent fine-tuning to adapt models for downstream tasks. Recent empirical studies ( Barez et al. 2025 ; Łucki et al. 2024 ; Hu et al. 2024 ; Tamirisa et al. 2024 ; Lynch et al. 2024 ) reveal a critical vulnerability in current unlearning approaches: knowledge erased during unlearning can unexpectedly resurface through downstream fine-tuning, even on data unrelated to the unlearning objective. This vulnerability suggests that existing unlearning approaches only remove unwanted knowledge temporarily . However, unlearning should ideally be permanent for a large variety of use cases including removing copyrighted or harmful material. Therefore, we ask: Q: How can we define ‘invariance’ and integrate it into LLM unlearning to enhance resilience against fine-tuning? Addressing Q necessitates a fundamental rethinking of LLM unlearning optimization to account for the impact of future model adaptations, a highly non-trivial challenge given the unforeseen nature of downstream fine-tuning operations. One straightforward yet computationally intensive approach is to leverage meta-learning ( Finn et al. 2017 ; Tamirisa et al. 2024 ) , which frames the unlearned model as a meta-model designed to be agnostic to downstream fine-tuning. However, this approach could be challenging to scale due to the need for gradient unrolling to simulate fine-tuning paths during the optimization process ( Tamirisa et al. 2024 ; Zhang et al. 2024b ) . Thus, instead of meta-learning, we explore invariance in the unlearned model—ensuring that forgotten knowledge remains irretrievable through parameter updates from irrelevant downstream fine-tuning tasks. To model and promote ‘invariance’, we adopt the technique of invariant risk minimization (IRM) ( Arjovsky et al. 2019 ) , incorporating its invariance regularization (which achieves invariant predictions agnostic to different training environments) into LLM unlearning. If downstream fine-tuning is treated as an environment that challenges unlearning, invariance regularization can be then integrated with the unlearning objective to obtain unlearning resilience against fine-tuning. In our work, the integration of IRM into unlearning leads to a new regularized optimization framework, termed invariant LLM unlearning ( ILU ), which optimizes the model to remain stationary under fine-tuning perturbations during unlearning, aiming to “immunize” against the revival of erased knowledge through fine-tuning. In addition, ILU employs lightweight gradient regularization that integrates seamlessly with state-of-the-art (SOTA) unlearning methods like negative preference optimization (NPO) ( Zhang et al. 2024a ) and representation misdirection for unlearning (RMU) ( Li et al. 2024 ) , avoiding the computational overhead of meta-learning while enabling robust knowledge erasure when subjected to fine-tuning. Furthermore, ILU generalizes effectively to diverse fine-tuning datasets, including those absent during (invariant) training. We summarize our contributions below: ❶ We introduce the concept of invariance into LLM unlearning, establishing a novel connection between IRM and LLM unlearning to enhance resilience against downstream fine-tuning. This integration results in ILU. ❷ We demonstrate that the invariance regularization in ILU, even with a simple formulation built upon a single unrelated fine-tuning set, can effectively enhance the resilience of an unlearned model against new, unforeseen downstream fine-tuning tasks. We also conduct a task vector analysis to elucidate the underlying rationale behind ILU. ❸ We show that ILU enhances SOTA LLM unlearning methods, such as NPO and RMU, when subjected to downstream fine-tuning. For instance, on the WMDP unlearning benchmark, ILU achieves a 23% average robustness improvement over RMU across 6 fine-tuning tasks, while maintaining fine-tuning accuracy. 2 Related Work Machine unlearning in LLMs. Recent advances in machine unlearning for LLMs have shown promise in addressing risks associated with undesired data retention ( Liu et al. 2024c ; Yao et al. 2024a ; Zhuang et al. 2024 ; Maini et al. 2024 ; Eldan & Russinovich 2023 ) . Practical implementations span critical applications, such as privacy protection through the removal of sensitive information ( Wu et al. 2023 ; Yu et al. 2023 ) , prevention of harmful content generation ( Lu et al. 2022 ; Li et al. 2024 ) , and elimination of memorized sequences ( Barbulescu & Triantafillou 2024 ; Jang et al. 2023 ) . Most LLM unlearning methods rely on effective and efficient optimization techniques to avoid computationally prohibitive retraining while aiming to ‘faithfully’ remove unwanted data-model influences ( Liu et al. 2024c ) . For instance, regularized optimization ( Yao et al. 2024b ; Liu et al. 2024c ; Li et al. 2024 ; Zhang et al. 2024a ) has been predominantly employed to balance unlearning effectiveness with preserved model utility post-unlearning. Some approaches employ localized interventions that target specific model components associated with unwanted capabilities ( Meng et al. 2022 ; Wei et al. 2024 ; Jia et al. 2024 ) . Other unlearning approaches leverage in-context learning ( Pawelczyk et al. 2023 ; Thaker et al. 2024 ) or task vector ( Ilharco et al. 2023 ) to negate the effects of unwanted data or model capabilities in LLMs. Robustness

Read the complete text