Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem

Zeguan Xiao, Siqing Li, Yong Wang, Xuetao Wei, Jian Yang, Yun Chen, Guanhua Chen · 2026

Read Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem by Zeguan Xiao, Siqing Li, Yong Wang, Xuetao Wei, Jian Yang, Yun Chen, Guanhua Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14808.txt.

Introduction

Xiao and six others recast language-model unlearning as an asymmetric two-task problem with retention primary and forgetting auxiliary. Their retention-prioritized gradient synthesis separates task-specific gradient extraction from conflict-aware combination. They adapt PCGrad to settle gradient conflicts and add SAGO, a new synthesis method with tighter alignment to the retain gradient. On WMDP Bio, target-model MMLU scores climbed from 44.6 percent naive to 94.0 percent with PCGrad and 96.0 percent with SAGO, while forgetting strength held steady. Reshaping gradient geometry, they conclude, beats merely rebalancing losses.

Catalog record

CREATORZeguan Xiao, Siqing Li, Yong Wang, Xuetao Wei, Jian Yang, Yun Chen, Guanhua Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36972
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14808

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem Zeguan Xiao, Siqing Li, Yong Wang, Xuetao Wei, Jian Yang, Yun Chen, Guanhua Chen Abstract Machine unlearning for large language models (LLMs) aims to remove targeted knowledge while preserving general capability. In this paper, we recast LLM unlearning as an asymmetric two-task problem: retention is the primary objective and forgetting is an auxiliary. From this perspective, we propose a retention-prioritized gradient synthesis framework that decouples task-specific gradient extraction from conflict-aware combination. Instantiating the framework, we adapt established PCGrad to resolve gradient conflicts, and introduce SAGO , a novel retention-prioritized gradient synthesis method. Theoretically, both variants ensure non-negative cosine similarity with the retain gradient, while SAGO achieves strictly tighter alignment through constructive sign-constrained synthesis. Empirically, on WMDP Bio/Cyber and RWKU benchmarks, SAGO consistently pushes the Pareto frontier: e.g., on WMDP Bio (SimNPO+GD), recovery of target model MMLU performance progresses from 44.6% (naive) to 94.0% (+PCGrad) and further to 96.0% (+ SAGO ), while maintaining comparable forgetting strength. Our results show that re-shaping gradient geometry, rather than re-balancing losses, is the key to mitigating unlearning-retention trade-offs. 1 Introduction Figure 1: Visualization of loss dynamics in LLM unlearning and retention-prioritized frameworks. Panels (a) and (b) show retain and forget losses on the WMDP Biosecurity benchmark using GradDiff, PCGrad, and SAGO. SAGO outperforms in maintaining low retain loss while achieving high forget performance, indicating reduced gradient conflicts and improved retention. Panel (c) shows that while GradDiff (Original) struggles with retention, PCGrad and SAGO dynamically refine gradients, achieving effective unlearning with stable retention. Large language models (LLMs) have achieved remarkable success in recent years. However, like many powerful technologies, LLMs are inherently dual-use and can be leveraged for both beneficial and harmful purposes. LLMs are trained on vast corpora collected from the Internet, which unavoidably contain personal information and potentially hazardous knowledge. Their capacity to memorize and reproduce training data can therefore be exploited to disclose sensitive information or to generate harmful content. A common mitigation is alignment training, which aims to teach LLMs to refuse harmful queries. Nevertheless, recent studies ( 35 ; 33 ; 27 ) find that adversaries can easily craft jailbreak prompts to circumvent these safeguards. To address these vulnerabilities, machine unlearning (MU) ( 1 ) has emerged as a promising solution to mitigate the risks associated with LLMs by directly removing private information and hazardous knowledge from the model. Unlearned models offer stronger inherent safety because even if they are jailbroken, they lack the knowledge necessary to enable malicious users. However, LLM unlearning faces a central challenge: The unlearning often degrades the model’s performance, leading to a trade-off between effective unlearning and preserving essential capabilities ( 26 ) . To make the above challenge concrete, we begin with the canonical method commonly used in unlearning: gradient ascent (GA) on the forget set. This method, while simple and directly enforcing forgetting, often leads to over-forgetting and significant performance degradation. To mitigate this, methods such as NPO ( 34 ) and SimNPO ( 4 ) regularize GA in two ways: (i) they transform the unbounded GA objective into a bounded one, which helps prevent catastrophic collapse; and (ii) they apply adaptive smoothing to the forget-set gradients, enabling more controlled divergence during unlearning. Another line of work, gradient difference (GradDiff ( 11 ) ), couples GA on the forget set with gradient descent (GD) on a retain set to preserve core capabilities. Despite these advances, conflicts between forget and retain gradients persist. To address conflicts between forgetting and retaining gradients, 20 recently proposed a bi-level optimization approach for LLM unlearning that prioritizes the forgetting objective over the retaining one. In this work , we model the trade-off between unlearning and retention as an asymmetric two-task learning problem, with retention as the primary task and unlearning as the auxiliary task. We explore two approaches to synthesize gradients. First, we adapt PCGrad ( 32 ) , a technique originally designed for mitigating gradient conflicts in multi-task learning to the unlearning scenario. This adaptation ensures that the gradients driving the unlearning process do not conflict destructively with those preserving the model’s utility. Second, we propose SAGO , a novel retention-prioritized gradient synthesis method that enhances unlearning efficacy without compromising retention performance. The key insight of SAGO lies in enforcing element-wise sign alignment between the synthesized gradients and the retention gradients, ensuring the update direction consistently supports retention. An intuitive visualization of our retention-prioritized framework can be found in Figure 1 (c). We conduct experiments on two widely used LLM unlearning benchmarks, WMDP ( 10 ) and RWKU ( 9 ) , and demonstrate that both PCGrad and SAGO significantly improve retention performance while maintaining competitive unlearning effectiveness compared to vanilla unlearning objectives. As shown in Figure 1 (a) and (b), our proposed SAGO method demonstrates particularly strong performance, notably achieving superior retention with effective unlearning. Our contributions are summarized as follows 1 1 1 Code: https://github.com/sustech-nlp/SAGO : \triangleright Asymmetric formulation. We reframe LLM unlearning as an asymmetric two-task problem and show that viewing retention as the primary objective leads to an effective generic framework. The resulting framework integrates seamlessly with diverse unlearning objectives, including existing GA+GD, NPO+GD, and SimNPO+GD, and is readily extensible to future objectives. \triangleright New gradient synthesis methods. We adapt the established PCGrad to resolve gradient conflicts, and introduce SAGO , a novel retention-prioritized gradient synthesis method. Theoretically, both variants ensure non-negative cosine similarity with the retain gradient, while SAGO achieves strictly tighter alignment through element-wise sign-constrained synthesis. \triangleright Empirical gains. SAGO consistently improves retention at comparable forgetting. On WMDP, MMLU gains are 17.8–30.7 points (Bio) and 4.1–11.7 points (Cyber) over the naive method, with an additional 0.4–1.2 points over PCGrad, keeping comparable or better forgetting effectiveness. Similar improvements are observed on RWKU. 2 Preliminaries 2.1 Problem Formulation Given an original model \mathcal{M} that is already trained on a dataset \mathcal{D} , Machine Unlearning (MU) ( 1 ) aims to remove specific information from \mathcal{M} , resulting in an unlearned model \mathcal{M}^{\prime} that no longer retains or utilizes this undesired information. Formally, we define the information to forget as a subset of \mathcal{D} , called the forget set \mathcal{D}_{f} . Ideally, after unlearning, the model should behave as if trained on retain set \mathcal{D}_{r}=\mathcal{D}\setminus\mathcal{D}_{f} . In the context of LLM unlearning, the forget set \mathcal{D}_{f} and retain set \mathcal{D}_{r} are typically text corpora. The unlearning process involves finetuning the original model \mathcal{M} on \mathcal{D}_{f} and/or \mathcal{D}_{r} with specific objectives to obtain \mathcal{M}^{\prime} . 2.2 LLM Unlearning Methods We denote the probability distribution defined by an LLM with parameters \boldsymbol{\theta} as p(x;\boldsymbol{\theta}) , where x represents a text sequence. The standard unlearning objective is to suppress the model’s likelihood on the forget set \mathcal{D}_{f} —that is, drive \log p(x;\boldsymbol{\theta}) downward for x\in\mathcal{D}_{f} . This is implemented by performing gradient ascent (GA) on the cross-entropy objective (equivalently, minimizing the negative cross-entropy) over \mathcal{D}_{f} : \mathcal{L}_{\mathrm{GA}}(\mathcal{D}_{f};\boldsymbol{\theta})=-\mathbb{E}_{x\sim\mathcal{D}_{f}}\big[-\log p(x;\boldsymbol{\theta})\big]. Minimizing the above GA objective reduces the assigned probabilities p(x;\boldsymbol{\theta}) , achieving the goal of minimizing the forget-set likelihood. Given the unbounded nature of GA, it can lead to over-forgetting and significant performance degradation. To mitigate this, methods such as NPO ( 34 ) and SimNPO ( 4 ) regularize

Read the complete text