Not All Tokens Are Meant to Be Forgotten

Xiangyu Zhou, Yao Qiang, Saleh Zare Zade, Douglas Zytko, Prashant Khanduri, Dongxiao Zhu · 2025

Read Not All Tokens Are Meant to Be Forgotten by Xiangyu Zhou, Yao Qiang, Saleh Zare Zade, Douglas Zytko, Prashant Khanduri, Dongxiao Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03142.txt.

Introduction

Xiangyu Zhou and co-authors tackle machine unlearning in a 2025 paper. Large language models can memorise private or protected text during training. Unlearning aims to erase such content, but current methods often erase too much, dulling the model on normal tasks.

Their framework, called TIF, first separates the words that carry the unwanted content from ordinary words in the same samples. A training step then pushes the model away from the unwanted words while actively protecting the ordinary ones. Tests on two standard benchmarks showed stronger forgetting with less loss of general skill than rival methods. The authors argue that careful targeting, rather than broad suppression, is the key to safe unlearning.

Catalog record

CREATORXiangyu Zhou, Yao Qiang, Saleh Zare Zade, Douglas Zytko, Prashant Khanduri, Dongxiao Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03142

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Not All Tokens Are Meant to Be Forgotten Xiangyu Zhou, Yao Qiang, Saleh Zare Zade, Douglas Zytko, Prashant Khanduri, Dongxiao Zhu Abstract Large Language Models (LLMs), pre-trained on massive text corpora, exhibit remarkable human-level language understanding, reasoning, and decision-making abilities. However, they tend to memorize unwanted information, such as private or copyrighted content, raising significant privacy and legal concerns. Unlearning has emerged as a promising solution, but existing methods face a significant challenge of over-forgetting. This issue arises because they indiscriminately suppress the generation of all the tokens in forget samples, leading to a substantial loss of model utility. To overcome this challenge, we introduce the Targeted Information Forgetting (TIF) framework, which consists of (1) a flexible targeted information identifier designed to differentiate between unwanted words (UW) and general words (GW) in the forget samples, and (2) a novel Targeted Preference Optimization approach that leverages Logit Preference Loss to unlearn unwanted information associated with UW and Preservation Loss to retain general information in GW, effectively improving the unlearning process while mitigating utility degradation. Extensive experiments on the TOFU and MUSE benchmarks demonstrate that the proposed TIF framework enhances unlearning effectiveness while preserving model utility and achieving state-of-the-art results. Our code is available at: https://github.com/xzhou98/LLM-Unlearning-TPO Figure 1: Comparison of our TPO and NPO zhang2024negative on key metrics: forget quality and model utility. The right panel illustrates example responses generated by the unlearned model with TPO (ours) and NPO on the retain set. The results are derived from the Forget05 task of the TOFU dataset maini2024tofu , with the models trained for 10 epochs. 1 Introduction Large Language Models (LLMs), pre-trained on vast text corpora, demonstrate exceptional capabilities in text generation and nuanced comprehension of language tasks brown2020language . However, LLMs exhibit a tendency to memorize parts of their training data carlini2021extracting . While memorization can be beneficial for tasks like question answering brown2020language and code generation jiang2024survey , it also raises significant security and safety concerns. Specifically, training data that includes personally identifiable information (PII) or copyrighted content poses risks of privacy violations or copyright infringement carlini2021extracting ; karamolegkou2023copyright ; li2024wmdp . To mitigate these issues, research efforts have focused on developing machine unlearning techniques to enable LLM dememorization. Figure 2: Illustration of the proposed TIF framework . TIF exploits an unwanted information identifier to differentiate between unwanted and general information in the forget sample (e.g., \text{A}_{\text{f}} in the right panel). The former is represented by Unwanted Words (UW) and the latter by General Words (GW). Instead of removing the entire response \text{A}_{\text{f}} , TIF selectively unlearns only UW while preserving general knowledge associated with GW by retraining on GW. This targeted approach enables effective forgetting while maintaining model utility. The right panel demonstrates a more complete and correct model response compared to the existing unlearning approach on the left. Machine unlearning cao2015towards ; bourtoule2021machine ; nguyen2022survey was developed as an efficient approach to remove the influence of specific training samples from (pre-)trained models, eliminating the need for full retraining. Recently, unlearning techniques have been adapted for LLMs, raising ongoing challenges in precisely removing private or copyrighted content learned from specific training samples yao2023large ; pawelczyk2023context ; eldan2023s ; chen2023unlearn . Early approaches for unlearning LLMs focused on fine-tuning the parameters of pre-trained models li2024wmdp ; chen2023unlearn ; si2023knowledge ; jang2022knowledge . Some methods, such as the one proposed in yao2023large , often rely on gradient ascent (GA) based optimization to achieve the goal of unlearning. Since the loss function generally has no upper bound, this approach makes the precise control over parameter updates difficult, frequently resulting in catastrophic collapse, where the overall performance of the unlearned model deteriorates substantially zhang2024negative . To address this issue, methods like Negative Preference Optimization (NPO) zhang2024negative and SimNPO fan2024simplicity have developed preference optimization-based frameworks that leverage negative examples to mitigate performance collapse. Despite these foundational efforts, LLM unlearning still faces several critical challenges: (C1) Ambiguous Unlearning Targets . Most existing approaches treat the entire forget sample as the unlearning target without differentiating between unwanted information (to be unlearned) and general information (to be retained), as shown in Figure 2 . This lack of distinction often results in significant degradation of model utility wang2024selective ; liu2024large ; lynch2024eight ; rezaei2024restor . (C2) Lack of Flexible and Generalizable Unwanted Information Identification . Recent methods attempt fine-grained unlearning but face critical limitations in information identification: ECO liu2024large employs a sentence-level identifier that overfits to specific keywords in the forget sample (e.g., the “college”), rather than aligning with the unlearning requester’s intent thaker2024position . This leads to insufficient unlearning when prompts containing these specific keywords are removed. While SEUL wang2024selective improves precision by leveraging generative models (e.g., ChatGPT) to identify continuous sensitive spans (e.g., PII), it remains limited to handle discontinuous and/or diverse unlearning targets (e.g., copyrighted content). This rigidity in identification compromises both effectiveness and generalizability. (C3) Sensitivity to Forget Set Size . Methods based on preference optimization zhang2024negative ; rafailov2024direct mitigate catastrophic collapse more effectively than other baselines, helping to preserve model utility. However, their effectiveness declines significantly as the forget set size increases, resulting in notable utility loss liu2024large , as shown in Figure 1 . To overcome the challenges (C1)-(C3) , we introduce the Targeted Information Forgetting (TIF) framework, as illustrated in Figure 2 . The key contributions of our work are listed below: (1) TIF Framework. To tackle (C1) , we propose a novel TIF framework for LLM unlearning. Different from existing unlearning approaches such as NPO, which predominantly unlearn entire information associated with the forget instances (e.g., \mathbb{A_{f}} in the left panel of Figure 2 ), our TIF is designed to unlearn only the targeted unwanted information, such as the city of born in the right panel’s example. General information is often associated with some “General Words ( GW )”, including stop words and commonly used phrases, which frequently appear in both retain and forget sets. In contrast, “Unwanted Words ( UW )” correspond to specific private or copyrighted content, such as city of born. By specifically targeting only UW for unlearning, our TIF preserves more general information compared to existing methods like NPO, effectively preventing over-forgetting and enabling the model to generate more complete and accurate responses, as demonstrated in the retain set answers in Figure 2 . (2) Unwanted Information Identification. To address (C2) , we develop flexible yet effective approaches for unwanted information identification: a generative model such as ChatGPT-4, and a discriminative model such as DistilBERT sanh2019distilbert , to effectively differentiate UW from GW. We evaluate the unlearning performance of both approaches and demonstrate the use cases for each. As a bottom line, even identifying function words (e.g., the, is, or an) as GW according to linguistics would improve model utility preservation. (3) A Novel Optimization Method to Retain Model Utility. To overcome (C3) , we advance preference optimization algorithms zhang2024negative ; fan2024simplicity by introducing Targeted Preference Optimization ( TPO ), a novel optimization objective designed to mitigate the significant utility degradation observed in NPO. Specifically, our TPO integrates two innovative components: Preservation loss (PL) to maintain general model utility by retraining on GW, and Logit preference loss (LPL) to unlearn unwanted information in UW. This optimization approach

Read the complete text