SepS : A Separability Measure for Robust Unlearning in LLMs
Wonje Jeung, Sangyeon Yoon, Albert No · 2025
Read SepS : A Separability Measure for Robust Unlearning in LLMs by Wonje Jeung, Sangyeon Yoon, Albert No (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14832.txt.
Introduction
Wonje Jeung, Sangyeon Yoon, and Albert No grade machine unlearning the way real prompts actually arrive: mixed. Their paper appeared in two thousand twenty-five. Unlearning should erase chosen knowledge while keeping the rest. Standard tests ask forget and retain questions separately. Real users blend both in one prompt. That blend breaks many current methods.
Their SEPS framework measures separability inside single mixed prompts. Experiments across three benchmarks expose two failure modes. Untargeted methods erase everything once a forget item appears. Targeted methods overfit lone questions and collapse on mixes. The metric gives builders a harder, fairer test. Safer unlearning for deployed models is the stake.
Catalog record
| CREATOR | Wonje Jeung, Sangyeon Yoon, Albert No |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14832
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SepS : A Separability Measure for Robust Unlearning in LLMs Wonje Jeung, Sangyeon Yoon, Albert No Abstract Machine unlearning aims to selectively remove targeted knowledge from Large Language Models (LLMs), ensuring they forget specified content while retaining essential information. Existing unlearning metrics assess whether a model correctly answers retain queries and rejects forget queries , but they fail to capture real-world scenarios where forget queries rarely appear in isolation. In fact, forget and retain queries often coexist within the same prompt, making mixed-query evaluation crucial. We introduce SepS , an evaluation framework that explicitly measures a model’s ability to both forget and retain information within a single prompt. Through extensive experiments across three benchmarks, we identify two key failure modes in existing unlearning methods: (1) untargeted unlearning indiscriminately erases both forget and retain content once a forget query appears, and (2) targeted unlearning overfits to single-query scenarios, leading to catastrophic failures when handling multiple queries. To address these issues, we propose Mixed Prompt (MP) unlearning, a strategy that integrates both forget and retain queries into a unified training objective. Our approach significantly improves unlearning effectiveness, demonstrating robustness even in complex settings with up to eight mixed forget and retain queries in a single prompt. We release code at https://github.com/AI-ISL/SEPS . 1 Introduction Large Language Models (LLMs) ( Achiam et al., 2023 ; Bai et al., 2023 ; Touvron et al., 2023 ; Dubey et al., 2024 ; Guo et al., 2025 ) have demonstrated remarkable capabilities in natural language processing tasks, becoming increasingly prevalent in real-world applications. However, several critical challenges persist, including copyright concerns ( Karamolegkou et al., 2023 ) , potential for harmful or biased outputs ( Li et al., 2023 ; Jeung et al., 2024 ) , factual inconsistencies ( Huang et al., 2023 ) , and the need for continuous model updates ( Jiang et al., 2024 ) . These challenges emerge naturally from the expanding training datasets. Among the proposed solutions, unlearning has gained substantial attention. In general, unlearning aims to selectively remove specific information from an already trained model ( Kurmanji et al., 2023 ; Golatkar et al., 2020 ; Zhang et al., 2024a ; Yuan et al., 2025 ; Eldan and Russinovich, 2023 ; Maini et al., 2024 ; Li et al., 2024 ; Chen and Yang, 2023 ) or constrain its responses to meet particular requirements ( Zhang et al., 2024b ; Lu et al., 2024 ; Liu et al., 2024b ) . These methods are typically paired with evaluation protocols ( Eldan and Russinovich, 2023 ; Li et al., 2024 ; Maini et al., 2024 ; Shi et al., 2025 ) to determine whether the request to ‘forget’ certain content has been effectively executed while preserving other essential knowledge. Despite rapid advances in unlearning, recent studies reveal an unsettling reality: even models that appear ‘unlearned’ under current evaluations remain highly fragile ( Thaker et al., 2024a ; Zhang et al., 2025 ) . For example, training on a small dataset related (but not identical) to the forget set ( Hu et al., 2025 ) or making slight modifications to prompt formats ( Doshi and Stickland, 2024 ; Joshi et al., 2024 ) can cause ‘forgotten’ information to resurface. These findings underscore a significant gap between measured unlearning performance and genuine, robust forgetting. Recently, Thaker et al. (2024a) found that certain unlearning methods ( e.g. , NPO ( Zhang et al., 2024a ) and ECO ( Liu et al., 2024a ) ) fail to differentiate retain and forget queries when both appear in a single prompt under the TOFU benchmark ( Maini et al., 2024 ) . This weakness suggests that current techniques cannot effectively separate information marked for forgetting from information that should be preserved, despite the fact that real-world prompts frequently blend these queries. To address this limitation, we propose a framework, SepS , that specifically tests whether a model can forget targeted information and retain unrelated content in the same prompt. We apply this framework to a wide range of existing unlearning methods, providing an extensive analysis of their performance. Our findings reveal two major pitfalls: (1) untargeted unlearning ( e.g. , Gradient Ascent) erases all knowledge in mixed prompts, thereby undermining the content intended for retention; and (2) targeted unlearning ( e.g. , producing “I don’t know” for forget queries) often overfits to single-query scenarios, ignoring follow-up queries, whether forget or retain. Surprisingly, when two consecutive forget questions are posed, the model often answers the initial forget query more accurately than it answers a second retain query in a two-retain scenario. This outcome underscores the pitfalls of single-query overfitting and highlights the need for more robust methods that can handle multiple or consecutive prompts. Building on the limitations of existing unlearning methods, we introduce the mixed prompt (MP) framework, which unifies forget and retain queries within a single training objective. It includes two variants: MP-ME for untargeted unlearning and MP-IDK for targeted unlearning, both trained on prompts containing interleaved forget and retain questions. Crucially, it ensures selective forgetting without compromising essential knowledge, overcoming the failures of prior methods under interleaved prompts. Our experiments demonstrate that MP-ME and MP-IDK not only show decent unlearning performance in controlled benchmarks but also generalize to complex real-world scenarios. Together, SepS and the mixed prompt framework advance unlearning research by exposing separability failures and offering a practical path toward models that reliably forget what they must while preserving what they should. 2 Related Work Machine unlearning has recently been extended to LLMs. Given the computational demands of large-scale data and model architectures, researchers have focused on scalable methodologies, including gradient-based optimization ( Chen and Yang, 2023 ; Jia et al., 2024 ; Yoon et al., 2025 ; Jeung et al., 2025 ; Maini et al., 2024 ; Zhang et al., 2024a ; Rafailov et al., 2023 ) , task arithematic ( Ilharco et al., 2023 ; Barbulescu and Triantafillou, 2024 ) , guardrails ( Thaker et al., 2024b ) , and in-context unlearning ( Pawelczyk et al., 2024 ) . Despite advances in machine unlearning, recent research has highlighted their inherent fragility. For example, content is often readily recoverable through relearning ( Hu et al., 2025 ) , paraphrasing ( Patil et al., 2024 ) , or few-shot prompting ( Jin et al., 2024 ; Lynch et al., 2024 ) . Building on these observations, our work reveals a critical fragility in unlearning, namely separability, when forget and retain prompts are simultaneously given, as models often struggle to correctly answer retain queries while refusing forget ones. For a comprehensive overview and historical background on machine unlearning, see Appendix A . 3 Preliminaries 3.1 Problem Setup The goal of machine unlearning is to remove the influence of forget set (e.g., copyrighted books) while preserving the general capabilities learned from the retain set . A language model parameterized by \theta defines a probability distribution p(\cdot|s;\theta) over the next token given an input sequence s . Let \mathcal{D}={(q^{(i)},a^{(i)})}_{i=1}^{N} be a training set, where q_{i} is the input query and a_{i} the corresponding answer. Fine-tuning a base model \theta_{b} on \mathcal{D} yields a reference model \theta_{r} . For unlearning, we partition \mathcal{D} into a forget set \mathcal{D}_{f}={(q_{f}^{(i)},a_{f}^{(i)})_{i=1}^{N_{f}}}