Erased or Dormant? Rethinking Concept Erasure Through Reversibility

Ping LiuDepartment of Computer ScienceUniversity of Nevada RenoReno, NV 89512, USA, Chi Zhang Department of MathematicsNational University of SingaporeSingapore 119076 · 2025

Read Erased or Dormant? Rethinking Concept Erasure Through Reversibility by Ping LiuDepartment of Computer ScienceUniversity of Nevada RenoReno, NV 89512, USA, Chi Zhang Department of MathematicsNational University of SingaporeSingapore 119076 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16174.txt.

Introduction

Tools that claim to scrub a notion from a picture model may only have taught it to hide. Ping Liu and Chi Zhang test that doubt in 2025 on two known scrub methods. The catalog's author field tangles school addresses into the names; the head byline is clean. Their probe is simple: tune the scrubbed model lightly and see whether the banned notion walks back in.

It did, again and again, with much of its old sharpness, after only a small tune. Both scrub styles, one that edits inner attention and one that retrains the whole, failed the same way. The pair say the field grades scrubbing too softly, on the very prompts the scrub was built to dodge. Real safety, they argue, needs cuts at the level of the model's stored form, and harder tests.

Catalog record

CREATORPing LiuDepartment of Computer ScienceUniversity of Nevada RenoReno, NV 89512, USA, Chi Zhang Department of MathematicsNational University of SingaporeSingapore 119076
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16174

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Erased or Dormant? Rethinking Concept Erasure Through Reversibility Ping LiuDepartment of Computer ScienceUniversity of Nevada RenoReno, NV 89512, USA, Chi Zhang Department of MathematicsNational University of SingaporeSingapore 119076 Abstract To what extent do concept erasure techniques in diffusion models truly remove, rather than merely suppress, targeted concepts? In this paper, we explore this question by introducing a diagnostic framework that leverages lightweight parameter adaptation to probe the robustness and reversibility of leading erasure methods. Central to our approach are two minimal yet general probes: (i) a Gradient-Guided Probe, which restores suppressed behavior by reversing gradient signals, and (ii) an Instance-Personalization Probe, which reinstates concepts through few-shot supervision. Across six erasure algorithms, multiple concept types, and diverse diffusion backbones, we consistently find that erased concepts can be recovered with high fidelity after only minimal adaptation. Our theoretical analysis reinforces these results, showing that reversed weight remain bounded to the original parameters, leaving much of the targeted representation intact. Together, these findings demonstrate that existing methods do not eliminate concepts but merely push them below the surface, where they can be readily revived. As such, our work calls for a rethinking of concept erasure: moving beyond superficial suppression toward approaches that dismantle latent structures at their core, alongside more rigorous standards for evaluating safety in generative models. 1 Introduction Text-to-image diffusion models Rombach et al. (2022) ; Ramesh et al. (2022) have emerged as a backbone of modern generative AI, capable of producing high-quality images from natural language prompts. Yet, their open-ended generative power also raises pressing safety and ethical concerns, including the potential for harmful outputs Bird et al. (2023) and violations of intellectual property Zhang et al. (2023) . To mitigate these risks, recent work has explored concept erasure Lu et al. (2024) ; Gong et al. (2024) , which seeks to suppress a model’s ability to generate undesired concepts such as offensive objects, copyrighted artistic styles, or personal identities. Existing approaches pursue this goal through a range of mechanisms, including projection in cross-attention layers Gandikota et al. (2023) ; Gandikota et al. (2024) ; Lu et al. (2024) ; Gong et al. (2024) ; Zhang et al. (2024b) , pruning strategies Yang et al. (2024) ; Chavhan et al. (2025) , regularization-based editing Huang et al. (2024) , and adversarial-guided erasure Zhang et al. (2024c) ; Bui et al. (2025) . Despite these advancements, a fundamental question remains: do current erasure techniques truly eliminate a model’s capacity to generate the targeted concept, or do they merely enforce conditional suppression? This difference is not merely theoretical but has direct consequences for how safely and reliably diffusion models can be used in real-world applications. If erasure were genuinely irreversible, the model’s representational space would lack usable traces of the concept, making recovery practically infeasible under minor perturbations or adaptations. In contrast, if latent representations remain dormant but intact, erased concepts may reappear when prompts are varied or through lightweight parameter adjustments. Such reversibility exposes serious risks: malicious actors could deliberately reactivate forbidden content, while benign users might also unintentionally trigger it in unexpected deployment scenarios. Recent work has begun to examine this fragility, but almost exclusively from the prompt perspective. Pham et al. (2024) showed that erased concepts can be revived with adversarial prompts, while Lu et al. (2025) demonstrated circumvention through prompt perturbation, inpainting, and noise-based probing. Studies of these approaches often remain confined to the prompt level, focusing primarily on identifying or crafting potentially malicious prompts. As a result, the latent parameter-level mechanisms driving the generation of erased concepts remain largely unexamined. In this paper, we extend beyond prompt-level approaches and systematically investigate concept erasure reversibility from a parameter-level perspective. To this end, we introduce a diagnostic framework designed to assess the vulnerability of existing erasure methods. The framework leverages two minimal yet general strategies: a Gradient-Guided Probe, which restores erased behavior by reversing suppression gradients within the model parameters, and an Instance-Personalization Probe, which reinstates concepts through few-shot personalization using a small set of reference images. Complementing these empirical probes, we provide a theoretical analysis that establishes formal bounds on deviations from the original model, demonstrating that erased concepts often remain recoverable with only minor parameter updates. We extensively evaluate the reversibility of erased concepts across six state-of-the-art erasure methods: ESD Gandikota et al. (2023) , UCE Gandikota et al. (2024) , MACE Lu et al. (2024) , FMN Zhang et al. (2024b) , AGE Bui et al. (2025) , and ConceptPrune Chavhan et al. (2025) . Our results reveal that erased concepts can often be reinstated following only minimal parameter adaptation, as confirmed by improvements across classification accuracy, CLIP alignment, and LPIPS similarity, while the model’s untargeted performance remains largely unaffected. These empirical observations closely align with our theoretical analysis, which demonstrates that the reactivated model frequently approximates the original, unerased model within a bounded error. Crucially, these findings are consistent across different methods, concept categories, and diffusion backbones, indicating that recoverability is a persistent limitation of current erasure techniques and that latent representations often persist despite apparent suppression. This underscores the need for future research to develop erasure strategies that explicitly dismantle residual representations and provide stronger guarantees of irreversibility. In summary, our contributions are as follows: • A diagnostic framework for reversibility. We introduce a parameter-level framework with two lightweight probes: a Gradient-Guided Probe that restores suppressed gradients, and an Instance-Personalization Probe that rebinds concepts from a few examples. This design goes beyond prompt-based circumvention, directly evaluating whether erased concepts can be readily recovered within the model’s weight space. • Theoretical and representation-level analysis. We derive reactivation bounds that quantify deviations of the model’s parameters from the original, unerased version. These theoretical results align with empirical measures by classification accuracy, CLIP alignment, and LPIPS perceptual similarity, providing an explanation for why erased concepts can often be recovered with only minimal adaptation. • Extensive empirical evaluation. We evaluate six state-of-the-art erasure methods across both object and style concepts, using multiple diffusion backbones. Results from both probes consistently demonstrate that erased concepts can be reinstated with high fidelity following only minimal adaptation, highlighting recoverability as a fundamental limitation of current concept erasure techniques. 2 Related Work Diffusion Models and Personalization. Text-to-image diffusion models have emerged as the dominant paradigm for generative image synthesis Rombach et al. (2022) ; Ramesh et al. (2022) ; Ho et al. (2020) . Their ability to generate semantically faithful and photorealistic images has enabled widespread adoption. Personalization methods such as DreamBooth Ruiz et al. (2023) , textual inversion Gal et al. (2022) , and parameter-efficient tuning Kumari et al. (2023) ; Shi et al. (2024) allow models to encode new concepts from limited data. While these techniques highlight the flexibility of diffusion models, they also amplify risks of misuse, motivating research on concept erasure Kim & Qi (2025) ; Xie et al. (2025) . Concept Erasure in Diffusion Models. Concept erasure aims to suppress a model’s ability to generate undesired objects, styles, or identities. Representative approaches include fine-tuning (e.g., ESD Gandikota et al. (2023) ), cross-attention editing methods such as UCE and MACE Gandikota et al. (2024) ; Lu et al. (2024) , and attention re-steering techniques like FMN Zhang et al.

Read the complete text