TRCE : T owards R eliable Malicious C oncept E rasure in Text-to-Image Diffusion Models
Ruidong Chen, Honglin Guo, Lanjun Wang, Chenyu Zhang, Weizhi Nie, An-An Liu · 2025
Read TRCE : T owards R eliable Malicious C oncept E rasure in Text-to-Image Diffusion Models by Ruidong Chen, Honglin Guo, Lanjun Wang, Chenyu Zhang, Weizhi Nie, An-An Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07389.txt.
Introduction
Ruidong Chen and five co-authors present a method that teaches image-making AI to forget harmful ideas, such as explicit content, even when prompts hide them in coded words. Their 2025 paper notes that blunt filters often fail on sly prompts, while heavy erasing can spoil normal image quality.
Their two-stage method first rewrites the hidden pull of a risky prompt toward a safe, nearby meaning inside the model's text handling. It then steers the early steps of image forming away from the unsafe path. Tests on standard erasure benchmarks showed the method removed targeted ideas while keeping the model's normal skill more intact than rival methods. The authors share their code for others to check.
Catalog record
| CREATOR | Ruidong Chen, Honglin Guo, Lanjun Wang, Chenyu Zhang, Weizhi Nie, An-An Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-07389
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TRCE : T owards R eliable Malicious C oncept E rasure in Text-to-Image Diffusion Models Ruidong Chen, Honglin Guo, Lanjun Wang, Chenyu Zhang, Weizhi Nie, An-An Liu Abstract Recent advances in text-to-image diffusion models enable photorealistic image generation, but they also risk producing malicious content, such as NSFW images. To mitigate risk, concept erasure methods are studied to facilitate the model to unlearn specific concepts. However, current studies struggle to fully erase malicious concepts implicitly embedded in prompts (e.g., metaphorical expressions or adversarial prompts) while preserving the model’s normal generation capability. To address this challenge, our study proposes TRCE, using a two-stage concept erasure strategy to achieve an effective trade-off between reliable erasure and knowledge preservation. Firstly, TRCE starts by erasing the malicious semantics implicitly embedded in textual prompts. By identifying a critical mapping objective(i.e., the [EoT] embedding), we optimize the cross-attention layers to map malicious prompts to contextually similar prompts but with safe concepts. This step prevents the model from being overly influenced by malicious semantics during the denoising process. Following this, considering the deterministic properties of the sampling trajectory of the diffusion model, TRCE further steers the early denoising prediction toward the safe direction and away from the unsafe one through contrastive learning, thus further avoiding the generation of malicious content. Finally, we conduct comprehensive evaluations of TRCE on multiple malicious concept erasure benchmarks, and the results demonstrate its effectiveness in erasing malicious concepts while better preserving the model’s original generation ability. The code is available at: http://github.com/ddgoodgood/TRCE . CAUTION: This paper includes model-generated content that may contain offensive material. † † footnotetext: ∗ Corresponding author: Lanjun Wang{wang.lanjun}@outlook.com, An-An Liu{anan0422}@gmail.com 1 Introduction Recently, large-scale text-to-image (T2I) diffusion models (DMs) [ 36 , 30 , 10 , 38 ] have attracted widespread attention due to their capabilities to generate highly realistic images. Trained on extensive internet-sourced data, these models acquire the ability to produce diverse visual concepts from textual prompts. However, due to the existence of toxic training data, these models also learn to generate inappropriate content. Consequently, they may be misused to produce NSFW (Not Safe For Work) images with prompts containing inappropriate concepts. To address these safety concerns, researchers have proposed various mechanisms for enhancing safety [ 48 ] , such as filtering out toxic data and retraining the models [ 35 ] , employing safety checkers to filter outputs [ 32 ] , and applying safeguard strategies to guide the generation process [ 40 , 47 ] . To further enhance safety, concept erasure (CE) [ 11 , 19 , 49 , 24 , 21 , 13 , 12 , 23 ] has been proposed to limit the ability of DMs to generate specific concepts. For example, if the concept “sexual” were erased by CE, when the prompt includes keywords such as “nudity, erotic, porn, etc.”, the model will generate images of clothed people instead. However, due to insufficient erasure, current CE methods are still not effective enough in preventing the generation of NSFW content [ 44 , 51 , 8 , 46 ] . Existing methods are typically designed to eliminate specific keywords by fine-tuning [ 11 , 19 , 49 , 24 , 23 , 21 ] or editing [ 12 , 13 , 23 ] model parameters. As a result, they struggle to erase malicious concepts, as they are often expressed metaphorically or associatively, without using keywords (e.g., the prompt in the second row of Fig. 2 ). In most scenarios, the semantics of malicious concepts are implicitly embedded in the prompts, which poses challenges for erasing malicious concepts. To eliminate such implicit semantics, existing methods often degrade the model’s original generation ability on irrelevant content to achieve stronger erasure reliability (Fig. 1 Right). Overall, the inability to balance reliable concept erasure with knowledge preservation constitutes the main limitation that prevents current CE methods from being effectively applied to eliminate malicious concepts. To address these limitations, this study proposes TRCE , which is based on a two-stage erasure design, aiming to ensure both reliable erasure and knowledge preservation through the cooperation of the two stages. Firstly, the TRCE begins with the “ Textual Semantic Erasure ” (Sec. 4.1 ), mitigating the influence of malicious semantics from the input prompt on the generation process. Recognizing the influence of special embeddings [ 31 , 9 ] on generation (as illustrated in Fig. 2 ), we note that the [EoT] (End of Text) embeddings capture semantics from the entire prompt and significantly contribute to the generation of salient regions [ 7 ] . By leveraging this insight, we can move away from existing methods [ 12 , 13 , 23 ] that directly map keyword embeddings to unrelated ones. Instead, we select [EoT] as the mapping objective and employ a closed-form solution [ 12 , 13 ] to modify the cross-attention matrices. Mapping prompts with malicious concepts to contextually similar prompts but with safe concepts, thus achieving the effective erasure of implicitly embedded malicious semantics. On the basis of the first stage, TRCE further proposes “ Denoising Trajectory Steering ” (Sec. 4.2 ) to avoid the final generation of malicious visual content. Typically, during the denoising process, a diffusion model first forms outlines of prominent regions and then produces specific visual details at the turning point [ 33 ] . Based on this behavior, at an early sampling stage of generation, we propose using a reference model to provide both safe and unsafe predictions before the turning point. Specifically, a contrastive loss is employed to steer the denoising prediction toward the safe direction and away from the unsafe one. As shown in Fig. 4 , take advantage of the deterministic properties [ 43 ] of diffusion model’s sampling trajectory, since the malicious semantics of generation guidance have been mostly eliminated in the first stage, by just adjusting the model’s early denoising, the model’s sampling trajectory can deviate from unsafe visual patterns without significantly affecting the image content, thus achieving better knowledge preservation ability for the concept erasure. We summarize the contribution of TRCE as follows: Figure 2 : Due to the attention mechanism, special embeddings [ 31 , 9 ] of input prompts (e.g. [EoT]) carry rich semantics about concepts and pay attention to the semantics of salient regions [ 20 ] . By eliminating the influence of the [EoT] token on generation (i.e., setting the attention map corresponding to [EoT] to 0), the image content will be dramatically affected. • TRCE proposes a collaborated two-stage erasure strategy, achieving reliable malicious concept erasure while taking minimalist effects on the model’s generation ability. • TRCE identifies a critical mapping objective for textual semantic erasure, specifically the [EoT] embeddings, which achieves effective erasure of the implicitly embedded malicious semantics. • TRCE proposes an effective denoising trajectory steering strategy, which optimizes the model’s early denoising through contrastive fine-tuning. • We conduct comprehensive evaluations of TRCE across various malicious content generation scenarios, including network prompts [ 40 ] , adversarial prompts [ 51 , 44 , 46 , 8 ] , and multi-concept erasure [ 40 ] , along with ablation studies. The results demonstrate the effectiveness of TRCE in conducting reliable malicious concept erasure. 2 Related Work 2.1 Concept