Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions Conference: Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Proceedings of the ACM Web Conference 2025 (WWW ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/36964
Zhiwen Li, Die Chen, Mingyuan Fan, Cen Chen, Yaliang Li, Yanhao Wang, Wenmeng Zhou · 2025
Read Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions Conference: Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Proceedings of the ACM Web Conference 2025 (WWW ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/36964 by Zhiwen Li, Die Chen, Mingyuan Fan, Cen Chen, Yaliang Li, Yanhao Wang, Wenmeng Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15427.txt.
Introduction
Zhiwen Li and co-authors published this machine learning paper in 2025, presented at the ACM Web Conference in Sydney. It addresses a known hazard of image-generating diffusion models: they can produce unsafe images and reflect social biases. Filters that block toxic words are brittle, and erasing concepts by fine-tuning can damage normal output while still missing some harmful prompts.
The team's method works in the space of text embeddings, the numeric form of a prompt inside the model. They find a direction in that space that points toward safety, then steer every prompt's embedding to stay inside a safe region. A light adapter technique keeps the steering from disturbing unrelated meanings. On benchmark tests, the method cut unsafe output and bias more than the rival approaches it was compared with. It can also be layered on top of existing safety methods.
Catalog record
| CREATOR | Zhiwen Li, Die Chen, Mingyuan Fan, Cen Chen, Yaliang Li, Yanhao Wang, Wenmeng Zhou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15427
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions Conference: Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Proceedings of the ACM Web Conference 2025 (WWW ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3696410.3714912 ISBN: 979-8-4007-1274-6/25/04 CCS: Security and privacy Human and societal aspects of security and privacy CCS: Computing methodologies Artificial intelligence Zhiwen Li, Die Chen, Mingyuan Fan, Cen Chen, Yaliang Li, Yanhao Wang, Wenmeng Zhou Abstract . The remarkable ability of diffusion models to generate high-fidelity images has led to their widespread adoption. However, concerns have also arisen regarding their potential to produce Not Safe for Work (NSFW) content and exhibit social biases, hindering their practical use in real-world applications. In response to this challenge, prior work has focused on employing security filters to identify and exclude toxic text, or alternatively, fine-tuning pre-trained diffusion models to erase sensitive concepts. Unfortunately, existing methods struggle to achieve satisfactory performance in the sense that they can have a significant impact on the normal model output while still failing to prevent the generation of harmful content in some cases. In this paper, we propose a novel self-discovery approach to identifying a semantic direction vector in the embedding space to restrict text embedding within a safe region. Our method circumvents the need for correcting individual words within the input text and steers the entire text prompt towards a safe region in the embedding space, thereby enhancing model robustness against all possibly unsafe prompts. In addition, we employ Low-Rank Adaptation (LoRA) for semantic direction vector initialization to reduce the impact on the model performance for other semantics. Furthermore, our method can also be integrated with existing methods to improve their social responsibility. Extensive experiments on benchmark datasets demonstrate that our method can effectively reduce NSFW content and mitigate social bias generated by diffusion models compared to several state-of-the-art baselines. WARNING: This paper contains model-generated images that may be potentially offensive. 1. Introduction Recently, large-scale text-to-image diffusion models ( Rombach et al. 2022 ; Nichol et al. 2022 ) have attracted much attention due to their ability to generate photo-realistic images based on textual descriptions. However, considerable concerns about these models also arise because their generated content has been found to be possibly unsafe and biased, containing pornographic and violent content, gender discrimination, or racial prejudice ( Schramowski et al. 2023 ; Brack et al. 2023b ; Fan et al. 2024b ) . There have been two common types of approaches employed to address such concerns. One class of methods involves integrating some external safety validation mechanisms ( Ramesh et al. 2022 ; Rando et al. 2022 ; Midjourney 2022 ) , which harness classifiers to detect toxic input from users and reject them, with diffusion models. However, these mechanisms might be unreliable, as some prompt texts that do not explicitly contain Not Safe for Work (NSFW) content can still result in images with such content. Taking the Stable Diffusion (SD) model as an example, the prompt “a beautiful woman” may lead to the generation of an image of a nude woman ( Schramowski et al. 2023 ) . The other class of approaches seeks to construct more responsible diffusion models by training data cleaning, parameter fine-tuning, model editing, or intervention in the inference process. A naive method ( Rombach et al. 2022 ) is to filter out inappropriate content from the training data of diffusion models to prevent them from internalizing such content. Although effective, retraining models on new datasets can be computationally intensive and often leads to performance degradation ( O’Connor 2022 ) . Therefore, more efforts have been made to fine-tune parameters so that models can ‘forget’ undesirable concepts ( Heng and Soh 2023 ; Shen et al. 2024 ; Gandikota et al. 2023 ; Kumari et al. 2023 ) . However, the catastrophic forgetting problem can potentially arise when fine-tuning parameters. Meanwhile, another line of studies ( Gandikota et al. 2024 ; Orgad et al. 2023 ) seeks to selectively edit certain parameters of pre-trained models to construct a responsible image generation model. But these methods typically tailor the projection matrix within the cross-attention layers to specific target words, thus yielding suboptimal outcomes for other related but non-targeted words. Finally, a few methods ( Brack et al. 2023a ; Schramowski et al. 2023 ) leverage the principle of classifier-free guidance. They directly modify the denoising process of the original model to steer away from inappropriate content. Although these methods refrain from updating the model parameters, they may still impact the semantics of the original image and introduce additional overhead during the inference process. In summary, despite the above methods being effective to some extent, there are considerable gaps in ensuring the responsibility of diffusion models. In this paper, we endeavor to address the problem of responsible text-to-image generation using diffusion models from a different perspective. Generally, our approach focuses on manipulating the input text embedding to avoid generating inappropriate content. As the encoded text prompt is fed into the U-Net as a condition and plays a critical role in the image generation process, it can be used to identify the global semantic direction related to a certain concept in the embedding space. Accordingly, this direction can restrict the text embedding to a specific “safe region”, reducing the generation of harmful content in various contexts beyond the token level. We note that previous prompt-tuning methods ( Ruiz et al. 2023 ; Gal et al. 2023 ; Kim et al. 2023b ) have also attempted to train one or more pseudo-tokens in the CLIP embedding space in a supervised manner. Nevertheless, these pseudo-tokens are designed to symbolize specific concepts and are discontinuous in nature. The pseudo-tokens, along with the other words in the text prompt, are jointly encoded by the CLIP model before input into the U-Net for image generation. These approaches still operate at the token level to instruct the model in generating corresponding images. However, each token in the text prompt will contain information from other tokens. Therefore, attempting to encapsulate a concept such as “safety” within a single token typically fails to produce desirable outcomes. In the computational linguistics domain, some methods such as prefix-tuning ( Li and Liang 2021 ) consider generating continuous pseudo-tokens for specific tasks. Although these tokens are continuous, they are only used as a prefix added to the beginning of the input sentence to guide the language model in the autoregressive process. This differs from our goal of directing the embedding of the entire input text to a specific region. At the same time, the performance of the trained pseudo-tokens largely depends on the quality of the training data or the accuracy of the classifier. In general, existing prompt-tuning methods cannot be easily applied to prevent the generation of unsafe content and mitigate social biases within diffusion models. Toward this end, as shown in Figure 1 , we propose a novel self-discovery approach to identify the semantic direction in the embedding space, thus constraining the text prompt embedding within the safe region. Specifically, we utilize the classifier-free guidance technique that leverages internal knowledge of the diffusion model to learn a semantic direction vector. Then, we use