Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models

Arian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban · 2026

Read Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models by Arian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26332.txt.

Introduction

Arian Komaei Koma and three colleagues' 2026 attack, BEAP, targets unlearned image diffusion models without model weights, while evading safety filters. A language model iteratively generates prompts, guided by an embedding-aware search balancing concept presence, picture-text fit, and image quality. Across five unlearning methods, BEAP reaches a macro-averaged attack success rate of ninety-seven point eight percent, beating the white-box baseline by forty-one point six points. Counting failed searches at full budget, it averages thirty-two point one image queries per prompt.

Catalog record

CREATORArian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46221
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26332

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models Arian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban Abstract Machine unlearning aims to remove specific concepts from pretrained text-to-image diffusion models, yet several white- and black-box attacks have been introduced to make the model generate such unlearned concepts. These attacks, nevertheless, do not assume a realistic threat model, i.e. they either assume access to the model weights, or result in gibberish adversarial prompts that could be easily detected even through naive rule-based safeguarding. We aim to address this gap in this paper. We introduce BEAP —a black-box, embedding-aware adversarial prompting attack that leverages a large language model (LLM) to iteratively generate effective adversarial prompts and exploit such hidden vulnerabilities. BEAP performs an embedding-aware search in text space, combining multiple reward signals—unlearned concept presence, text–image alignment, and image quality—to refine generated prompts. Unlike previous attack methods, BEAP keeps its prompts undetectable to safety filters while producing high-quality images. Extensive experiments show that BEAP improves the Attack Success Rate (ASR) by more than 60% over prior methods, while requiring only an average of fifteen prompts per successful attack. Warning: This paper contains model outputs that may be offensive or upsetting in nature. 1 Introduction Text-to-image (T2I) diffusion models have rapidly advanced, enabling realistic image synthesis directly from text inputs. Models such as DALL·E 3 Betker et al. , Stable Diffusion Rombach et al. 2022 ; Podell et al. 2023 ; Esser et al. 2024 , and MidJourney Team 2023 show strong semantic understanding and flexible visual generation across diverse domains. However, because these systems are trained on large-scale web data, they can also be misused to generate inappropriate or unsafe content, raising significant ethical concerns Thiel 2023 ; Schramowski et al. 2023 ; Rando et al. 2022 . To mitigate these risks, recent diffusion models incorporate Not Safe For Work (NSFW) safeguards before or after generation to filter or neutralize problematic prompts Zhang et al. 2024c ; Li et al. 2024 ; Hanu and Unitary team 2020 ; CompVis 2022 ; Li 2022 . Yet leaning on such external filters is not a true fix for safety of these models, since they are model-agnostic and are post-hoc interventions. In fact prior studies, have already shown that these methods fall short when it comes to stopping Diffusion Models from producing harmful outputs Gandikota et al. 2023 ; Brack et al. 2023 ; Zhang et al. 2024a . Additionally, prior work has shown that these measures are easily evaded by jailbreaking attacks Huang et al. 2025 ; Yang et al. 2024b ; Dong et al. 2024 ; Deng and Chen 2023 . In T2I jailbreak attacks, an attacker modifies a disallowed NSFW prompt into a stealthy adversarial prompt that circumvents protections and results in the creation of the NSFW image. Hence these safety measures are not reliable to stop these models from producing unsafe outputs. To further alleviate this issue another line of work has explored machine unlearning Cao and Yang 2015 —the process of selectively removing specific knowledge or concepts from a trained model without retraining it from scratch. Unlike safeguarding methods that filter model’s input or outputs, unlearning aims to remove the unwanted concept or object itself from the model’ parameters, eliminating the underlying information and preventing the model from generating it while keeping the model’s overall generative ability intact. Methods such as ESD Gandikota et al. 2023 , SPM Lyu et al. 2024 , MACE Lu et al. 2024 , CA Kumari et al. 2023 , SalUn Fan et al. 2023 and SLD Schramowski et al. 2023 achieve this by fine-tuning or adjusting model parameters to redirect the removed concept toward a neutral alternative. While these approaches can effectively reduce explicit visual traces of undesired content, they often fail to guarantee complete forgetting, as parts of the unlearned concept may still remain encoded within the model’s parameters. George et al. 2025 ; Tsai et al. 2023 ; Zhuang et al. 2023 . We demonstrate this problem in Section A as unlearned models can still regenerate unlearned concepts through simple prompting or paraphrasing, indicating that parts of the removed knowledge remain within the model’s parameters. Follow-up works such as QF-Attack Zhuang et al. 2023 , Generate-or-Not Zhang et al. 2024b , and Ring-a-Bell Tsai et al. 2023 show that these residual traces can be exploited. Specifically, Generate-or-Not attacks are white-box and require access to model weights or gradients, which limits their practicality. Others like Ring-a-Bell Tsai et al. 2023 and QF-Attack perform embedding-space search, but their method additionally requires information about the target model’s text encoder to execute the attack effectively, which is unrealistic in real-world scenarios. In addition, all of the mentioned attack methods generate semantically meaningless or low-quality prompts with abnormally high perplexity (i.e., the text is assigned very low probability by a standard language model), which makes them easily detectable by simple safeguards such as perplexity-based or gibberish detectors Jindal 2021 . These findings show that current attacks are not effectively deployable in real-world scenarios, as they either rely on full access to model parameters, require prior knowledge of the target model, or fail when confronted with even basic safety mechanisms. To overcome these limitations, we introduce BEAP , a black-box attack framework that leverages a large language model (LLM) to search directly in the natural text space and generate coherent adversarial prompts. Because LLMs operate over fluent natural language, BEAP can systematically explore a rich manifold of semantically meaningful paraphrases, without resorting to explicit optimization in text-embedding space that drives prompts into unnatural or nonsensical regions. Our attack iteratively prompts the unlearned model, and at each iteration our LLM refines the previous prompts through reward feedback based on a concept-presence detector, text–image alignment, and image quality. This process yields natural-language prompts that remain undetected by safety filters while still exposing the supposedly unlearned concept. To further enhance this process, we provide the model with a bag of surrogate terms whose embedding are close to that of the concept we aim to attack and encourage the model to use these alternatives. By combining natural-language prompting, reward-guided refinement, and embedding-aware search, our framework exposes traces of unlearned knowledge without requiring access to model weights or gradients which reveal persistent vulnerabilities in unlearned diffusion models. Our main contributions are as follows: • We propose the first LLM-driven fully black-box attack for text-to-image diffusion models that generates, human-readable adversarial prompts. • We design a reward-guided iterative search strategy using multiple feedback signals to refine the adversarial prompt generation process. • We introduce an embedding-aware similarity component that integrates semantic proximity to the forgotten concept, improving the effectiveness of text-space exploration. • We show that current unlearning methods remain fundamentally vulnerable: our attack generates undetectable adversarial prompts that produce high-quality images revealing residual, supposedly removed knowledge in diffusion models, underscoring that existing unlearning defenses fail in realistic, real-world settings. 2 Related Works 2.1 Unlearning Methods Machine unlearning (MU) Nguyen et al. 2025 Ginart et al. 2019 Cao and Yang 2015 aims to remove specific concepts, styles, or data influences from a trained text-to-image (T2I) diffusion model without retraining it from scratch. Early methods such as ESD Gandikota et al.

Read the complete text