When There Is No Decoder: Removing Watermarks from Stable Diffusion Models in a No-box Setting

Xiaodong Wu, Tianyi Tang, Xiangman Li, Jianbing Ni, Yong Yu · 2025

Read When There Is No Decoder: Removing Watermarks from Stable Diffusion Models in a No-box Setting by Xiaodong Wu, Tianyi Tang, Xiangman Li, Jianbing Ni, Yong Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03646.txt.

Introduction

Hidden marks placed in AI-made images can be stripped out, Xiaodong Wu and co-authors show in a 2025 study. Such marks tag where an image came from, to help spot false or harmful content. The team test marks built into the image model itself, with no access to the mark reader.

Three attacks are tried: one based on image edges, one that blurs the image, and one that retrains the model. Simple edge tricks fail, but blur and retraining work well. The best attack cut mark spotting to about half. Tests of stronger guards still left mark reading below a safe level, the authors report.

Catalog record

CREATORXiaodong Wu, Tianyi Tang, Xiangman Li, Jianbing Ni, Yong Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-03646

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When There Is No Decoder: Removing Watermarks from Stable Diffusion Models in a No-box Setting Xiaodong Wu, Tianyi Tang, Xiangman Li, Jianbing Ni, Yong Yu Abstract Watermarking has emerged as a promising solution to counter harmful or deceptive AI-generated content by embedding hidden identifiers that trace content origins. However, the robustness of current watermarking techniques is still largely unexplored, raising critical questions about their effectiveness against adversarial attacks. To address this gap, we examine the robustness of model-specific watermarking, where watermark embedding is integrated with text-to-image generation in models like latent diffusion models. We introduce three attack strategies: edge prediction-based, box blurring, and fine-tuning-based attacks in a no-box setting, where an attacker does not require access to the ground-truth watermark decoder. Our findings reveal that while model-specific watermarking is resilient against basic evasion attempts, such as edge prediction, it is notably vulnerable to blurring and fine-tuning-based attacks. Our best-performing attack achieves a reduction in watermark detection accuracy to approximately 47.92%. Additionally, we perform an ablation study on factors like message length, kernel size and decoder depth, identifying critical parameters influencing the fine-tuning attack’s success. Finally, we assess several advanced watermarking defenses, finding that even the most robust methods, such as multi-label smoothing, result in watermark extraction accuracy that falls below an acceptable level when subjected to our no-box attacks. 1 Introduction Although generative AI holds the potential to boost efficiency and tackle capacity limits, it also raises critical questions about human creativity, originality, and copyright [ 5 , 26 ] . Distinguishing AI-generated content (AIGC) from human-generated content is increasingly difficult, complicating copyright issues, especially as misuse of models like Stable Diffusion [ 22 ] on social media leads to fake images and face-swapping of celebrities. To counter such misuse, watermarking techniques [ 30 , 27 ] enable the traceability of AIGC by embedding identifiable marks. These watermarks, created by a pre-trained encoder and detected by a corresponding decoder, verify an image’s origin by embedding and later recognizing predefined watermarks. Most watermarking approaches [ 8 ] rely on an autoencoder structure with encoder and decoder models and can be divided into model-specific and data-specific techniques. Model-specific methods [ 3 , 28 ] integrate the encoder directly into the generative model, producing watermarked images during inference without additional processing. Data-specific methods [ 32 , 25 ] apply the watermark after generation, keeping the encoder separate from the generative model. Despite their potential, watermarking techniques are vulnerable to adversarial attacks. Simple evasion methods, such as cropping, rotation, or brightness adjustment [ 1 ] , reduce watermark visibility but often introduce noticeable image alterations, limiting their real-world applicability. More sophisticated attacks, often performed in a white-box setting, leverage full access to the watermark decoder. For example, methods proposed by Jiang et al. [ 10 ] and Lukas et al. [ 17 ] use gradient information to evade the detection. However, white-box attacks are rarely feasible in practice due to strict access restrictions on proprietary watermarking decoders. To address this, black-box attacks are proposed with the attempt to circumvent these access restrictions; however, they come with their own set of limitations. For instance, Jiang et al.’s HopSkipJump [ 2 ] method requires frequent queries to the decoder, which undermines stealth by creating identifiable patterns of activity that could be flagged as suspicious. Kassis et al.’s spectral optimization method [ 11 ] is another black-box approach that can effectively remove watermarks; however, it often causes slight visual degradation, reducing the overall quality and authenticity of the image. In conclusion, existing approaches have limitations due to their reliance on varying degrees of decoder access. None of these methods address a no-box setting, where watermark removal could be achieved without any dependence on decoder access, highlighting a gap in practical and minimally invasive solutions for bypassing watermarks in generative models. This paper examines the robustness of model-specific text-to-image watermarking techniques in a no-box setting, where attackers have no access to the watermark decoder. We propose three attacks aimed at removing watermarks while preserving image quality, assessing watermark resilience under these real-world constraints. First, we introduce an edge-based attack that manipulates image edges to disrupt watermarks, which cannot remove the watermark without damaging the quality of the target image. Next, we develop a blurring technique that applies box blurring to remove the watermark, followed by deblurring to restore image quality, achieving a watermark detection accuracy of around 0.5 while maintaining visual fidelity. Finally, we explore a fine-tuning attack, leveraging the widespread practice of fine-tuning generative models for custom applications (e.g., specialized fields or unique styles). This approach modifies model weights using a surrogate watermark decoder, effectively erasing watermarks with no noticeable loss in image quality. Unlike previous research, which often depends on a white-box setting (with full decoder access) or a black-box setting (with query-based decoder access), our approach introduces a novel framework that completely removes reliance on the decoder. Through this no-box analysis, we evaluate the effectiveness of each attack method and provide insights into the inherent vulnerabilities of model-specific watermarking, contributing a new perspective to watermark resilience research in generative AI systems. Our contributions are as follows: • We propose and implement three distinct attack strategies: edge-based, box blurring, and fine-tuning, to systematically assess the robustness of generative watermarking techniques in a no-box setting. • Through extensive experiments, we show that while current watermarking methods withstand simple edge-based attacks, they exhibit significant vulnerabilities to box blurring and fine-tuning, resulting in a marked drop in watermark bit accuracy to 47.92%. • We evaluate three recently proposed defense methods aimed at enhancing watermark robustness, finding that although they mitigate some attack impact, watermark extraction accuracy still falls below an acceptable threshold, underscoring the effectiveness of our no-box attacks. 2 Related Works In this section, we review the existing watermark techniques and attacks on the generative text-to-image models. 2.1 Watermarking Techniques on Text-to-image Models To protect copyright for images generated from text-to-image models, invisible watermarks can be embedded using autoencoder networks, where an encoder embeds a watermark and a decoder later extracts it. Watermarking methods are either model-specific or data-specific, depending on whether the encoder is integrated with the generative model. In model-specific methods [ 16 , 19 ] , the encoder is needed only during training, and the watermarks are embedded directly into the images at inference. A pioneering approach using this method was proposed by Fernández et al. [ 3 ] , who integrated the encoder into the state-of-the-art text-to-image model, Stable Diffusion. Inspired by this work, Xiong et al. [ 28 ] designed an improved network capable of embedding a flexible, user-assigned message into each generated image, rather than generating images with the same watermark once the diffusion model is fine-tuned. This improvement was achieved by appending additional modules to the original diffusion model, enabling the embedding of messages during the generation process. In data-specific methods [ 18 , 15 ] , the encoder is not integrated into the generative model itself. Instead, an original image is first generated by a clean generator and then watermarked by a pre-trained encoder [ 12 ] . Zhu et al. [ 32 ] pioneered this approach with their HiDDeN framework, which watermarks images using a convolutional neural network (CNN)

Read the complete text