Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models

Chaoyi Zhu, ZAITANG LI, Renyi Yang, Robert Birke, Pin-Yu Chen, Tsung-Yi Ho, Lydia Y. Chen · 2025

Read Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models by Chaoyi Zhu, ZAITANG LI, Renyi Yang, Robert Birke, Pin-Yu Chen, Tsung-Yi Ho, Lydia Y. Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06018.txt.

Introduction

Chaoyi Zhu and six colleagues show that AI image watermarks can be forged without any optimization step, in a 2025 paper. Watermarks are meant to prove an image came from a particular generator, but earlier attacks already copied them with adversarial tuning. Their attack, called PnP, instead re-runs the image through an off-the-shelf regenerative diffusion model, which carries the watermark across by itself.

The method uses the watermark hidden in the source image plus the look and text context of the target image to guide regeneration. Across 24 combinations of models, data, and watermark schemes, the forgeries reached up to 100 percent detectability while keeping the best visual quality. The authors warn this widens the threat to watermark-based governance.

Catalog record

CREATORChaoyi Zhu, ZAITANG LI, Renyi Yang, Robert Birke, Pin-Yu Chen, Tsung-Yi Ho, Lydia Y. Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06018

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models Chaoyi Zhu, ZAITANG LI, Renyi Yang, Robert Birke, Pin-Yu Chen, Tsung-Yi Ho, Lydia Y. Chen Abstract Watermarking becomes one of the pivotal solutions to trace and verify the origin of synthetic images generated by artificial intelligence models, but it is not free of risks. Recent studies demonstrate the capability to forge watermarks from a target image onto cover images via adversarial optimization without knowledge of the target generative model and watermark schemes. In this paper, we uncover a greater risk of an optimization-free and universal watermark forgery that harnesses existing regenerative diffusion models. Our proposed forgery attack, PnP (Plug-and-Plant), seamlessly extracts and integrates the target watermark via regenerating the image, without needing any additional optimization routine. It allows for universal watermark forgery that works independently of the target image’s origin or the watermarking model used. We explore the watermarked latent extracted from the target image and visual-textual context of cover images as prior to guide sampling of the regenerative process. Extensive evaluation on 24 scenarios of model-data-watermark combinations demonstrates that PnP can successfully forge the watermark (up to 100% detectability and user attribution), and maintain the best visual perception. By bypassing model retraining and enabling adaptability to any image, our approach significantly broadens the scope of forgery attacks, presenting a greater challenge to the security of current watermarking techniques for diffusion models and the authority of watermarking schemes in synthetic data generation and governance. Our code is available at the repository: https://github.com/chaoyitud/PnP-Watermark-Forgery . 1 Introduction The rapid advancement of generative models 10521640 has led to a significant increase in the creation of high-quality images. As synthetic images proliferate, tracking their provenance bing ; synthid is becoming ever crucial. Tracing their usages and verifying their origins are key steps, especially in case of misuse or misconduct. As a result, the demand for effective watermarking techniques to protect intellectual property and ensure the authenticity of generated content has become increasingly important. Watermarks are a promising solution for embedding information into images, enabling the identification of the creator or source of the content. Post-processing watermarks PereiraP00 ; BiSHYH07 studied for decades add a covert marker into images as noise on the pixels, but impact visual quality and have low resilience to post-editing attacks. Semantic watermarking jiang2024watermarkbasedattributionaigeneratedcontent overcomes such limits by adding the watermark at the latent space of synthetic data, often during the sampling/generation process. For instance, TreeRing treerings2023 and Gaussian Shading yang2024gaussian are watermarks for images which are embedded at the initial noise of the diffusion process. This allows the watermark to maintain minimum impact to the image quality, while remaining detectable under various post-editing attacks, e.g., paraphrasing and translation attacks. Furthermore, semantic watermarks can be straightforwardly integrated into existing diffusion models. Forgery is one of the long standing challenges for data governance, where adversaries can falsify the data without legit authorization. Recent studies mueller2024forgery demonstrate that the aforementioned semantic watermark for synthetic images can be extracted and forged onto different images by using proxy diffusion models. Reprompt mueller2024forgery , see Fig. 1 (a), leverages a proxy diffusion model, specifically text-to-image, and uses different prompts and the extracted “implicit” watermark to generate new images. Imprint mueller2024forgery , see Fig. 1 (b), demonstrates a stronger forgery attack on general diffusion models by explicitly embedding the watermark through adversarial optimization. However, reprompting fails to forge watermarks across any images, while imprinting produces watermarked outputs with notable quality degradation and incurs long optimization time, rendering it impractical for large-scale use. In this work, we unveil Plug-and-Plant ( PnP ), a watermark forgery attack for images that are generated by diffusion models with either semantic watermarks. Our key proposition is to leverage existing regenerative diffusion models, such as super-resolution chen2310pixart ; wang2024exploiting , face restoration lin2024diffbir , or watermark removal liu2024image , as a backbone for the forgery, without needing any additional finetuning, see Fig. 1 (c). PnP allows for universal watermark forgery that works independently of the source image’s origin or the watermarking model used. PnP is a light-weight forgery method that uses a controllable regenerative diffusion model to forge the watermark onto either synthetic or real-world images. Specifically, we first extract the watermarked latent from the target image and then explore such latent as prior to regenerate the cover image under its visual-textual guidance as constraints. The forged watermark and key characteristics of cover images are preserved in the generated images. Thus, PnP has two-fold advantages. First, as we are an optimization-free solution, the forged images can be generated up to 50 times faster than existing methods. Secondly, the regenerative diffusion model allows for improved image resolution and quality, overcoming the quality degradation typically associated with adversarial optimization-based methods. Figure 1 : Comparison of different Watermark forgery methods for diffusion models (DM): (a) Reprompt forges the watermarks on text-to-image diffusion models; (b) Imprint forges watermarks on cover images using adversarial optimization on the proxy DM; and (c) PnP , our proposed optimization-free forgery uses controllable regenerative DM. Our findings underline the ease in forging watermarks with off-the-shelf generative models. This allows even users with no forgery expertise to launch such attacks. Hence, model owners cannot reliably depend on current watermarking approaches for data governance, suggesting demands on more advanced and forgery-proof watermarking methods. Our specific technical contributions are: • We propose, PnP , a plug-and-play universal watermark forgery attack that forges the watermark of a target image onto any arbitrary cover image, by harnessing regenerative diffusion models. • PnP extracts the forged watermark latents and exploits both the forged latents and visual-textual contexts to condition the regeneration of cover images with the forged watermarks. • Extensive experimental validation, across multiple state-of-the-art watermarking techniques and regenerative diffusion models, demonstrates that, PnP not only forges watermarks successfully but also improves the overall image perceptuality. 2 Background and Related Studies 2.1 Image Watermarking Methods Image watermarking methods can be broadly classified into two categories: post-processing and in-processing methods. Post-processing watermarks modify the final output, embedding the watermark in the image content PereiraP00 ; BiSHYH07 . In-processing watermarks, on the contrary, are applied during image generation and embed the watermark in the model’s internal representation. A unique subcategory of in-processing methods is semantic watermarking . In this approach, the watermark is embedded directly into the initial latent space of the image. This latent representation carries the watermark through the image generation process, ensuring its persistence. Moreover, the watermark can be recovered from the generated image using a diffusion inversion process. We focus on two popular semantic watermarks: Tree-Ring treerings2023 and Gaussian Shading yang2024gaussian . Tree-Ring watermarking is a pioneering semantic watermarking method that embeds a unique pattern into the Fourier domain of the initial latent space. This pattern consists of concentric rings arranged in a tree-like structure. Watermark detection involves the diffusion inversion process, followed by evaluating the distance between the recovered initial latent space and the predefined tree-ring structure in the Fourier domain. An image is watermarked if the distance falls below a model-specific threshold Gaussian Shading watermarking embeds a secret code by manipulating the quadrants of the initial latents within the Gaussian distribution. During detection, the watermark is recovered by inverting the diffusion process and checking whether the

Read the complete text