Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch

Qichen Zhao, Shengfang Zhai, Xinjian Bai, Qingni Shen, Qiqi Lin, Yansong Gao, Zhonghai Wu · 2026

Read Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch by Qichen Zhao, Shengfang Zhai, Xinjian Bai, Qingni Shen, Qiqi Lin, Yansong Gao, Zhonghai Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13028.txt.

Introduction

Qichen Zhao and co-authors test if image protections survive when an attacker uses an unlike pipeline. Such protections add small noise to images to block bad edits by diffusion tools. Owners cannot pick the tools an attacker runs after release. Their study tries two purifiers that clean the file first. EditorClean rebuilds the image with a diffusion transformer to exploit that gap. Once cleaned, the guard signal was mostly gone and edits worked freely.

Catalog record

CREATORQichen Zhao, Shengfang Zhai, Xinjian Bai, Qingni Shen, Qiqi Lin, Yansong Gao, Zhonghai Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76911
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13028

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch Qichen Zhao, Shengfang Zhai, Xinjian Bai, Qingni Shen, Qiqi Lin, Yansong Gao, Zhonghai Wu Abstract Diffusion models enable high-fidelity image editing, but they can also be misused for unauthorized style imitation and harmful content generation. To mitigate such risks, proactive image protection methods embed small, often imperceptible adversarial perturbations into images before sharing, aiming to disrupt downstream editing and fine-tuning. In realistic post-release settings, content owners cannot control downstream reprocessing or the choice of diffusion-based editing pipelines. Consequently, protections optimized against a surrogate model may break down when attackers purify or edit images using a mismatched pipeline built on a different model. Prior purification methods can weaken protections, but they often trade off effective perturbation removal against preserving image utility, and they rarely evaluate model architectural mismatch explicitly. We introduce a unified purification framework after release to evaluate protection survivability under model mismatch. We propose two practical purifiers: VAE-Trans , which corrects protected images via latent-space projection, and EditorClean , which performs instruction-guided reconstruction using a Diffusion Transformer to leverage architectural heterogeneity. Both operate without access to protected images or defense internals. We conduct extensive experiments on 2,100 editing tasks across six representative protection methods. EditorClean consistently restores editability. Compared to unpurified protected inputs, it improves PSNR by 3–6 dB and reduces FID by 50–70% on downstream edits. Compared to prior purification baselines, it typically achieves an additional about 2dB PSNR gain and 30% lower FID. Our findings reveal a largely purify-once, edit-freely failure mode: once an attacker succeeds in purification, the protective signal is largely erased, enabling subsequent unrestricted editing. This is primarily due to the limited cross-model transferability of perturbations and the reconstructive nature of diffusion-based pipelines operating with heterogeneous models. Overall, these results highlight the importance of evaluating robustness under model mismatch and the need to design adversarial protections that remain effective against heterogeneous attackers. 2 2 footnotetext: Corresponding authors. 1 Introduction Diffusion models have become a foundational technique for image synthesis and editing [ 40 , 14 , 33 ] , enabling high-fidelity semantic manipulation, inpainting, and style transfer from natural language instructions [ 28 , 12 , 1 , 48 ] . These capabilities substantially expand creative and editing workflows, but they also introduce new avenues for misuse, such as unauthorized style imitation and harmful content generation. As a result, protecting visual content against downstream misuse has become an increasingly important security concern. Proactive image protection has emerged as a promising defensive strategy to address this challenge [ 36 , 39 , 23 , 22 , 42 , 5 ] . By embedding small and often imperceptible adversarial perturbations into images prior to release, these methods aim to disrupt subsequent generative editing or model fine-tuning. However, once images are released, they may be further processed or modified by users in uncontrolled ways [ 43 , 37 , 16 ] . In practice, attackers can freely collect protected images and reprocess them using a broad spectrum of tools, ranging from simple image transformations to generative editing pipelines built on heterogeneous model architectures [ 37 , 16 , 2 , 51 ] . Existing protection methods are typically optimized against a specific surrogate model and evaluated in matched settings [ 36 , 22 , 23 , 5 ] . As a result, their effectiveness is not evaluated in practice when applied across mismatched diffusion-based editing pipelines. A key threat in this setting arises from purification attacks, in which attackers attempt to remove or attenuate protective perturbations before performing unauthorized edits or downstream fine-tuning. Prior studies [ 37 , 16 , 29 , 2 , 51 ] show that even simple preprocessing or reconstruction operations can weaken protections. However, existing approaches often struggle to simultaneously eliminate protective signals and preserve image utility. They also rarely evaluate purification explicitly under the mismatch that matters in practice: the defender crafts perturbations against a surrogate model, but the attacker may purify the image before editing it with a different model. Together, these limitations highlight the need for a more systematic and realistic evaluation of protection survivability under purification-based attacks. We address these gaps by proposing a unified purification framework after release for evaluating perturbation-based image protection [ 9 ] . In our formulation, defenders cannot observe or constrain the attacker-side pipeline after release, while attackers may freely choose heterogeneous models and preprocessing pipelines [ 16 , 2 , 51 ] . Our framework models an attacker who either directly edits the protected image x_{\mathrm{adv}} using a different diffusion-based editing pipeline, or prepends a purification operator \mathcal{P} before editing. This formulation enables systematic measurement of protection survivability across purification strategies and model architectures by focusing on editability after release. Our key observation is that while protective perturbations can be highly effective on the surrogate models they are optimized against, they often generalize poorly across architectures. Leveraging this observation, we propose two practical purification methods to instantiate the framework. VAE-Trans fine-tunes a VAE encoder to correct protected images through latent-space projection, probing non-robustness to distribution shift within the same model family. EditorClean formulates purification as instruction-guided reconstruction using a Diffusion Transformer [ 30 ] , adapting the ICEdit in-context editing framework [ 50 ] to adversarial purification. It exploits architectural mismatch by leveraging the limited transferability of perturbations optimized for a surrogate model, showing that model mismatch can already be effective in practical scenarios. Both methods are trained solely on public data and require no access to protected images or defense internals, making them practical in realistic settings after release. Following the DiffusionGuard evaluation protocol [ 5 ] , we conduct extensive experiments across six representative protection methods: PhotoGuard [ 36 ] , AdvDM [ 22 ] , MIST [ 23 ] , SDS [ 47 ] , DiffusionGuard [ 5 ] , and AdvPaint [ 19 ] . We evaluate 2,100 image editing tasks spanning celebrity portraits, objects, and animals. Our experiments show that purification under model mismatch can substantially restore editability. Compared with unpurified protected inputs, EditorClean improves PSNR by about 3–6 dB and reduces FID by about 50–70% across all six defenses. Relative to existing purification baselines, it further boosts PSNR by about 1–3 dB and lowers FID by 10–40%, substantially narrowing the gap to clean-image edits. Our experiments reveal a severe post-release failure mode of current proactive image protection: once an image is successfully purified, the protective signal is largely removed, enabling unrestricted downstream editing thereafter. In summary, this work makes the following contributions: • We show that protective perturbations transfer poorly across heterogeneous models, and introduce a benchmark that comprehensively evaluates the protection effectiveness under model mismatch. • We propose VAE-Trans and EditorClean , two practical purifiers trained on public data. We show they substantially restore the editability of protected images and can further restore the utility of unlearnable examples. • We highlight the practical vulnerability of protective perturbations where heterogeneous models effectively act as purifiers, enabling attackers to eliminate protections and then edit freely, motivating more realistic evaluation and defense design under model mismatch. 2 Related Work 2.1 Image Generation and Editing Diffusion probabilistic models [ 40 , 14 , 8 , 7 ] have emerged as

Read the complete text