Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss Minsu Gong, Nuri Ryu, Jungseul Ok, Sunghyun Cho Abstract Recent advances in image editing leverage latent diffusion models (LDMs) for versatile, text-prompt-driven edits across diverse tasks. Yet, maintaining pixel-level edge structures—crucial for tasks such as photorealistic style transfer or image tone adjustment—remains as a challenge for latent-diffusion-based editing. To overcome this limitation, we propose a novel Structure Preservation Loss (SPL) that leverages local linear models to quantify structural differences between input and edited images. Our training-free approach integrates SPL directly into the diffusion model’s generative process to ensure structural fidelity. This core mechanism is complemented by a post-processing step to mitigate LDM decoding distortions, a masking strategy for precise edit localization, and a color preservation loss to preserve hues in unedited areas. Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing. Our code will be publicly released at https://github.com/gongms00/SPL . † † footnotetext: ∗ Equal contribution. † Work done while the author was at POSTECH. 1 Introduction Image editing has seen remarkable progress, yet developing a universal image editing method that preserves pixel-level edge structures of the input image remains a long-standing challenge. By pixel-level edge structure, we mean the fine-grained spatial discontinuities in intensity that define object contours and texture details. Maintaining these structures is crucial for various tasks like relighting, tone adjustment, image harmonization, photorealistic style transfer, time-lapse generation, seasonal or weather changes, and background replacement. In these editing tasks, even minor structural distortions in the edited image, relative to the input, can compromise the intention of the edit. Traditionally, pixel-level-edge-preserving image editing tasks have been approached individually using tailored methods [ 77 , 51 , 30 , 56 , 29 ] . While these methods achieve high structural fidelity, they face two significant drawbacks. First, none of these methods offer a unified framework that can handle multiple editing tasks, which results in task-specific pipelines with limited generalization. Second, lacking a generative prior, they struggle to introduce additional details or creative variations aligned with desired edits. Recent image editing methods leverage robust generative priors such as large-scale text-to-image latent diffusion models (LDMs) [ 53 , 49 ] . This has enabled general image editing through text-based instructions, allowing them to address multiple tasks within a single framework [ 4 , 20 , 6 , 25 , 40 , 67 ] . While these LDM-based editing methods overcome the limitations of generalization and detail generation in traditional image editing methods, they often fall short of preserving the pixel-level structure of the input image even when strict preservation is needed. This challenge mainly stems from the absence of explicit guidance to maintain pixel-level edge structures. Moreover, LDM-based editing methods also suffer from structural distortions arising from the lossy RGB-to-latent mapping, exacerbating structure preservation difficulties. Figure 1 : Structure-preserving edit. Our method achieves pixel-level structural fidelity without compromising the intended edit. To tackle these challenges, we propose Structure Preservation Loss (SPL), a novel loss function designed to enforce pixel-level edge structure fidelity between the input and edited output. The key idea behind the structure preservation loss is to leverage the notion of local linear models (LLMs) in image processing [ 19 , 33 , 80 , 18 ] to capture and maintain structural relationships between images. We incorporate the structure preservation loss into the sampling process of diffusion models, achieving high structure fidelity to the input image without harming its editing capabilities as demonstrated in Fig. 1 . Our approach is plug-and-play and does not require any additional training on the base LDM-based editing model. Furthermore, the modularity of our loss offers users fine-grained control to tailor structural constraints to their specific intent. We also introduce a post-processing step that refines the decoded output to correct structural distortions arising from the latent-to-RGB mapping, ensuring pixel-level edge structure preservation. Additionally, we propose a simple masking strategy that generates a high-resolution edit mask based on the edit text prompt to enable precise edit localization. Our method establishes a general framework for structure-preserving editing, unifying a class of tasks traditionally handled by separate, specialized methods. By integrating our loss, conventional LDM-based editors can now overcome their inherent limitation in preserving pixel-level structures, significantly expanding their applicability and making them more versatile, all-purpose editing tools. Experiments conducted on various editing tasks confirm that our method successfully produces edited images that maintain the input image’s pixel-level edge structure while being faithful to the provided edit prompts. To summarize, our main contributions are as follows: • We introduce a novel structure preservation loss that maintains the pixel-level edge structure of the input image during image editing. • We propose a training-free latent-diffusion-model-based image editing method that allows pixel-level edge structure preservation while leveraging latent-diffusion models’ generative image editing capabilities. • We suggest a simple strategy to acquire a high-resolution edit mask from the diffusion model’s internal features to support local editing. • We demonstrate state-of-the-art quality in pixel-level structure-preserving image editing through comprehensive experiments. 2 Related Work Structure-Preserving Image Editing Prior to the emergence of diffusion models, structure-preserving image editing tasks were predominantly tackled using specialized, task-dependent methods. Techniques for image relighting leveraged physics-based priors [ 12 , 66 , 77 , 44 , 28 , 58 , 42 ] , while tone adjustment relied on color transformations or look-up tables [ 7 , 15 , 70 , 17 , 27 , 64 , 73 , 31 ] . Harmonization and background replacement employed image gradient or color statistics manipulation [ 23 , 47 , 59 , 61 , 10 , 48 , 52 , 68 , 38 , 9 ] , and style transfer utilized image statistics or deep feature matching [ 52 , 48 , 13 , 34 , 57 , 39 , 71 , 36 , 9 ] . While these task-specific methods often demonstrated high structural fidelity within their respective domains, they are inherently limited in scope and generalization due to their domain-specific assumptions. Diffusion-Based Image Editing Large-scale text-to-image LDMs [ 53 , 49 ] have enabled versatile image editing methods, offering text-driven edits with a single pre-trained model [ 25 , 3 , 26 , 4 , 14 , 76 , 40 , 11 , 2 , 41 , 63 , 24 , 21 , 20 , 62 , 45 , 6 , 67 ] . These methods broadly fall into training-based and training-free approaches, both facing challenges in preserving fine-grained structural details from the input image. Training-based approaches include methods that overfit to a single image [ 25 , 3 ] or learn from editing datasets [ 26 , 4 , 14 , 76 ] . However, overfitting to a single image incurs a significant computational overhead, limiting its scalability, while dataset-trained models are limited by dataset biases. Furthermore, the training objectives of training-based methods do not guarantee pixel-level fidelity. Training-free methods leverage a pre-trained diffusion model without additional training or fine-tuning. Early approaches edit images by steering the generative process of diffusion models to match the edit prompt by modifying the intermediate noisy latent representations [ 40 , 11 , 2 ] . However, these techniques are often limited in editing performance and struggle to retain the original image’s structure. More recent training-free methods have improved on fidelity by manipulating the diffusion model’s denoising U-Net attention maps [ 20 , 62 , 45 , 6 , 67 ] . For example, Prompt-to-Prompt [ 20 ] reuses cross-attention maps generated during input image generation to maintain its global layout, while Plug-and-Play [ 62 ] injects self-attention to preserve local structure. Unifying these ideas, InfEdit [ 67 ] combines controls for both attention types. However, lacking an explicit pixel-level guidance, these methods only allow for coarse structure preservation. Full-Reference Metrics for Structural Fidelity While full-reference metrics are often used as loss functions to preserve fidelity to a reference, they are ill-suited for structure-preserving editing tasks where appearance (e.g., color, brightness, contrast) is intentionally modified. These metrics typically fail to disentangle structural content from appearance attributes. For instance, pixel-wise losses like L1 and MSE penalize valid appearance changes, while perceptual metrics like LPIPS [ 75 ] do not explicitly enforce structural accuracy. SSIM [ 65 ] does account for structure yet remains sensitive to changes in brightness and contrast, as it evaluates luminance and contrast alongside structural correlation. Figure 2 : Motivation of the structure preservation loss . (Fig 2-1) An edited image may contain both structure-preserving (marked in blue) and structure-breaking regions (marked in orange). Our approach is motivated by the local linear model’s ability to analyze these regions on a local window-by-window basis. (Left) When structure is preserved, the model finds an accurate linear fit, resulting in low error. (Right) When structure is broken, the model fails to find a good fit, producing a high error that signals the distortion. (Fig 2-2) Unidirectional structural comparison fails to fully capture mutual differences, motivating the bidirectional design of our structure preservation loss. 3 Method Our method performs LDM-based image editing while preserving the pixel-level edge structure of the input. To achieve this, we first introduce a novel structure preservation loss that explicitly quantifies structural discrepancies ( Section 3.1 ). We then integrate SPL into the LDM’s denoising pipeline ( Section 3.2 ). Additionally, to offer users precise control over which regions should remain unchanged, we also propose a text-driven edit mask generation scheme to be used in conjunction with the structure preservation loss ( Section 3.3 ). 3.1 Structure Preservation Loss Local Linear Model for Structure Preservation Our structure preservation loss is designed to quantify structural differences between a source image I^{S} and an edited output I^{E} . To achieve this, our loss is grounded in the local linear model, which is well-established for its structure awareness and has been applied to various tasks that require accurate structure preservation, e.g., alpha matting, edge-aware filtering, dehazing, joint upsampling, and super-resolution [ 19 , 33 , 80 , 18 ] . The key assumption is that for structure to be preserved, the edited image I^{E} must be a local linear transformation of the source image I^{S} . We visualize this idea in Fig. 2 -1. We formalize this relationship mathematically within a local window \omega_{k} as: I^{S}_{i}=a_{k}\,I^{E}_{i}+b_{k},\hskip 10.00002pt\forall i\in\omega_{k}, (1) where a_{k} and b_{k} are coefficients assumed to be constant within the window centered at pixel k . A key property of this model is that it preserves edges, since taking the gradient of Eq. 1 yields \nabla I^{S}_{i}=a_{k}\nabla I^{E}_{i} . The coefficients are obtained by solving a least-squares problem, yielding the closed-form solution: \displaystyle a_{k}= \displaystyle\frac{\frac{1}{|\omega_{k}|}\sum_{i\in\omega_{k}}I^{E}_{i}I^{S}_{i}-\mu^{E}_{k}\mu^{S}_{k}}{\left(\sigma^{E}_{k}\right)^{2}+\rho},~~~~\textrm{and} (2) \displaystyle b_{k}= \displaystyle\mu^{S}_{k}-a_{k}\mu^{E}_{k}, (3) where \mu_{k}^{E} and \mu_{k}^{S} are the mean intensities of I^{E} and I^{S} in \omega_{k} , respectively. \sigma^{E}_{k} is the standard deviation of I^{E} in \omega_{k} , and |\omega_{k}| is the number of pixels in the window. \rho is a small constant to avoid degenerate solutions. We provide a detailed derivation in the supplementary. While previous works employ the local linear model as a filtering mechanism, our key contribution is to adapt it as a differentiable loss. This loss serves as an explicit structural guide for the diffusion model’s generative process, ensuring fidelity across a wide range of editing tasks. Structure Preservation Loss Based on Eq. 1 , we define the directional structure difference between I^{E} and I^{S} as: D_{k}(I^{E},I^{S})=\frac{1}{|\omega_{k}|}\sum_{i\in\omega_{k}}\left(I_{i}^{E\to S}-I_{i}^{S}\right)^{2}, (4) where I_{i}^{E\to S} represents a linearly transformed local patch of I^{E} , defined as I_{i}^{E\to S}=a_{k}I_{i}^{E}+b_{k} . Here, D_{k}(I^{E},I^{S}) quantifies the discrepancy between the transformed and original images. It is noteworthy that D_{k}(I^{E},I^{S})\neq D_{k}(I^{E},I^{S}) , as the operation is not symmetric. Consequently, D_{k}(I^{E},I^{S}) alone may fail to fully capture the structural differences between two images. For example, consider a case where a region in I^{S} is structurally flat while the corresponding region in I^{E} contains structural details (see highlighted region in Fig. 2 -2). In such a scenario, a trivial transformation ( a_{k}\approx 0,b_{k}\approx\mu_{k}^{S} ) can map I^{E} to I^{S} . This results in a deceptively low value for D_{k}(I^{E},I^{S}) despite the significant structural disparities, as shown in the corresponding error map ( Fig. 2 -2-c). To address this limitation, we introduce a reverse mapping D_{k}(I^{S},I^{E}) ( Fig. 2 -2-d). Finally, we combine the forward and reverse mappings to define the structure preservation loss: \mathcal{L}_{\text{SPL}}(I^{E},I^{S})=\frac{1}{N}\sum_{k}\left(D_{k}(I^{E},I^{S})+D_{k}(I^{S},I^{E})\right), (5) where N is the total number of pixels in the image, and the summation is performed over all pixels k . By enforcing consistency across both forward ( E\to S ) and reverse ( S\to E ) transformations, this loss function robustly penalizes mismatches in underlying structures. For RGB images, we convert them to HSI color space, and use the I channel to compute \mathcal{L}_{\textrm{SPL}} in our approach. Color Preservation Loss In tasks like local color editing, we must ensure that changing an object’s color does not alter the hues of its surroundings. To address this, we introduce an optional color preservation loss, \mathcal{L}_{\text{CPL}} , that can be used alongside \mathcal{L}_{\text{SPL}} . We compute \mathcal{L}_{\text{CPL}} as the mean squared error between the Cb and Cr channels of I^{A} and I^{B} in the YCbCr color space: \mathcal{L}_{\text{CPL}}(I^{E},I^{S})=\frac{1}{N}\sum_{k}\left\|\text{CbCr}(I^{E}_{k})-\text{CbCr}(I^{S}_{k})\right\|_{2}^{2}, (6) We empirically found that using HSI for \mathcal{L}_{\textrm{SPL}} and YCbCr for \mathcal{L}_{\textrm{CPL}} yields higher-quality results. This is because they provide simple linear conversions from the RGB color space. This simplicity enables more efficient loss optimization. Furthermore, the I channel in the HSI color space, calculated as the average of the color channels with equal weights, ensures uniform updates across all RGB components when optimizing \mathcal{L}_{\textrm{SPL}} . Figure 3 : Structure-preserving denoising loop . At each denoising timestep t , we decode the predicted clean latent \hat{z}_{0}^{(t)} to compute our structure preservation loss \mathcal{L}_{\text{SPL}} in the image space. The resulting gradient is then used to update the latent, producing a corrected version \tilde{z} that steers the generation trajectory to maintain structural fidelity for the subsequent denoising step. 3.2 Structure-Preserving Editing with LDMs In this section, we explain how our structure preservation loss enhances the denoising process of LDMs to achieve pixel-level structure-preserving image editing. We first outline conventional coarse-structure-preserving editing in LDMs and its limitations, then describe our approach, which integrates \mathcal{L}_{\text{SPL}} through the denoising diffusion process and a post-processing step. We provide an overview in Fig. 3 . Review of Coarse-Structure-Preserving Image Editing in LDMs In LDMs, an encoder \mathcal{E} maps images to latents, and a decoder \mathcal{D} reconstructs images from these latents. The diffusion process of LDMs is defined in this latent space, enabling efficient image generation and manipulation [ 49 , 53 ] . Coarse-structure-preserving image editing with LDMs, such as Prompt-to-Prompt [ 20 ] and InfEdit [ 67 ] , typically involves three inputs: a source image I_{\text{src}} , its associated text prompt p_{\text{src}} , and a target edit prompt p_{\text{edit}} . The editing process begins with a latent code z_{T} sampled from pure Gaussian noise, i.e., z_{T}\sim\mathcal{N}(0,I) , where T is the maximum timestep. The reverse diffusion process then iteratively denoises this latent from t=T to t=1 , guided by a noise prediction model \epsilon_{\theta} , parameterized by \theta . To achieve coarse-structure-preserving edits, \epsilon_{\theta} is conditioned on the target prompt p_{\text{edit}} and source features f^{\text{src}}_{t} (e.g., attention maps, estimated noise), extracted from the generation of the source image I_{\text{src}} using its prompt p_{\text{src}} [ 20 , 67 ] . This conditioning preserves the coarse structure of the source image during the generation of the edited image. For a forward process defined as z_{t}=\alpha_{t}z_{0}+\sigma_{t}\epsilon with \epsilon\sim\mathcal{N}(0,I) , the predicted denoised latent \hat{z}_{0}^{(t)} at each timestep t is computed as: \hat{z}_{0}^{(t)}=\frac{1}{a_{t}}\left(z_{t}-b_{t}\epsilon_{\theta}\left(z_{t},t,p_{\text{edit}},f^{\text{src}}_{t}\right)\right). (7) From \hat{z}_{0}^{(t)} , the noisy latent for t-1 is computed as: z_{t-1}=\mathcal{S}\left(\hat{z}_{0}^{(t)},z_{t},t,\hat{\epsilon}_{t}\right), (8) where \mathcal{S} denotes a sampling function and \hat{\epsilon}_{t} is the estimated noise \epsilon_{\theta}\left(z_{t},t,p_{\text{edit}},f^{\text{src}}_{t}\right) . The iterative denoising process then advances to timestep t-1 . While the attention conditioning strategy leveraging f_{t}^{\textrm{src}} in Eq. 7 preserves coarse image layouts, it does not retain fine-grained structural details of the input image, as will be shown in Section 4 . Editing with Structure Preservation Loss Our task of preserving pixel-level edge structure is fundamentally defined in the image space and thus requires a corresponding pixel-space guidance. Hence, we apply \mathcal{L}_{\text{SPL}} to the VAE-decoded image of the denoised latent \hat{z}_{0}^{(t)} at intermediate timesteps, following prior works that have successfully guided latent denoising with pixel-space objectives [ 32 , 54 , 79 , 37 ] . At each timestep t , we compute \hat{z}_{0}^{(t)} using Eq. 7 . Next, we update \hat{z}_{0}^{(t)} by solving: \tilde{z}=\mathop{\mathrm{argmin}}_{\hat{z}}\mathcal{L}_{\textrm{SPL}}\left(I_{\textrm{src}},\mathcal{D}(\hat{z})\right)+\lambda\mathcal{L}_{\textrm{CPL}}\left(I_{\textrm{src}},\mathcal{D}(\hat{z})\right) (9) where \hat{z} is initially set to \hat{z}_{0}^{(t)} , \tilde{z} is the updated latent, and \lambda is a weight for the optional color preservation loss, which we set to a small value. Finally, \tilde{z} is substituted into Eq. 8 in place of \hat{z}_{0}^{(t)} , and the denoising process continues to timestep t-1 . The updated latent \tilde{z} guides the subsequent denoising steps to not only retain the source image’s structural details, but also synthesize natural-looking image contents that align with the retained details. To solve Eq. 9 , we adopt the gradient descent method. However, direct application of gradient descent requires repeated, computationally expensive gradient calculations due to backpropagation through \mathcal{D} . To circumvent this problem, we introduce an auxiliary image \hat{I} , initialized as \hat{I}=\mathcal{D}(\hat{z}_{0}^{(t)}) , and reformulate the optimization problem as: \tilde{I}=\mathop{\mathrm{argmin}}_{\hat{I}}\mathcal{L}_{\textrm{SPL}}\left(I_{\textrm{src}},\hat{I}\right)+\lambda\mathcal{L}_{\textrm{CPL}}\left(I_{\textrm{src}},\hat{I}\right). (10) After solving this, we compute \tilde{z}=\mathcal{E}(\tilde{I}) and substitute it into Eq. 8 . Eq. 10 is efficiently solved using a gradient descent update: \hat{I}\leftarrow\hat{I}-\eta\nabla_{\hat{I}}\left\{\mathcal{L}_{\text{SPL}}(I_{\text{src}},\hat{I})+\lambda\mathcal{L}_{\text{CPL}}(I_{\text{src}},\hat{I})\right\}, (11) where \eta is the learning rate. We apply this optimization selectively during the later stages of the denoising process (i.e., timesteps t\leq t_{\text{SPL}} ). This approach allows for early denoising steps to maintain generative flexibility while later stages prioritize accurate preservation of structural details. Further analysis of this scheduling is provided in the supplementary. Post-Processing Step While the diffusion process with the structure preservation loss improves structural consistency, the final decoding \hat{I}_{0}=\mathcal{D}(\hat{z}_{0}) may still lose fine details due to its inherent compression characteristics. To mitigate this, we introduce a post-processing step that refines \hat{I}_{0} directly in the image space by solving Eq. 10 . This step ensures pixel-level fidelity to the source image’s fine structures, compensating for the decoder’s limitations. Figure 4 : Cross-attention mask upsampling. For the background editing task, the baseline method unintentionally distorts the foreground (b). By upsampling the coarse attention map (c), our method generates a sharp, high-resolution mask (d). This enables targeted application of our structure loss, preserving the foreground’s fidelity in the final edit (e). 3.3 Cross-Attention Mask Upsampling for Structure-Preserving Localized Editing For localized editing such as background replacement ( Fig. 4 ), we apply structure and color preservation losses to specific regions defined by a binary mask. We derive an initial coarse mask by thresholding the low-resolution ( 16\times 16 ) cross-attention map corresponding to the edit prompt [ 20 ] ( Fig. 4 -c). However, naively upsampling this mask with bilinear interpolation produces blurry and misaligned boundaries. To overcome this, we introduce a simple yet effective iterative refinement algorithm. Starting with the coarse mask, we progressively upscale it by a factor of two. At each step, we use the input image, downscaled to the current mask’s resolution, as a guidance image for the guided filter [ 19 ] . This process sharpens and aligns the mask’s boundaries by transferring structural details from the guide image. The result is a high-resolution mask with sharp, structurally accurate boundaries ( Fig. 4 -d), enabling precise and artifact-free local editing ( Fig. 4 -e). We provide a pseudocode and further details in the supplementary. Figure 5 : Qualitative comparison of global editing tasks. Our method (b) successfully applies the edit while preserving fine-grained structural details. Other methods (c-g) exhibit either low prompt fidelity or significant structural distortions Figure 6 : Qualitative comparison of local editing tasks. Our method can generate an edit mask from the text prompt (b, top-left) to enable precise local editing. Other methods (c-g) fail to preserve the structure of the content shared between the source and target prompts. 4 Experiment Implementation Details We integrate our method into InfEdit [ 67 ] , a training-free, latent-diffusion-based framework for image editing. While InfEdit is our primary backbone, our approach is model-agnostic. We demonstrate its application to another backbone in the supplementary. For tasks requiring localized edits, we use the color preservation loss and the upscaled edit masks as detailed in Section 3.3 . We report the hyperparameters in the supplementary. 4.1 Evaluation Details Benchmark for Diffusion-Based Image Editing Models We primarily evaluate our method on a curated subset of PIE-Bench [ 24 ] . This curation is necessary because standard benchmarks, such as AnyEdit [ 72 ] or RealEdit [ 60 ] , often do not strictly distinguish edits requiring pixel-level edge preservation, frequently mixing them within broader categories. Our curated benchmark provides a more controlled setting for a targeted evaluation. We categorize the editing tasks based on the scope of structure preservation: 1) Global editing : Edits applied across the entire image, requiring structure preservation of the whole input image. Examples include relighting, photorealistic style transfer, tone adjustment, and time-lapse generation. 2) Local editing : Edits requiring structure preservation of specific regions. Examples include background replacement and weather changes, where only the structure of the foreground needs to be preserved. For this adapted subset, we generate task-specific textual prompts using GPT-4o [ 43 ] , followed by manual refinement, resulting in approximately 470 image-prompt pairs. We also provide a supplementary evaluation on the standard AnyEdit [ 72 ] benchmark and show that the results confirm the findings from our main experiments. We conduct our evaluation on three categories from AnyEdit that align with our focus on structure-preserving edits: tone transfer, color alteration, and background change. Evaluation Metric We assess our method based on two key criteria: (1) Preservation : We measure how well the edited image preserves the original image’s features. Our evaluation relies on a suite of standard, independent metrics, including SSIM [ 65 ] for structural similarity, LPIPS [ 75 ] for perceptual similarity, FSIM [ 74 ] for feature similarity, and GMSD [ 69 ] for gradient similarity. We also report our structure preservation loss as it offers a direct measure for the fine-grained structural fidelity that current metrics do not exclusively capture. (2) Prompt fidelity : We assess how well the edited image aligns with the edit text prompt. We employ CLIP [ 50 ] Score to measure the similarity between the target edit prompt and the edited image, and the CLIP Directional Similarity to quantify consistency between the textual prompt changes and corresponding image edits. Method Preservation Prompt fidelity SPL ( \times 10^{2} ) \downarrow SSIM \uparrow LPIPS \downarrow FSIM \uparrow GMSD \downarrow CLIP S. \uparrow CLIP D. \uparrow Ours 0.107 0.854 0.212 0.927 0.062 0.272 0.163 InfEdit (Baseline) [ 67 ] 0.748 0.600 0.375 0.792 0.180 0.284 0.163 IP2P [ 4 ] 0.517 0.668 0.348 0.839 0.134 0.262 0.153 DDPMInv [ 21 ] 0.710 0.702 0.238 0.841 0.161 0.271 0.091 NT+P2P [ 41 , 20 ] 0.545 0.736 0.214 0.881 0.126 0.256 0.135 GNRI [ 55 ] 0.972 0.635 0.286 0.801 0.184 0.247 0.066 Table 1 : Quantitative comparison of LDM-based image editing on PIE-Bench Our method outperforms other models in LDM-based image editing, scoring the highest in preservation metrics. It also stays faithful to the edit prompt, as shown in the prompt fidelity scores. Method Preservation Prompt fidelity SPL \downarrow SSIM \uparrow LPIPS \downarrow FSIM \uparrow GMSD \downarrow CLIP S. \uparrow CLIP D. \uparrow Ours 0.085 0.906 0.120 0.953 0.046 0.263 0.078 InfEdit (Baseline) [ 67 ] 0.865 0.648 0.280 0.813 0.176 0.277 0.103 IP2P [ 4 ] 0.576 0.683 0.307 0.840 0.140 0.261 0.118 DDPMInv [ 21 ] 0.838 0.711 0.217 0.838 0.168 0.276 0.077 NT+P2P [ 41 , 20 ] 0.637 0.759 0.170 0.887 0.125 0.265 0.084 GNRI [ 55 ] 1.120 0.669 0.229 0.816 0.179 0.253 0.050 Table 2 : Quantitative comparison of LDM-based image editing on AnyEdit benchmark. Consistent with our findings on PIE-Bench, our method demonstrates state-of-the-art performance in all preservation metrics while maintaining competitive prompt fidelity. 4.2 Comparison with SoTA Editing Methods We compare our method against state-of-the-art latent diffusion-based editing methods, including InstructPix2Pix [ 4 ] , InfEdit [ 67 ] , DDPM Inversion [ 21 ] , GNRI [ 55 ] and a combination of null-text inversion [ 41 ] and Prompt-to-Prompt [ 20 ] . InstructPix2Pix relies on supervised training from editing datasets, InfEdit uses unified attention control for text-guided edits, DDPM Inversion employs inversion-based editing, GNRI introduces a fast inversion technique for real-time editing. Prompt-to-Prompt shows a source image layout preserving editing, where the source image is inverted with null-text inversion. Quantitative evaluations on both PIE-Bench ( Table 1 ) and the AnyEdit benchmark ( Table 2 ) indicate that our method significantly outperforms baselines in preservation. This improvement is largely attributed to our structure preservation loss, achieved without compromising prompt fidelity. Qualitative comparisons further illustrate these advantages. In global editing tasks ( Fig. 5 ), InstructPix2Pix often generates either overly exaggerated or insufficient edits, DDPM Inversion and GNRI struggle with both structure preservation and prompt fidelity, and InfEdit preserves only coarse-level structure, compromising fine structural details. In contrast, our method accurately reflects edit prompts while precisely preserving pixel-level edge structures. For local editing tasks ( Fig. 6 ), our proposed mask acquisition method effectively confines edits to regions specified by the prompt (e.g., weather or background), maintaining key elements that remain the same in both source and target prompts. Conversely, methods without region-specific editing inadvertently change the entire image, failing to respect localized changes described by the prompt. In the supplementary, we also provide comparisons with task-specific structure-preserving methods, including photo-realistic style transfer, image harmonization, tone adjustment, seasonal change, and time-lapse editing, alongside qualitative results on the AnyEdit benchmark. 4.3 Analysis Comparison with Other Structure-Similarity Losses We illustrate the effectiveness of our structure preservation loss compared to other common structural similarity measures, e.g., SSIM, MSE, and the directional structure difference loss in Eq. 4 . The editing task transforms the input image ( Fig. 7 -a) into a “futuristic night” style. Since MSE and SSIM penalize pixel-wise luminance and contrast differences, using them leads to results that fail to fully reflect the intended night-style contrast and brightness ( Fig. 7 -b, c). The directional structure difference loss does not effectively penalize structural artifacts when the source image can be approximated by the edit image with a local linear transform, causing visible distortions at object boundaries ( Fig. 7 -d). In contrast, our full structure preservation loss successfully maintains pixel-level edge fidelity while accurately following the target edit prompt ( Fig. 7 -e). Component-wise Ablation The ablation studies in Fig. 8 and Table 3 show the contribution of each component in the structure preservation loss. In Fig. 8 , we edit the color of the cover text from red to yellow. The baseline undesirably alters elements like clothing and cover text ( Fig. 8 -b). Adding SPL first improves overall structural preservation ( Fig. 8 -c). Then, CPL with a mask excluding the book specifically restores the original clothing color ( Fig. 8 -d), and post-processing recovers fine details like the cover text ( Fig. 8 -e). Quantitative results in Table 3 also falls in line with the qualitative result, where each component builds up better preservation without loss of prompt fidelity. We also show the necessity of incorporating the structure preservation loss within the diffusion pipeline in Fig. 9 . Note that applying our post-processing step to the edited result obtained without this intermediate guidance is equivalent to applying SPL directly to the edited image ( Fig. 9 -b). However, this approach cannot prevent LDM from generating structural changes that go beyond what the post-processing step can handle ( Fig. 9 -c). Computational Cost We analyze the computational cost of our method in Table 4 , with all experiments conducted on a single NVIDIA A6000 GPU. Our method is substantially faster than inversion-based approaches like DDPM Inversion [ 21 ] and the combination of null-text inversion [ 41 ] and Prompt-to-Prompt [ 20 ] . Compared to the baseline InfEdit [ 67 ] , our approach adds a modest overhead of approximately 2 seconds. We argue this is a valuable trade-off, as this overhead enables the precise, pixel-level structural control that the baseline lacks. Figure 7 : Ablation study on different loss functions for optimization. While other structural losses (b-d) penalizes valid appearance changes (e.g., brightness and contrast), our proposed loss (d) successfully disentangles structure from appearance, enabling a faithful edit while preserving structural fidelity. Figure 8 : Qualitative component-wise ablation of our editing method. Each component of our editing method progressively improves the quality of the baseline edit (b) to our final result (e). PP: Post-processing. Method Preservation Prompt fidelity SPL ( \times 10^{2} ) \downarrow SSIM \uparrow LPIPS \downarrow FSIM \uparrow GMSD \downarrow CLIP S. \uparrow CLIP D. \uparrow Baseline 0.594 0.653 0.312 0.834 0.151 0.281 0.172 +SPL 0.351 0.720 0.265 0.890 0.102 0.274 0.168 +CPL 0.351 0.720 0.252 0.892 0.102 0.272 0.161 +PP 0.107 0.853 0.209 0.927 0.063 0.272 0.163 Table 3 : Quantitative component-wise ablation of our editing method. Adding each component progressively improves all preservation scores while maintaining strong prompt fidelity. PP: Post-processing. Figure 9 : Guidance vs. Post-Processimg. Applying our loss only as post-processing (c) fails to fix the baseline’s severe structural errors (b). In contrast, our iterative guidance (d) prevents these errors from forming in the first place. PP: Post-processing. Method Time(s) Peak memory(GB) GNRI [ 55 ] 0.39 13.99 InfEdit (Baseline) [ 67 ] 2.26 5.82 Ours without PP. 4.18 5.82 Ours with PP. 4.58 5.82 IP2P [ 4 ] 5.29 2.61 DDPMInv [ 21 ] 23.64 6.09 NT+P2P [ 41 , 20 ] 130.99 14.9 Table 4 : Computational cost comparison. Our method demonstrates competitive performance, being substantially faster than inversion-based methods like DDPMInv and NT+P2P, while adding a modest overhead to the fast InfEdit baseline for significantly improved structure preservation. PP: Post-processing. 5 Conclusion We presented a novel structure preservation loss that penalizes the genuine structural differences between two images. Our method integrates this loss into a training-free LDM-based editing pipeline, achieving edits that preserve pixel-level structure. Our approach also includes a color preservation loss and a text-driven edit mask generation scheme for precise local control. Together, these contributions enable a universal image editing method that preserves pixel-level edge structures of the input image, outperforming state-of-the-art baselines as demonstrated in our evaluations. Limitations and Future Work Our method’s performance depends on the underlying diffusion model’s quality and speed. While our approach will benefit from future advancements in LDMs, its effectiveness may be limited when paired with older or lower-quality backbones. Additionally, the optimization-based application of our loss introduces a computational overhead. We provide an analysis of this overhead and our method’s failure cases, including the impact of imperfect masks and challenging edit scenarios, in Appendix A . Future work would explore integrating our loss directly into the pre-training of diffusion models to mitigate the computational overhead. \thetitle Supplementary Material Appendix A Additional Experiments and Analyses A.1 Comparison with Task-Specific Structure-Preserving Methods In addition to LDM-based editing models, we compare our method against task-specific methods across several key structure-preserving tasks. Photo-realistic Style Transfer synthesizes images by merging content and style from separate images. We compare our method with PCAKD [ 9 ] , utilizing 60 content-style image pairs from the DPST dataset [ 39 ] . The evaluation is performed using prompts generated via the method described in Appendix E . In Fig. 1 , our method better captures subtle stylistic features compared to PCAKD [ 9 ] , producing higher-quality results. Image Harmonization adjusts a foreground object’s color and brightness to match a composite image’s background. Similar to style transfer, we derive prompts using GPT-4o [ 43 ] . In Fig. 2 , our method yields results that blend more consistently with the background than PCTNet [ 16 ] . Image Tone Adjustment modifies the brightness, contrast, and color balance of the input image. We compare our method with CLIPtone [ 31 ] using a subset of [ 5 ] , which is a test set consisting of approximately 500 images and around 50 different tone descriptions. The source and edit text prompt pairs are constructed in the format ”a normal photo of…” \rightarrow ”a [tone] photo of…” by combining image captions generated by BLIP [ 35 ] and tone descriptors. Compared to CLIPtone [ 31 ] , our method more accurately and naturally achieves the intended tone. Seasonal change alter environmental contexts. We compare with CycleGAN’s [ 78 ] pre-trained summer-to-winter model, using approximately 550 provided test set images. The source and edit text prompt pairs follow the format ”a photo of summer …” \leftrightarrow ”a photo of winter …”. Fig. 4 demonstrate our method maintains pixel-level structures and achieves better edit quality compared to CycleGAN, thanks to our diffusion-based prior. Time-lapse Editing alter temporal contexts. We evaluate against Pix2pix [ 22 ] , using its pre-trained day-to-night model. We use the night-to-day dataset of [ 29 ] , and use 350 daytime images. The source and edit text prompt pairs are manually created in the form ”a photo of … at day” \rightarrow ”a photo of … at night”. Fig. 5 demonstrate our method maintains pixel-level structures and achieves better edit quality compared to Pix2pix thanks to our diffusion-based prior. Quantitative Comparison. For the evaluation, we use the same metrics used in Section 4.1 . As shown in Table 1 , our model achieves superior prompt fidelity compared to methods specifically designed for each structure-preserving image editing task, with notable advantages in structural preservation through our SPL. LPIPS, being a perceptual metric, may be lower for our model compared to CLIPtone due to our stronger emphasis on structure preservation rather than perceptual similarity alone. As discussed in Section 4.3 , we note that SSIM scores can be misleading in tasks involving significant brightness changes (e.g., image tone adjustment or time-lapse), since SSIM strongly penalizes brightness variations. A.2 Qualitative Comparison on the AnyEdit Benchmark We provide qualitative comparisons on the AnyEdit benchmark [ 72 ] in Fig. 6 and Fig. 7 further support these findings. The visualizations highlight our method’s ability to preserve fine-grained, pixel-level edge structures, whereas competing methods often introduce noticeable structural artifacts or fail to fully respect the source image’s pixel-level edge structures. Overall, these results strongly corroborate the conclusions drawn from our main experiments on the PIE-Bench subset. Figure 1 : Qualitative comparison on photorealistic style transfer . Given a content image (a) and a style image (b), our method effectively transfers the sunrise style, producing a naturally stylized result (d). In contrast, PCAKD (c) fails to transfer this style effectively. Figure 2 : Qualitative comparison on image harmonization . (a) Composed image with foreground mask (top-right). The Result from PCTNet (b) exhibits clear lighting inconsistencies. Our method (c) seamlessly blends the foreground and background. Figure 3 : Qualitative comparison on tone adjustment . While the result of CLIPtone (b) is overly saturated, our method (c) produces a naturally-toned result well-aligned with the text description. Figure 4 : Qualitative comparison on season change . While CycleGAN (b) produces an incoherent result, our method (c) successfully generates a natural seasonal transformation, realistically integrating effects like snow. Figure 5 : Qualitative comparison on time-lapse . Pix2Pix (b) introduces significant structural distortions. Our method (c) achieves a realistic time-of-day change while maintaining the structure. Method Preservation Prompt fidelity SPL( \times 10^{2} ) \downarrow SSIM \uparrow LPIPS \downarrow CLIP S. \uparrow CLIP D. \uparrow Photorealistic Style Transfer Ours 0.006 0.879 0.182 0.254 0.147 PCAKD 0.752 0.478 0.346 0.248 0.130 Image Tone Adjustment Ours 0.010 0.680 0.250 0.237 0.078 CLIPTone 0.016 0.862 0.154 0.226 0.075 Time-lapse Ours 0.070 0.240 0.471 0.234 0.192 Pix2pix 0.534 0.396 0.623 0.235 0.136 Season/Weather Change Ours 0.061 0.856 0.266 0.197 0.170 CycleGAN 1.358 0.463 0.350 0.198 0.097 Table 1 : Quantitative comparison with task-specific editing methods. Our method consistently achieves the best structure preservation loss across all tasks while maintaining high prompt fidelity. Note that SSIM is sensitive to luminance and contrast variations; thus, for tasks requiring brightness or contrast adjustments (e.g., tone adjustment, time-lapse), SSIM scores may not accurately reflect structural preservation. Additionally, LPIPS primarily captures perceptual similarity rather than structural fidelity. Figure 6 : Qualitative comparison on global editing tasks. Our method (b) successfully applies the edit while preserving fine-grained structural details. Other methods (c-g) exhibit either low prompt fidelity or significant structural distortions Figure 7 : Qualitative comparison of local editing tasks. Our method can generate an edit mask from the text prompt (b, bottom-left) to enable precise local editing. Other methods (c-g) fail to preserve the structure of the content shared between the source and target prompts. Figure 8 : Cross-attention mask upsampling for SDXL [ 49 ] . By upsampling the coarse attention map (b), our method generates a sharp, high-resolution mask (c). Figure 9 : Cross-attention mask upsampling for FLUX. By upsampling the coarse attention map (b), our method generates a sharp, high-resolution mask (c). A.3 Cross Attention Mask Upsampling with Different Backbones Our cross-attention mask upsampling method extends naturally to various diffusion model architectures beyond the original LDM [ 53 ] , such as SDXL [ 49 ] and FLUX 1 1 1 https://huggingface.co/black-forest-labs/FLUX.1-dev . For U-Net based backbones like SDXL, we follow Prompt-to-Prompt [ 20 ] and extract the cross-attention maps of resolution 32 \times 32 from the bottleneck layer. For FLUX, which has a DiT [ 46 ] structure, we extract the average attention map of resolution 64 \times 64 from intermediate blocks 12 through 18. Our guided upsampling algorithm then refines these coarse initial maps to the model’s native output resolution—from 16 \times 16 to 512 \times 512 for the original LDM, and 32 \times 32 to 1024 \times 1024 for SDXL and 64 \times 64 to 1024 \times 1024 FLUX. This demonstrates the flexibility of our approach in adapting to different architectures. We show qualitative results for SDXL in Fig. 8 and for FLUX in Fig. 9 , respectively. A.4 Failure Case Analysis Cross-Attention Mask Upsampling As demonstrated in Fig. 10 , our guided upsampling technique is generally effective at capturing object silhouette with high fidelity. However, the accuracy of the final mask is fundamentally dependent on the quality of the initial coarse cross-attention map. In some cases, if the initial map is not well-localized, the refined mask may cover a slightly larger region than the target object, as shown in the red box of Fig. 10 . When this occurs, SPL is inadvertently applied to this slightly oversized region. While the resulting edit still adheres to the prompt, this can cause subtle edge structures from the source image to be preserved. Structure Preservation Loss The limitation of SPL becomes apparent in tasks that are inherently ambiguous for a structure-preserving method, where an edit may not be entirely aligned with the user’s intention. We demonstrate this with a challenging material editing task: transforming a leather bag into a denim one, shown in Fig. 11 . As intended, SPL successfully preserves the bag’s overall macro-structure, such as the arm strap and its buckles, and also the fine-grained edge details of the original material’s texture. The final edited result is coherent and appears natural with appropriate shading. However, if a user intends a structure-breaking material change that completely replaces the micro-texture, such an edit falls outside the intended scope of SPL. Figure 10 : Imprecise Edit Mask Example. The soft boundaries of the upsampled mask (b) can sometimes extend slightly beyond the foreground object. As a result, subtle structural details from the source image (a) are unintentionally preserved near the cat’s silhouette (c). Figure 11 : Material editing example from leather to denim . Our method (c) preserves the fine-grained texture and wear patterns of the original leather, while the baseline (b) breaks the structure and replaces the material entirely. Figure 12 : Examples of image distortions. (b-c) Non-structural distortions, (d-f) Structural distortions. Figure 13 : Scheduling of attention conditioning and structure preservation loss. We analyze the impact of attention conditioning and the structure preservation loss on structural fidelity. As shown in Fig. 13 , applying attention conditioning throughout the generative process helps retain coarse structures but fails to preserve fine details, even at full conditioning. In contrast, incorporating our structure preservation loss effectively maintains pixel-level edge fidelity, even with reduced attention conditioning. Distortion SPL ( \times 10^{2} ) \downarrow PSNR \uparrow SSIM \uparrow LPIPS \downarrow Color change 0.080 11.06 0.927 0.504 Darken 0.063 10.60 0.664 0.195 Lens blur 0.241 24.77 0.776 0.301 White noise 0.356 20.56 0.338 0.551 Jitter 0.325 22.20 0.783 0.196 Table 2 : Comparison of image similarity metrics across different types of distortions. Figure 14 : Generalizability of our method across different baseline model. Our method can be integrated into diverse LDM-based image editing pipelines (e.g., Null-text inversion + Prompt-to-Prompt), enhancing their ability to preserve the structural details of the input image during editing. Figure 15 : Additional qualitative comparison with LDM-based image editing methods . The first and second rows demonstrate global editing results. The Last row shows additional local editing results. A.5 Scheduling of Attention Conditioning and Structure Preservation Loss. We explore how the scheduling of attention conditioning and structure preservation loss affects edits. As we can see in Fig. 13 , keeping the attention conditioning strengthens coarse structure preservation, but even with full attention conditioning, the fine structural details of the input image are distorted. However, when combined with structure preservation loss, we can see that the pixel-level edge structural fidelity is kept. A.6 Validation as a Structural Difference Metric To validate our proposed loss as a robust metric for structural similarity, we evaluate its response to a range of image distortions as seen in Fig. 12 . The results in Table 2 demonstrate that our metric successfully disentangles structure from appearance. It registers a low penalty for non-structural distortions (e.g., color and brightness shifts) while correctly identifying structural distortions. In contrast, common metrics like SSIM [ 65 ] and LPIPS [ 75 ] often conflate these two, assigning high penalties to non-structural changes. A.7 Generalization to Different Backbones To demonstrate the generality and modularity of our approach, we apply our structure-preserving editing method to a different baseline: the combination of Null-text inversion [ 41 ] and Prompt-to-Prompt [ 20 ] . As shown in Fig. 14 , our method enhances the structural fidelity of the baseline’s output, confirming its effectiveness across different editing frameworks. A.8 Additional Qualitative Comparison We provide additional qualitative comparison results with LDM-based editing methods in Fig. 15 . We can see that our method persistently achieves the best pixel-level edge structure preservation without the loss of prompt fidelity. Appendix B Additional Implementation Details For all experiments, we use total inference steps of T=15 . For the structure preservation loss, defined via a local linear model in Equation Eq. 1 , we configure a window size of \omega_{k}=11 and a regularizer \epsilon=10^{-4} . In the optimization-driven denoising process, we apply stochastic gradient descent with a learning rate \eta=1 and momentum 0.9. During optimization we use \lambda=10^{-4} for to emphasize structural fidelity ( Eq. 10 ). We fix the number of optimization iterations at k=s=100 and structure preservation loss threshold timestep t_{\text{SPL}}=12 . For tasks requiring localized edits, we employ the upscaled masks from Section 3.3 . All experiments were conducted on an NVIDIA A6000 GPU. The models used in our experiments are as follows: for InfEdit [ 67 ] , we used LCM Dreamshaper v7; for InstructPix2Pix [ 4 ] , the official model provided by the authors was employed. Both DDPMInv [ 21 ] and NT+P2P [ 41 , 20 ] were implemented using Stable Diffusion v1.4. When applying the coarse structure preservation through attention conditioning as detailed in Section 3.2 , we observe that applying this attention conditioning across all denoising timesteps can overly constrain the latent, reducing fidelity to the edit prompt p_{\text{edit}} . To balance coarse structure preservation with edit flexibility, we schedule the attention conditioning, applying f^{\text{src}}_{t} only for timesteps t\geq t_{\text{attn}} . We set this attention conditioning scheduling timestep as t_{\text{attn}}=12 . Appendix C Derivation of the LLM Coefficients Given images I^{E} and I^{S} , the local linear model defines the relationship between these two images within a local window \omega_{k} as: I^{S}_{i}=a_{k}\,I^{E}_{i}+b_{k},\quad\forall i\in\omega_{k}, (12) We derive the coefficients a_{k} and b_{k} by minimizing the cost function E(a_{k},b_{k}) within a local window \omega_{k} : E(a_{k},b_{k})=\sum_{i\in\omega_{k}}\left((a_{k}I^{E}_{i}+b_{k})-I^{S}_{i}\right)^{2}. (13) The minimum is found by setting the partial derivatives with respect to a_{k} and b_{k} to zero. Derivation of the Offset Coefficient b_{k} Differentiating E with respect to b_{k} and setting the result to zero yields: \displaystyle\frac{\partial E}{\partial b_{k}}=2\sum_{i\in\omega_{k}}(a_{k}I^{E}_{i}+b_{k}-I^{S}_{i})=0 \displaystyle\implies \displaystyle a_{k}\sum_{i\in\omega_{k}}I^{E}_{i}+\sum_{i\in\omega_{k}}b_{k}-\sum_{i\in\omega_{k}}I^{S}_{i}=0 Dividing by |\omega_{k}| , the number of pixels in the window, gives the means \mu_{k}^{E} and \mu_{k}^{S} . Hence, solving for b_{k} : \displaystyle a_{k}\mu^{E}_{k}+b_{k}-\mu^{S}_{k}=0 \displaystyle\implies \displaystyle b_{k}=\mu^{S}_{k}-a_{k}\mu^{E}_{k}. Derivation of the Scaling Coefficient a_{k} Next, we differentiate E with respect to a_{k} and set the result to zero: \frac{\partial E}{\partial a_{k}}=2\sum_{i\in\omega_{k}}(a_{k}I^{E}_{i}+b_{k}-I^{S}_{i})I^{E}_{i}=0 Substituting our expression for b_{k}=\mu^{S}_{k}-a_{k}\mu^{E}_{k} : \displaystyle\sum_{i\in\omega_{k}}(a_{k}I^{E}_{i}+(\mu^{S}_{k}-a_{k}\mu^{E}_{k})-I^{S}_{i})I^{E}_{i}=0 \displaystyle\implies \displaystyle\sum_{i\in\omega_{k}}(a_{k}(I^{E}_{i}-\mu^{E}_{k})-(I^{S}_{i}-\mu^{S}_{k}))I^{E}_{i}=0 Solving for a_{k} : a_{k}=\frac{\sum_{i\in\omega_{k}}(I^{S}_{i}-\mu^{S}_{k})I^{E}_{i}}{\sum_{i\in\omega_{k}}(I^{E}_{i}-\mu^{E}_{k})I^{E}_{i}} This expression is equivalent to the covariance of I^{E} and I^{S} divided by the variance of I^{E} . Dividing the numerator and denominator by |\omega_{k}| and including the regularization term \rho , we arrive at the final form: a_{k}=\frac{\frac{1}{|\omega_{k}|}\sum_{i\in\omega_{k}}I^{E}_{i}I^{S}_{i}-\mu^{E}_{k}\mu^{S}_{k}}{(\sigma^{E}_{k})^{2}+\rho}, (14) where (\sigma^{E}_{k})^{2} is the variance of I^{E} in \omega_{k} . Appendix D Algorithms Algorithm 1 Optimization-Driven Denoising Process 1: Source image I_{\text{src}} , edit prompt p_{\text{edit}} , source features f^{\text{src}}_{t} , noise prediction model \epsilon_{\theta} , encoder \mathcal{E} , decoder \mathcal{D} , maximum timestep T , coefficients a_{t},b_{t} from noise schedule, scheduling timestep threshold t_{s} , learning rates \eta,w , number of optimization steps k,s , loss weights \lambda 2: Edited image \hat{I}_{0} 3: Initialize latent z_{T}\sim\mathcal{N}(0,\mathbf{I}) 4: for t in (T,1) do 5: \hat{\epsilon}_{t}\leftarrow\epsilon_{\theta}\left(z_{t},t,p_{\text{edit}},f^{\text{src}}_{t}\right) 6: \hat{z}_{0}^{(t)}\leftarrow\frac{1}{a_{t}}\left(z_{t}-b_{t}\hat{\epsilon}_{t}\right) 7: if t\leq t_{s} then \triangleright Scheduling optimization to preserve details 8: \hat{I}\leftarrow\mathcal{D}(\hat{z}_{0}^{(t)}) \triangleright Decode to image space 9: for i in (0,k) do \triangleright Gradient descent step 10: \hat{I}\leftarrow\hat{I}-\eta\nabla_{\hat{I}}\left\{\mathcal{L}_{\text{SPL}}(I_{\text{src}},\hat{I})+\lambda\mathcal{L}_{\text{CPL}}(I_{\text{src}},\hat{I})\right\} 11: end for 12: \tilde{z}_{0}^{(t)}\leftarrow\mathcal{E}(\hat{I}) \triangleright Re-encode optimized image 13: end if 14: z_{t-1}\leftarrow\mathcal{S}\left(\tilde{z}_{0}^{(t)},z_{t},t,\hat{\epsilon}_{t}\right) 15: end for 16: \hat{I}_{0}\leftarrow\mathcal{D}(z_{0}) 17: for i in (0,s) do \triangleright Post processing in image space 18: \hat{I}_{0}\leftarrow\hat{I}_{0}-\eta\nabla_{\hat{I}_{0}}\left\{\mathcal{L}_{\text{SPL}}(I_{\text{src}},\hat{I}_{0})+\lambda\mathcal{L}_{\text{CPL}}(I_{\text{src}},\hat{I}_{0})\right\} 19: end for Algorithm 2 Iterative Guided Mask Upsampling 1: Initial cross-attention map M_{init} , reference image I , target size T , initial radius r , radius increment \Delta r 2: Refined mask M 3: M\leftarrow\text{Binarize}(M_{init},0.4) 4: s\leftarrow\text{size}(M) \triangleright Get initial resolution. 5: while s