DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing

Zixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, Yao Zhao · 2025

Read DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing by Zixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, Yao Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02560.txt.

Introduction

Zixiang Li and five co-authors improve the first step of AI photo editing, in this 2025 paper: turning a real picture back into the noise that would have generated it. Diffusion editors can only change a photo if they first invert it, recovering its latent code. Inversion has a stubborn trade-off: pin the code down tightly and the rebuilt photo is faithful but refuses to be edited; loosen it and edits flow while the original's identity drifts.

Their method, dual-conditional inversion, steers the search with two anchors at once: the words of the source prompt and the look of the reference image. Framed as a fixed-point problem, the inversion shrinks both the gap in the latent code and the reconstruction error. Experiments report the best current balance of fidelity and editability. For prompt-based photo surgery, a steadier anchor means the patient still looks like themselves afterward.

Catalog record

CREATORZixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, Yao Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-02560

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing Zixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, Yao Zhao Abstract Diffusion models have achieved remarkable success in image generation and editing tasks. Inversion within these models aims to recover the latent noise representation for a real or generated image, enabling reconstruction, editing, and other downstream tasks. However, to date, most inversion approaches suffer from an intrinsic trade-off between reconstruction accuracy and editing flexibility. This limitation arises from the difficulty of maintaining both semantic alignment and structural consistency during the inversion process. In this work, we introduce Dual-Conditional Inversion (DCI) , a novel framework that jointly conditions on the source prompt and reference image to guide the inversion process. Specifically, DCI formulates the inversion process as a dual-condition fixed-point optimization problem, minimizing both the latent noise gap and the reconstruction error under the joint guidance. This design anchors the inversion trajectory in both semantic and visual space, leading to more accurate and editable latent representations. Our novel setup brings new understanding to the inversion process. Extensive experiments demonstrate that DCI achieves state-of-the-art performance across multiple editing tasks, significantly improving both reconstruction quality and editing precision. Furthermore, we also demonstrate that our method achieves strong results in reconstruction tasks, implying a degree of robustness and generalizability approaching the ultimate goal of the inversion process. 1 Introduction Figure 1: Pipelines of different inversion methods in diffusion-based image editing. Each sub-figure illustrates the specific process: (a) DDIM inversion; (b) NTI and NPI; (c) DirectInv; (d) SPDInv; (e) our Dual-Conditional Inversion(DCI). Obviously, DCI significantly reduces both latent noise gap( D_{noi} ) and reconstruction error( D_{rec} ). Diffusion models have made significant progress in the field of generative artificial intelligence. Among them, latent Diffusion Models (LDMs) LDM perform the diffusion process in a compressed latent space rather than the pixel space, enabling more efficient and high-quality image generation and editing. This architectural design has made LDMs a powerful and flexible backbone for a wide range of downstream tasks, such as text-to-image generation nichol2021glide ; ramesh2022hierarchical ; saharia2022photorealistic , image editing meng2021sdedit ; masactrl ; plugandplay ; brooks2023instructpix2pix , image restoration lin2024diffbir ; wang2024exploiting ; wu2024one , style transfer wang2023stylediffusion ; wang2024instantstyle ; chung2024style , etc . In the image editing tasks, the editing is achieved by manipulating the diffusion latent representations. However, in most cases, the corresponding latent representation for a given image is not directly available, which means that we must first perform an inversion process to obtain their latent representations. The earliest inversion method is DDPM ho2020denoising , and it has inspired the development of numerous related methods tsaban2023ledits ; Brack2023LEDITSLI ; huberman2023edit . DDPMs add random noise at each timestep, which leads to the loss of information contained in the original image, resulting in poor reconstruction and editing effects. DDIM inversion song2020denoising ; dhariwal2021diffusion reformulates the diffusion process to be deterministic as solving an implicit equation under the assumption that consecutive points along the denoising trajectory remain close. However, in practice, especially when using a limited number of denoising steps, this assumption often breaks down, leading to significant inaccuracies in the inversion results. In order to improve the reconstruction effect of DDIM inversion, multiple works have proposed effective optimization methods, such as null-text embedding(NTI) NTI and negative prompt(NPI) NPI in the inversion process. As illustrated in figure 1 , both NTI and NPI attempt to reduce the reconstruction gap( D_{rec} ) by optimizing the text embeddings. In the meanwhile, the researchers have proposed some alternative solutions from a non-optimization perspective. For instance, DirectInv directinv introduces a target-aware branch to correct the source branch trajectory, improving reconstruction quality. It performs well especially in terms of content preservation, and it is faster than optimization-based inversion methods. Renoise garibi2024renoise is based on the linear assumption that the direction from z_{t} to z_{t+1} can be approximated by the reverse direction from z_{t} to z_{t-1} . By calculating the direction from z_{t} to z_{t+1} multiple times and taking the average, a more accurate direction from z_{t} to z_{t+1} could be obtained. SPDInv li2024source SPDInv uses an optimization method to bridge the latent gap on each timestep, but the improvement of reconstruction gap ( D_{rec} ) is limited. Although these methods have achieved certain success, they still face an intrinsic trade-off between reconstruction accuracy and editing flexibility. As illustrated in figure 1 , such approaches struggle to reconcile semantic precision with structural consistency, particularly when textual supervision is sparse or ambiguous. In this work, we present Dual-Conditional Inversion (DCI) , a new perspective on diffusion-based image editing that unifies text and image conditioned inversion within a fixed-point optimization framework. DCI addresses this limitation by introducing a dual-conditioning mechanism: it jointly leverages the source prompt p_{s} and the reference image x_{0} to guide the inversion process. At the core of our formulation is a two-stage iterative procedure. The first stage, reference-guided noise correction , refines the predicted noise at each timestep by anchoring it to a visually grounded reference derived from the source image. The second stage, fixed-point latent refinement , imposes self-consistency by optimizing each latent variable z_{t} as a fixed point of the generative trajectory defined by DDIM dynamics. Formally, we cast inversion as a dual-conditioned fixed-point optimization problem that minimizes two objectives: (1) the discrepancy between the predicted and reference noise vectors across timesteps, and (2) the reconstruction error between the generated image and the original reference. This formulation not only improves inversion stability but also yields latent representations that are inherently editable and semantically aligned. To sum up, our framework enables a plug-and-play integration with a variety of existing diffusion models, requiring neither retraining nor any modification to the original model. Through extensive experiments across multiple editing tasks, DCI achieves superior reconstruction quality and editing fidelity when compared to prior inversion baselines. Moreover, we demonstrate that the proposed dual-conditional fixed-point formulation facilitates stable convergence and generalizes well across a wide range of editing scenarios, highlighting the robustness and scalability of the proposed approach. 2 Related Work 2.1 Image Editing with Diffusion Models In recent years, a large number of works based on diffusion models in the field of image editing demonstrate significant potential and adaptability across diverse tasks. These methods utilize diverse forms of guidance, such as text prompts, image references and segmentation maps to achieve editing objectives. kwon2022diffusion ; kim2022diffusionclip ; couairon2022diffedit ; huang2023region ; li2025unsupervised These advances better enable the ability to maintain editing precision and semantic consistency. The rapid development of diffusion models has significantly improved image generation capabilities. Among them, the widespread use of models such as GLIDE nichol2021glide , Imagen saharia2022photorealistic , DALL·E2 ramesh2022hierarchical , and Stable Diffusion(SD) LDM has gradually expanded downstream tasks based on image generation. Prompt-to-Prompt(P2P) hertz2022prompt modifies cross-attention maps in diffusion models to enable text-driven image editing while preserving spatial structure through localized prompt adjustments. Pix2pix-zero parmar2023zero achieves zero-shot image-to-image translation by aligning latent features with text guidance. Plug-and-Play plugandplay integrates task-specific modules into pretrained diffusion backbones without retraining. MasaCtrl masactrl enhances real-time spatial control in diffusion models by injecting mask-guided attention constraints for precise region-specific manipulation. IP-Adapter ye2023ip injects visual features into the attention mechanism, enabling personalized generation without fine-tuning. ControlNet zhang2023adding introduces an auxiliary

Read the complete text