On Exact Editing of Flow-Based Diffusion Models
Zixiang Li, Yue Song, Jianing Peng, Ting Liu, Jun Huang, Xiaochao Qu, Luoqi Liu, Wei Wang, Yao Zhao, Yunchao Wei · 2025
Read On Exact Editing of Flow-Based Diffusion Models by Zixiang Li, Yue Song, Jianing Peng, Ting Liu, Jun Huang, Xiaochao Qu, Luoqi Liu, Wei Wang, Yao Zhao, Yunchao Wei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24015.txt.
Introduction
Zixiang Li and nine co-authors fix a drift problem in image editing with flow-based diffusion models. Modern editors transform a source image directly toward a target distribution without inversion. Errors in the latent velocity accumulate along the path, so edits lose the source's structure or miss the intended meaning. Their framework, Conditioned Velocity Correction, recasts editing as distribution transport driven by a known source prior.
The velocity at each step splits into two branches: one that preserves the source trajectory's structure, and one that steers semantically toward the target. Because the conditional velocity field carries a measurable absolute error against the true trajectory, the authors correct it with a posterior-consistent update grounded in empirical Bayes and Tweedie estimation. Across diverse editing tasks, the corrected flow delivered higher fidelity, better semantic alignment, and more reliable behavior than existing methods.
Catalog record
| CREATOR | Zixiang Li, Yue Song, Jianing Peng, Ting Liu, Jun Huang, Xiaochao Qu, Luoqi Liu, Wei Wang, Yao Zhao, Yunchao Wei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24015
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On Exact Editing of Flow-Based Diffusion Models Zixiang Li, Yue Song, Jianing Peng, Ting Liu, Jun Huang, Xiaochao Qu, Luoqi Liu, Wei Wang, Yao Zhao, Yunchao Wei Abstract Recent methods in flow-based diffusion editing have enabled direct transformations between source and target image distribution without explicit inversion. However, the latent trajectories in these methods often exhibit accumulated velocity errors, leading to semantic inconsistency and loss of structural fidelity. We propose Conditioned Velocity Correction (CVC) , a principled framework that reformulates flow-based editing as a distribution transformation problem driven by a known source prior. CVC rethinks the role of velocity in inter-distribution transformation by introducing a dual-perspective velocity conversion mechanism. This mechanism explicitly decomposes the latent evolution into two components: a structure-preserving branch that remains consistent with the source trajectory, and a semantically-guided branch that drives a controlled deviation toward the target distribution. The conditional velocity field exhibits an absolute velocity error relative to the true underlying distribution trajectory, which inherently introduces potential instability and trajectory drift in the latent space. To address this quantifiable deviation and maintain fidelity to the true flow, we apply a posterior-consistent update to the resulting conditional velocity field. This update is derived from Empirical Bayes Inference and Tweedie correction, which ensures a mathematically grounded error compensation over time. Our method yields stable and interpretable latent dynamics, achieving faithful reconstruction alongside smooth local semantic conversion. Comprehensive experiments demonstrate that CVC consistently achieves superior fidelity, better semantic alignment, and more reliable editing behavior across diverse tasks. Figure 1 : Conditional Velocity Correction (CVC) We propose an inversion-free, text-guided image editing method. This figure shows six editing examples, presenting how our method performs fine-grained modifications to visual content based on textual instructions. CVC can adopt various image editing tasks, including element addition and removal, attribute modification, concept conversion, and style transfer. The excellent results demonstrate our method’s superior ability to precisely control the image subject, environment, materials, and state. 1 Introduction Diffusion models have achieved remarkable success across a wide range of generative tasks, including image 30 ; 36 ; 11 ; 46 , video 20 ; 28 , 3D 33 ; 26 , and even music 16 generation. In image synthesis, the most common paradigm is text-to-image generation, where a textual prompt guides the model to create visually consistent content. Building upon this foundation, text-guided image editing 13 ; 17 ; 23 ; 21 ; 47 has emerged as an equally important research direction, enabling users to modify existing images according to new textual instructions. Most image editing methods with diffusion models follow a similar process. First, the source image is inverted into its corresponding noise representation, which is usually called inversion. Then, this noise is gradually changed during the generation process to achieve the desired modification. Some approaches perform editing by directly manipulating the model’s attention maps (e.g., Prompt-to-Prompt 13 , MasaCtrl 5 ), while the others employ spatial masks 9 or adapters 49 to localize edits and preserve identity. These approaches have proven effective within the U-Net-based diffusion architecture. Recently, Diffusion Transformers (DiTs) 32 have become the new backbone for generative models, showing impressive scalability and strong cross-modal capability. Early DiT-based editing methods still follow the same inversion-editing process. For instance, Stable-Flow 1 first inverts the source image into noise via inversion process. Then, it injects the attention features of the guidance into the key layers of the DiT model. RF-Inversion 37 achieves image inversion and editing by constructing controlled forward Ordinary Differential Equations (ODEs) and reverse ODEs. However, the inversion step in such pipelines face fundamental challenge: mapping an image back to its noise representation and reconstructing it often leads to degraded fidelity 29 . Although numerous subsequent works 44 ; 23 ; 37 have been proposed to address this limitation, they theoretically cannot attain the lossless inversion. To overcome this limitation, FlowEdit proposes a new paradigm that avoids explicit inversion and instead constructs a direct transport path between the source and target distributions. Rather than repeatedly mapping between image and noise, FlowEdit defines a continuous flow path that moves directly from the source to the target manifold. Nevertheless, this direct transformation introduces new challenges: the intermediate latent states between the source and the target are not matched. This issue originates from an inherent limitation of FlowEdit, which applies different guidance strengths to the source and target branches. The mismatch in guidance scales introduces an unavoidable and non-negligible error from the unconditional term. Moreover, it leads to accumulated velocity errors over time and ultimately causes structural distortion or unwanted edits in the final result. To address these issues, we introduce Conditioned Velocity Correction (CVC) , a principled framework that redefines flow-based image editing through physically interpretable and mathematically consistent velocity modeling. Our key insight is to solve the structural distortions observed in prior flow-editing methods originate from accumulated velocity errors during transport between the source and target distributions. Instand of merely blending source and target velocities, CVC reconstructs the latent velocity field from two complementary perspectives: one branch preserves the structural consistency of the source image, while the other captures semantic deviation toward the target prompt. This dual-branch design ensures that the editing process remains faithful to the ideal manifold while achieving controlled semantic transformation. To further address the inherent bias in velocity field estimation, we reformulate velocity computation as an inverse problem under Empirical Bayes Inference, conditioned on the known source prior. Within this formulation, residual trajectory errors are adaptively corrected through a Tweedie-based posterior refinement. We regularize the integration process with an alignment loss that penalizes discrepancies between the predicted and corrected flows, enabling iterative refinement of the semantic path and ensuring both geometric fidelity and semantic precision. Through this formulation, CVC bridges the gap between the empirical stability of flow-based editing and the theoretical consistency of diffusion dynamics. It achieves both faithful reconstruction and smooth semantic transitions, allowing image edits to remain visually coherent, semantically aligned, and physically grounded within the diffusion manifold. Our main contributions are summarized as follows: 1. We provide a new interpretation of velocity misalignment in flow-based image editing, revealing that accumulated unconditional errors are the key cause of structural distortion. 2. We propose Conditioned Velocity Correction (CVC) , which consists of two components: a dual-perspective velocity computation module that jointly models structural preservation and semantic deviation, and a posterior-consistent velocity correction module that refines the reconstructed flow using Tweedie-based Empirical Bayes Inference. 3. Comprehensive experiments demonstrate that CVC achieves higher reconstruction accuracy and more stable semantic transitions compared to existing flow-editing baselines, validating its theoretical soundness and practical effectiveness. 2 Related Work 2.1 Image Generation and Editing Image generation has made rapid progress in recent years. Early GAN-based methods, such as GAN 12 and StyleGAN 18 ; 19 , present the foundation for controllable and high-quality image synthesis. The development of diffusion models 14 ; 39 further push the field forward, improving resolution, realism, and diversity. Representative models like GLIDE 30 , Imagen 38 , DALL·E 2 35 , and Stable Diffusion 36 achieve remarkable performance on many generation tasks. Based on these models, a lot of diffusion-based editing methods have been proposed. Prompt-to-Prompt (P2P) 13 controls image edits by adjusting cross-attention maps. Pix2Pix-Zero 31 performs