REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models
Gal Almog 1 Ariel Shamir 1 Ohad Fried 1 1 Reichman University, Israel · 2025
Read REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models by Gal Almog 1 Ariel Shamir 1 Ohad Fried 1 1 Reichman University, Israel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18989.txt.
Introduction
Gal Almog, Ariel Shamir, and Ohad Fried address a flaw in picture editing with artificial intelligence. Modern editing tools pass images back and forth between pixel form and a compact code form. Each round trip adds noise and flaws, so editing the same image again and again quickly degrades it. Some fixes keep the whole edit chain in code form, but then only a fixed set of edits is allowed.
The authors retrain the encoder that turns pictures into code so that quality survives many round trips. With their trained encoder, users can chain mixed edits freely, some driven by text prompts and some by standard tools, with each step building on the last. Tests across many editing tasks show cleaner results and more edits that land as intended. The team presents the work as a benchmark for this new multi-method editing task.
Catalog record
| CREATOR | Gal Almog 1 Ariel Shamir 1 Ohad Fried 1 1 Reichman University, Israel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18989
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models Gal Almog 1 Ariel Shamir 1 Ohad Fried 1 1 Reichman University, Israel Abstract While latent diffusion models achieve impressive image editing results, their application to iterative editing of the same image is severely restricted. When trying to apply consecutive edit operations using current models, they accumulate artifacts and noise due to repeated transitions between pixel and latent spaces. Some methods have attempted to address this limitation by performing the entire edit chain within the latent space, sacrificing flexibility by supporting only a limited, predetermined set of diffusion editing operations. We present a re - e ncode d ecode (REED) training scheme for variational autoencoders (VAEs), which promotes image quality preservation even after many iterations. Our work enables multi-method iterative image editing: users can perform a variety of iterative edit operations, with each operation building on the output of the previous one using both diffusion-based operations and conventional editing techniques. We demonstrate the advantage of REED-VAE across a range of image editing scenarios, including text-based and mask-based editing frameworks. In addition, we show how REED-VAE enhances the overall editability of images, increasing the likelihood of successful and precise edit operations. We hope that this work will serve as a benchmark for the newly introduced task of multi-method image editing. Our code and models will be available at: https://github.com/galmog/REED-VAE . \ConferencePaper \CGFccbyncnd † † teaser: REED-VAE (top) preserves image quality over multiple editing iterations, allowing users to perform multiple edit operations using a combination of frameworks and techniques. The Vanilla VAE (bottom) accumulates many artifacts and noise along the way, becoming very apparent once multiple iterative edit operations are performed. The total edit sequence consists of 14 steps, of which only the last 4 are shown here for brevity and to highlight the differences in the final picture. Four types of edit operations are performed: text-guided editing [ BHE23 ] , external editing (not diffusion-based), mask-guided editing [ AFL23 ] , and example-guided editing [ YGZ∗23 ] . 1 Introduction The ability to edit high-resolution images has long been a fundamental aspect of visual content creation, enabling artists and designers to achieve desired aesthetics and convey specific messages. Traditional image editing techniques range from basic adjustments such as color correction, cropping, and sharpening, to more advanced methods such as applying various filters and layering elements. More recently, diffusion models [ HJA20 ] have led to great advancements not only in high-resolution image generation, but also in editing methods that allow controllable manipulation of existing images. Diffusion-based editing models can receive various conditioning such as text instructions, reference images, and localization masks, and perform a wide range of tasks such as object addition/removal, object replacement, background replacement, and style or texture changes [ AFL23 , BHE23 , CVSC22 , HMT∗22 , NDR∗21 , YGZ∗23 ] . Although many of these achieved remarkable results, they center around single-operation editing procedures. Ideally, users should be able to integrate the strengths of both diffusion-based models and traditional editing techniques to manipulate images, while applying and interleaving several different editing frameworks. In practice, there exists an inherent problem in combining diffusion-based operations with traditional methods in the same editing session. This is because diffusion-based models primarily work in the latent space , while traditional methods are applied in the pixel space . Therefore, each time one wishes to switch between the two types of techniques, it is necessary either to encode or decode the image into the appropriate representation. The variational autoencoder (VAE) [ KW13 ] is the most common model used for this task. As we show in this paper, this iterative cycle of encoding and decoding destroys the quality of the image by accumulating noise and artifacts with each iteration (see Figure 1 ). We define multi-method iterative image editing as the process of performing multiple (e.g. more than 5 ) successive edit operations on an input image; each operation uses the previous output as its input, leveraging both diffusion-based models (latent space) and conventional editing techniques (pixel space). Our work aims to enable such multi-method iterative image editing by mitigating the artifacts introduced by the VAE in the iterative autoencoding process. We train a new VAE using a novel re - e ncode d ecode (REED) training scheme. Our training procedure utilizes an iterative training process together with dynamic incrementation and a first-step loss, that together improve image quality retention over many encode-decode iterations. We demonstrate the impact of replacing the Vanilla-VAE with our REED-VAE through experiments using a wide range of diffusion-based image editing models. As our REED-VAE is based on the architecture of the vanilla VAE used in Stable Diffusion [ RBL∗22 ] , it can be easily swapped into the vast majority of models. In addition to improving multi-method iterative image editing, our REED-VAE facilitates integration between different editing paradigms; for example, between GAN-based editing methods and diffusion-based methods. Furthermore, this improvement facilitates a seamless transition between editing with multiple different diffusion models that may have different latent spaces, for example, SD2 (4-channel) [ RBL∗22 ] and SD3 (16-channel) [ EKB∗24 ] . Beyond image editing, reducing the noise and artifacts accumulated through iterative VAE use has applications in other domains, such as NeRF editing methods in which it is very common to apply the VAE multiple times. In ED-NeRF [ PKY23 ] , it has been demonstrated that performing the NeRF editing process entirely in the latent space, thus avoiding repeated applications of the VAE, leads to significantly better results. We will publish our code and trained REED-VAE model in hope that they can be a useful contribution to the community. AAA. (1) Vanilla-VAE AAAa (2) REED-VAE Input 5 10 15 20 25 Num. Encode/Decode Iterations Figure 1: Even without a diffusion model in the pipeline, the Vanilla-VAE (top row) accumulates artifacts and exhibits significant distortion very quickly throughout encode-decode iterations. The tiger’s features lose their distinct shapes and edges, appearing more globular and less defined. The color palette is altered, with a noticeable increase in blue tones and a decrease in the richness of the orange and greens. Fine details such as the grass and the fur are largely lost or blurred. REED-VAE (bottom row), produces successive images that are robust to such artifacts and distortions. The tiger retains its shape, color, and surface details, demonstrating remarkably high fidelity to the original image. The subtle variations in orange and white hues are preserved, and fine elements remain visible. 2 Related work Latent Diffusion Models Diffusion models [ HJA20 , RDN∗22 , SCS∗22 ] are a class of deep generative models trained to convert random noise to an image sample from a given distribution. These models generate images in an iterative manner, removing a small amount of noise at each time step. To reduce the computational cost of high-resolution image generation with diffusion models, [ RBL∗22 ] introduced their latent diffusion model (LDM), which utilize a separately-trained autoencoding model that learns to map images to a latent space. This latent space is perceptually equivalent to the original pixel space, but significantly reduces computational complexity. LDMs achieve impressive results in both image generation and editing, and thus form the