The Promise of RL for Autoregressive Image Editing
Saba Ahmadi Rabiul Awal Ankur Sikarwar Amirhossein Kazemnejad, Ge Ya Luo Juan A. Rodriguez Sai Rajeswar, Siva Reddy Christopher Pal Benno Krojer Aishwarya Agrawal · 2025
Read The Promise of RL for Autoregressive Image Editing by Saba Ahmadi Rabiul Awal Ankur Sikarwar Amirhossein Kazemnejad, Ge Ya Luo Juan A. Rodriguez Sai Rajeswar, Siva Reddy Christopher Pal Benno Krojer Aishwarya Agrawal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01119.txt.
Introduction
Saba Ahmadi and co-authors test ways to make text-guided image editing more reliable in two thousand twenty-five. Modern tools can create fine images from long prompts, yet short edit requests still often fail. The team compares three routes inside one model that handles text and image tokens in a single stream.
Plain fine-tuning, step-by-step reasoning, and reinforcement learning were all tried. Learning from rewards, checked by a large vision and language model, worked best. Their model, called EARL, matches strong rival systems on a wide range of edits while trained on far less data. The code, data, and trained models were released in public for others to use.
Catalog record
| CREATOR | Saba Ahmadi Rabiul Awal Ankur Sikarwar Amirhossein Kazemnejad, Ge Ya Luo Juan A. Rodriguez Sai Rajeswar, Siva Reddy Christopher Pal Benno Krojer Aishwarya Agrawal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01119
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Promise of RL for Autoregressive Image Editing Saba Ahmadi Rabiul Awal Ankur Sikarwar Amirhossein Kazemnejad, Ge Ya Luo Juan A. Rodriguez Sai Rajeswar, Siva Reddy Christopher Pal Benno Krojer Aishwarya Agrawal Abstract While image generation techniques are now capable of producing high quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner. We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EARL : E diting with A utoregression and RL , a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github.com/mair-lab/EARL . 1 Introduction With internet-scale image-text pairs [ 43 ] and diffusion models [ 41 , 39 , 24 ] we have seen impressive progress on open-ended image generation in recent years. At this point, the latest text-to-image models can often adhere to detailed prompts that span several sentences [ 5 , 14 ] . However, synthesizing images from a prompt alone is often not sufficient for end-users and broader ML applications. In reality, a person might want to alter highly specific details in a given image instead of creating one from scratch. Beyond direct user applications, e.g. in the domains of robotics and planning, one might want to “imagine into the future” with an image editing model acting as a simulator [ 62 , 27 , 8 ] , e.g. “how does this scene look like if the robot pushes the mug?”. In both cases, the model must faithfully preserve all details of the original image while modifying the elements intended for editing. From a capability perspective, most current editing models [ 64 , 9 , 60 ] cover arguably simpler object and attribute edits ( replace, change color, add , …), yet only few works [ 27 , 47 ] tackle more complex edits that require e.g. action understanding or reasoning (spatial, counting, physical dynamics). From a modeling perspective, a standard recipe to improve editing is to apply supervised fine-tuning (SFT) to a diffusion-based image generation model [ 9 , 64 , 60 ] , rarely incorporating more recent post-training methods such as reinforcement learning (RL). A parallel line of work introduces additional bounding-box conditioning, either explicitly provided by the user [ 31 ] or implicitly predicted [ 17 ] , leaving these methods far from an end-to-end solution. Hence, in this paper we ask: What is the most effective approach to address both simple and complex edits with a unified end-to-end model? And specifically, what are the key learning paradigms that can move the field forward? To this end, we conduct a series of experiments with three different learning paradigms (SFT, RL and chain-of-thought reasoning) and mixes of data. However diffusion-based models would not directly allow a consistent setup where all training approaches could be plugged in out of the box. For this reason we choose Emu3 [ 53 ] as our starting point, a fully autoregressive generative model which was pre-trained on captioning and image generation. Since Emu3 is a unified image and language generation model, we can easily use it to study CoT reasoning and online RL methods such as GRPO [ 44 ] , on top of SFT. We note that RL for visual generation with diffusion or flow-matching is non-trivial [ 7 , 32 ] . At the end of our exploration, we arrive at a simple recipe and propose EARL : E diting with A utoregression and RL , a fully autoregressive generative model that trains on simple and complex editing data during the SFT and RL post-training stages. Specifically, we find the combination of GRPO with a strong MLLM-based verifier to be the most effective. While variants of CLIP-Score are more commonly used verifiers [ 30 , 32 ] , they often require fine-tuned on preferences and lack fine-grained understanding [ 63 , 1 ] . Instead, we identify large MLLMs with fine-grained understanding such as Qwen2.5-VL-72B as effective verifiers for a broad range of edits. We empirically show that EARL performs well across many types of edits by evaluating on 6 diverse benchmarks in both IID and OOD settings. We achieve better results than prior state-of-the-art models on the OmniEdit [ 55 ] , AURORA [ 27 ] , and VisMin [ 2 ] benchmarks. Moreover, our method outperforms the strongest prior work [ 60 ] , despite using five times less data, and also surpasses baselines that use a comparable amount of data, such as [ 27 ] . See Figure 1 for samples from EARL before and after applying our RL recipe, showing improved performance even on challenging tasks that require spatial understanding. Notably, modeling textual and visual generation as one autoregressive stream has emerged as a new exciting paradigm [ 53 , 48 , 49 , 59 ] , and we push the frontier of such models for the image editing task , outperforming prior related work EditAR [ 38 ] . Finally, we highlight a surprising finding from our in-depth analysis of different training paradigms: teaching the model to explicitly reason about the intermediate steps (chain-of-thought style reasoning) before generating the actual edit does not seem to improve performance, and sometimes it even hurts. Our contributions are as follows: 1. We release EARL : A unified end-to-end editing model that performs well on the whole spectrum of edit types. 2. EARL outperforms the strongest open-source diffusion baseline (Omnigen [ 60 ] ), while also outperforming a more comparable fully autoregressive editing model EditAR [ 38 ] . 3. A simple yet effective online RL pipeline for training an autoregressive editing model. 4. A systematic analysis of how training objectives interplay and at what stages to bring in simple vs. complex edits. The results interestingly reveal that various CoT reasoning settings do not bring any clear improvements. Our findings also show that complex edits are not beneficial during the SFT stage but are effective in the RL post-training stage. Figure 1: Qualitative comparison between SFT-only and EARL across diverse editing instructions. EARL extends the SFT model by leveraging reinforcement learning to better align image edits with natural language prompts. While both models handle simple edits reasonably well, EARL exhibits clear improvements in precise editing on simple as well as complex edit instructions. Simple edit instructions are shown in blue, and complex edit instructions are shown in pink. 2 Related Work Image Editing Models. Enabled by the development of diffusion models with open-ended text-conditioning for image generation [ 42 , 39 ] , image editing models can now