Towards Robust Sequential Decomposition for Complex Image Editing

Zilai Zeng, Mingdeng Cao, Zijie Li, Xiaochen Lian, Yichun Shi, Peihao Zhu, Chen Sun, Peng Wang · 2026

Read Towards Robust Sequential Decomposition for Complex Image Editing by Zilai Zeng, Mingdeng Cao, Zijie Li, Xiaochen Lian, Yichun Shi, Peihao Zhu, Chen Sun, Peng Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09233.txt.

Introduction

Zeng and seven colleagues make complex image editing robust by breaking tangled tasks into simpler steps, without letting errors compound. Single turn editing tries all edits in one pass, often gets the task wrong and causes stray edits, while step by step editing splits the task but suffers piling errors. The team studies both methods under one in context framework, asking how step by step splits balance against error build up drawbacks.

They build fake data that creates tasks of varied hard levels, making a large set of high quality split runs. Training on fake data shows well designed methods let step splits keep getting better even as task hard levels rise. Split skills from fake tasks transfer to real images when co trained with real world data, showing sim to real gains. The work promises broader complex editing across domains through careful splits, not single pass heroics.

Catalog record

CREATORZilai Zeng, Mingdeng Cao, Zijie Li, Xiaochen Lian, Yichun Shi, Peihao Zhu, Chen Sun, Peng Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09233

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Robust Sequential Decomposition for Complex Image Editing Zilai Zeng, Mingdeng Cao, Zijie Li, Xiaochen Lian, Yichun Shi, Peihao Zhu, Chen Sun, Peng Wang Abstract Recent advances in visual generative models have enabled high-fidelity image editing guided by human instructions. However, these models often struggle with complex instructions involving combinatorial editing operations or inter-step dependencies. This difficulty stems from the limitations of two canonical paradigms: (1) single-turn editing, which attempts to apply all instructed edits in one pass, often fails to parse the complex instruction accurately and causes undesired edits; and (2) sequential editing can decompose the task into simpler steps but suffers from compounding errors introduced by the sequential execution, leading to low-fidelity results. To derive a robust solution for complex image editing, we examine editing behaviors of different paradigms under a unified in-context editing framework, and study how the benefits of sequential decomposition can be balanced against its error-accumulation drawbacks. We further develop a synthetic data pipeline that constructs editing tasks of varying instruction complexity, allowing us to curate a large-scale editing dataset with high-quality decomposed sequences. By finetuning on synthetic data, we discovered that with properly designed editing paradigms, sequential decomposition yields robust improvements even as task complexity increases. Furthermore, the decomposition skills learned from synthetic tasks can transfer to real images by co-training with real-world editing data, demonstrating the promise of sim-to-real generalization for tackling complex image editing across broader domains. 1 Introduction Image generative models have achieved remarkable progress in synthesizing high-quality visual content aligned with user instructions, unlocking a wide range of applications in text-to-image (T2I) generation Ramesh et al. 2021 ; Esser et al. 2024 ; Betker et al. ; Nichol et al. 2022 ; Ramesh et al. 2022 ; Saharia et al. 2022 and user instruction-guided image editing Brooks et al. 2022 ; Zhao et al. 2024 ; Wei et al. 2024 . While state-of-the-art T2I models can generate visually compelling images from complex instructions, their counterparts for image editing are often limited to handling a single type of edit or restricted target regions , standing in stark contrast to real-world scenarios that usually demand more compositional modifications. Figure 1 : Single-Turn Editing versus Sequential Editing with a complex instruction: ”Move the speaker onto the modern coffee table, move the all-purpose cleaner forward, then add a shaver on the floor near the speaker, and replace the lightbulb with a basketball”. Incorrect edits are labeled with red boxes on the image. To enable complex image editing from instructions containing combinatorial editing operations, a model must first identify distinct edits specified by the instruction and then apply them to their corresponding target regions in the source image. Two straightforward paradigms can be considered for this task: (1) single-turn editing, which performs all required edits in a single step, and (2) sequential editing, which decomposes the complex instruction into simpler sub-edits and executes them sequentially. However, neither paradigm reliably produces accurate results under complex instructions. Single-turn editing often yields incomplete or even wrong edits because the model must simultaneously recognize and execute multiple operations from a compounded instruction. Conversely, although sequential editing intuitively simplifies the task through decomposition, it in practice suffers from severe error accumulation Yang et al. 2025 ; Zhang et al. 2023 ; Zhou et al. 2025 ; Qu et al. 2025 ; Sheynin et al. 2023 , where inaccuracies introduced in early steps propagate through subsequent ones, ultimately degrading the fidelity of the final result. As shown in Figure 1 , Gemini 2.5 Flash Image Team 2025 fails to perform correct edits in either single-turn or sequential editing, with sequential editing yielding more undesired modifications. In this work, we aim to develop a robust solution that addresses complex image editing via sequential decomposition. The core challenge lies in achieving an effective trade-off between the advantages of multi-step instruction decomposition and its accompanying issues of error accumulation. To examine this trade-off, we study sequential editing through the lens of in-context editing framework OpenAI 2025 ; Qu et al. 2025 , where a multimodal model generates interleaved sequences of decomposed instructions and their corresponding edited images while conditioning on the full history of contextual inputs. This formulation unifies different editing paradigms within a single model and enables a fair comparison of their behaviors. More importantly, by conditioning each edit on rich contextual history, in-context editing makes sequential decompositions more informed and adaptive to intermediate outcomes, while opening up more editing paradigms to explore. These properties make in-context editing a compelling framework for investigating and improving sequential decomposition in complex image editing scenarios. To profile editing paradigms under different task complexities, we develop a synthetic data generation pipeline in Blender, where editing tasks are curated by manipulating procedurally generated scenes with a set of atomic editing operations. The pipeline sequentially performs a varying number of editing operations to generate editing sequences, constructing complex tasks by treating the initial scene rendering as the source image and the concatenation of editing operations as the corresponding instruction. Since each operation deterministically modifies the scene, the pipeline can produce sequences of “decomposed” steps with consistently accurate edits, along with a reference target image rendered from the final scene. To further enhance task complexity, we introduce inter-step dependencies, in which the instruction of a later operation requires referencing results from earlier edits (as illustrated in Figure 2 ). This pipeline not only provides a controlled environment for analyzing editing behaviors, where we can modulate task complexity by adjusting the editing length and the dependency ratio, but also enables scalable curation of high-quality editing chains of arbitrary length and dependency without degradation. We instantiate our in-context editing framework with unified model BAGEL Deng et al. 2025 and finetune it on synthetic editing sequences curated from our pipeline to induce sequential decomposition behaviors. Using the finetuned model, we implement different editing paradigms during inference and evaluate their performance on unseen synthetic tasks of varying complexity. We find that several state-of-the-art editing models suffer performance drops when sequential editing is applied in a zero-shot manner; in contrast, decomposition capabilities learned through our approach not only improve editing performance compared to single-turn editing baselines, but also continue to benefit from additional decomposition steps. Notably, context-guided sequential editing, a paradigm that can flexibly modulate the impact from prior editing results, consistently yields robust improvements across various settings and achieves the best GPT-evaluated scores on tasks with inter-step dependencies. This demonstrates that robust sequential decomposition can be enabled by training on high-quality editing sequences, integrated with a well-designed inference paradigm. Furthermore, to transfer the decomposition skills to the real-world domain, we finetune BAGEL on a mixture of synthetic editing sequences and real-world single-turn editing pairs. We find that the two-step context-guided editing successfully improves both instruction following and identity preservation on complex real-world editing tasks, highlighting the promise of sim-to-real transfer for enhancing complex image editing across broader domains. 2 Related Work Figure 2 : Overview of Synthetic Data Pipeline . We build the synthetic data pipeline on Blender, in which we construct complex editing tasks by sequentially applying editing operations on a randomly initialized scene. After constructing the editing chain, we take the initial and final rendering of the scene as the editing pair

Read the complete text