InstGenIE : Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling

Xiaoxiao Jiang †∗ , Suyi Li †∗ , Lingyun Yang † , Tianyu Feng † , Zhipeng Di, Weiyi Lu, Guoxuan Zhu, Xiu Lin, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang † † Hong Kong University of Science and Technology Alibaba Group · 2025

Read InstGenIE : Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling by Xiaoxiao Jiang †∗ , Suyi Li †∗ , Lingyun Yang † , Tianyu Feng † , Zhipeng Di, Weiyi Lu, Guoxuan Zhu, Xiu Lin, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang † † Hong Kong University of Science and Technology Alibaba Group (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20600.txt.

Introduction

Xiaoxiao Jiang and co-authors speed up AI image editing in a 2025 systems paper. Cloud editing tools change only the masked part of a picture, leaving the rest untouched, yet standard serving recomputes everything. Their system exploits that sparsity by caching intermediate results for the untouched regions and reusing them.

A pipeline overlaps cache loading with computing so no time bubbles remain. A rolling batching scheme lets new requests join mid-flight instead of waiting, and a load balancer evens out the uneven work that varied masks create. Together the tricks beat leading diffusion serving systems. The authors frame masked sparsity as the key efficiency lever.

Catalog record

CREATORXiaoxiao Jiang †∗ , Suyi Li †∗ , Lingyun Yang † , Tianyu Feng † , Zhipeng Di, Weiyi Lu, Guoxuan Zhu, Xiu Lin, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang † † Hong Kong University of Science and Technology Alibaba Group
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-20600

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InstGenIE : Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling Xiaoxiao Jiang †∗ , Suyi Li †∗ , Lingyun Yang † , Tianyu Feng † , Zhipeng Di, Weiyi Lu, Guoxuan Zhu, Xiu Lin, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang † † Hong Kong University of Science and Technology Alibaba Group Abstract . Generative image editing using diffusion models has become a prevalent application in today’s AI cloud services. In production environments, image editing typically involves a mask that specifies the regions of an image template to be edited. The use of masks provides direct control over the editing process and introduces sparsity in the model inference. In this paper, we present InstGenIE , a system that efficiently serves image editing requests. The key insight behind InstGenIE is that image editing only modifies the masked regions of image templates while preserving the original content in the unmasked areas. Driven by this insight, InstGenIE judiciously skips redundant computations associated with the unmasked areas by reusing cached intermediate activations from previous inferences. To mitigate the high cache loading overhead, InstGenIE employs a bubble-free pipeline scheme that overlaps computation with cache loading. Additionally, to reduce queuing latency in online serving while improving the GPU utilization, InstGenIE proposes a novel continuous batching strategy for diffusion model serving, allowing newly arrived requests to join the running batch in just one step of denoising computation, without waiting for the entire batch to complete. As heterogeneous masks induce imbalanced loads, InstGenIE also develops a load balancing strategy that takes into account the loads of both computation and cache loading. Collectively, InstGenIE outperforms state-of-the-art diffusion serving systems for image editing, achieving up to 3 × 3\times higher throughput and reducing average request latency by up to 14.7 × 14.7\times while ensuring image quality. * * footnotetext: Equal contribution 1. Introduction Diffusion models are making significant strides in generative AI art, enabling the creation of high-quality, contextually accurate images. One of the daily-use image generation tasks is image editing, which modifies specific regions of an existing image template to achieve a desired outcome ( Couairon et al. 2023 ; HuggingFace 2025b ; Meng et al. 2022 ) . Image editing has a wide range of applications, from personal use to professional Photoshop ( Adobe 2025c ; Adobe 2025a ) , and has fostered various use cases, such as virtual try-on ( Xu et al. 2025 ; Choi et al. 2021 ) , face swapping ( Wang et al. 2024 ) , and image retouching ( Art 2023 ) . Due to its wide applicability, image editing has matured into a service offered to users by Adode and Midjourney ( Adobe 2025b ; Midjourney 2025 ) . In a recent public diffusion model serving trace ( Li et al. 2025b ) , 70% of requests require image editing service to edit or retouch an image. Similarly, in our production system, we collect a two-week trace that documents a large-scale image editing service using 20k GPU cards to generate 34 million images. Typically, users employ a mask alongside other input conditions, such as textual prompts and images, to edit an image template. The use of a mask provides direct control, enabling users to precisely specify a region of arbitrary shape that they wish to modify while leaving the surrounding areas untouched ( Choi et al. 2021 ; Couairon et al. 2023 ; Kim et al. 2024 ; HuggingFace 2025b ; Meng et al. 2022 ) . As illustrated in Fig. 1 , the mask acts as a guide in the image editing process and is particularly favored by production services that demand accurate editing. Notably, even if some image editing systems do not explicitly require users to provide masks, they will generate one based on other inputs, such as textual prompts, to facilitate the editing of a specific area in an image ( Yu et al. 2024 ; Art 2023 ) . From our conversation with the production team, most image editing services require the mask guidance, which is either provided by the users or imposed by the platforms. Figure 1. A virtual try-on example of image editing using a SDXL model on H800. InstGenIE achieves a model inference speedup of 1.7\times and ensures image quality. The Rightmost image : Naively disregarding unmasked regions in image editing will distort the output image. Despite the remarkable efficacy of image editing, serving their requests is challenging. In existing systems, the computational complexity of editing an image is roughly equivalent to that of generating an entirely new image ( von Platen et al. 2022 ; Li et al. 2025b ; Li et al. 2024 ; Liu et al. 2025 ) . This is because a diffusion model should model the relationships among all the images pixels in both the masked and unmasked regions to generate an image, using the attention mechanism ( Vaswani et al. 2017 ) : naively disregarding the unmasked region to paint the masked region solely can distort the output image, as shown in Fig. 1 . Consequently, the requests will suffer from the high computational load of diffusion models, resulting in high inference latency and low serving throughput ( Li et al. 2024 ; Li et al. 2025b ; Agarwal et al. 2024 ) . For example, generating a 1024 \times 1024 image with the SDXL model ( Podell et al. 2024 ) requires 676T FLOPs ( Li et al. 2024 ) , saturating a high-end GPU like A100 ( Li et al. 2025b ) . To address this challenge, existing approaches suggest leveraging multiple GPUs to accelerate the diffusion model inference ( Li et al. 2024 ; Li et al. 2025b ) or reusing intermediate activations in the inference process to skip computations during inference ( Agarwal et al. 2024 ; Yu et al. 2024 ; Liu et al. 2025 ) . However, employing multiple GPUs can negatively impact throughput, as existing work ( Li et al. 2024 ) achieves only a 2.8\times speedup using 8 \times more GPUs. Worse, blindly skipping computation can degrade image quality ( Li et al. 2025b ) , which we will show in § 6.2 . In addition to the high computational load, there has been limited attention given to the batching and routing of requests within diffusion model serving systems, resulting in a significant optimization gap. Existing systems ( Diffusers 2025 ; Li et al. 2025b ; Li et al. 2024 ) typically employ a static batching policy ( Anyscale 2025 ) , where the running batch size remains fixed until its inference completes. As a result, requests that arrive while the model is executing inference cannot be processed until the current execution concludes, leading to prolonged queuing times that can increase tail serving latency by 35% (§ 6.4 ). Moreover, blindly applying optimized strategies, such as continuous batching ( Kwon et al. 2023 ; Yu et al. 2022 ) , to image editing serving systems can yield suboptimal performance. Additionally, image editing requests vary in the size of utilized mask, as demonstrated by our characterization studies (§ 2.2 ), and this heterogeneity should be considered by the request routing algorithm. In

Read the complete text