LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing
Achint Soni, Meet Soni, Sirisha Rambhatla · 2025
Read LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing by Achint Soni, Meet Soni, Sirisha Rambhatla (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21541.txt.
Introduction
Achint Soni, Meet Soni, and Sirisha Rambhatla present in 2025 a method for editing images with words while keeping the rest of the picture intact. Text-guided editing should change only the region the words describe. Yet the masks used to find that region come from attention maps that chase meaning, not place. So edits often smear or warp the image.
Their system, LOCATEdit, sharpens those attention maps with a graph-based step built on the graph Laplacian, which respects the layout of the image. The cleaner mask steers the edit to the right spot. The goal is edits that follow instructions without breaking the scene around them.
Catalog record
| CREATOR | Achint Soni, Meet Soni, Sirisha Rambhatla |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-21541
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing Achint Soni, Meet Soni, Sirisha Rambhatla Abstract Text-guided image editing aims to modify specific regions of an image according to natural language instructions while maintaining the general structure and the background fidelity. Existing methods utilize masks derived from cross-attention maps generated from diffusion models to identify the target regions for modification. However, since cross-attention mechanisms focus on semantic relevance, they struggle to maintain the image integrity. As a result, these methods often lack spatial consistency, leading to editing artifacts and distortions. In this work, we address these limitations and introduce LOCATEdit , which enhances cross-attention maps through a graph-based approach utilizing self-attention-derived patch relationships to maintain smooth, coherent attention across image regions, ensuring that alterations are limited to the designated items while retaining the surrounding structure. LOCATEdit consistently and substantially outperforms existing baselines on PIE-Bench, demonstrating its state-of-the-art performance and effectiveness on various editing tasks. Code can be found on https://github.com/LOCATEdit/LOCATEdit/ 1 Introduction Diffusion models have become popular for image generation, yet practical applications demand precise control for editing. Text-guided editing techniques [ 49 , 43 , 4 ] have emerged as powerful tools to facilitate such modifications across domains, from digital art [ 12 , 8 , 17 , 38 ] to medical imaging [ 42 , 25 ] , enabling more intuitive image manipulation through natural language prompts. However, prompt-driven editing is often imprecise [ 15 , 2 , 6 ] . To attain precise control in text-guided image editing, recent studies use masks derived from cross-attention maps; however, inaccuracies in these maps can result in edits spilling over unintended regions, causing problems such as object identity loss [ 18 , 35 , 44 ] and background drift [ 23 , 49 ] . Because of this, techniques that depend exclusively on cross-attention could make global changes when only localized modifications are required [ 9 , 15 ] . These problems highlight the necessity for a method that precisely identifies editing areas without jeopardizing the integrity of the overall image. Recent methods have demonstrated improved mask accuracy through the utilization of cross- and self-attention masks, while simultaneously adopting the dual branch editing paradigm [ 43 , 49 ] . Additionally, they incorporate target image embeddings as auxiliary guidance derived from source image embeddings and the editing information contained in source-target prompt pairs. Despite these, challenges such as unintended spills continue to be a problem, which can be seen in Figure 2 . Our key observation is that naively combining cross-attention and self-attention results in significant information loss. Consequently, we propose to induce spatial consistency and precise identification of regions to be edited via a graph-based approach. Given graphs \mathcal{G}_{\text{src}} and \mathcal{G}_{\text{tgt}} for the source and target branch, respectively, each of these graphs is constructed using the respective Cross and Self-Attention, hence CASA graphs, encoding cross-attention maps as nodes and self-attention relationships as weighted edges. With this abstraction, these graphs intrinsically depict the structure of the image, thereby connecting local and global contexts, while also maintaining the semantic relevance. We explicitly enforce graph structure by proposing a graph Laplacian regularizer on \mathcal{G}_{src} and \mathcal{G}_{tgt} to impose spatial consistency, motivated by the effectiveness of Laplacian regularization in image denoising and mesh editing [ 27 , 37 ] . Prior works on segmentation and spatial regularization [ 41 , 51 ] also demonstrate that this Laplacian constraint effectively maintains object boundaries and preserves local detail, thus disentangling the areas of interest from unrelated regions. Furthermore, Belkin and Niyogi [1] and Lim et al. [22] illustrate that this regularization also enhances the separation of semantic characteristics. By integrating a Laplacian smoothness factor into the diffusion process, LOCATEdit optimizes the attention values across interconnected patches without any additional training, hence reducing background drift and limiting global changes as can be seen in Figure . This ensures that modifications are confined to designated areas while preserving the overall structural integrity of the original image. Notably, this optimization admits a closed-form solution, hence eliminating the need for iterative refinement [ 43 ] . Figure 2 : Example of over-editing caused due to imprecise masks. Overall, our contributions can be summarized as follows: • CASA Graph: We introduce LOCATEdit , a method which encapsulates word-to-pixel relevance through pixel-to-pixel relationships by modeling attention maps with CASA graph. • Improved spatial consistency: By optimizing masks through graph Laplacian regularization on the CASA graph, we maintain object structure and confine changes to intended regions, hence minimizing distortions. • Disentangled and faithful editing: Leveraging Laplacian smoothing, LOCATEdit achieves precise semantic modifications while preserving the original image context, ensuring disentangled editing. 2 Related Work Recent advances in image editing have leveraged a range of conditioning modalities—including text, reference images, and segmentation maps—to drive semantic, structural, and stylistic modifications [ 13 ] . In this work, we focus specifically on text-guided image editing with an emphasis on preserving the original content and ensuring effective foreground-background disentanglement. 2.1 Text-guided Image Editing Early methods exploited the power of CLIP [ 30 ] to align images and text. For example, [ 16 ] fine-tuned diffusion models during reverse diffusion using a CLIP loss to adjust image attributes, though these approaches were limited to global changes and often suffered from degraded image quality. Later works such as DiffuseIT [ 19 ] and StyleDiffusion [ 46 ] improved performance by introducing semantic or style disentanglement losses; however, they are computationally expensive and typically confined to specific style modifications. More recent frameworks like InstructPix2Pix [ 3 ] preserve source content using text instructions, yet require carefully curated instruction-image pair datasets and supervised training. Additionally, methods that manipulate text embeddings for disentangled editing have been explored [ 47 ] , though they often yield only marginal improvements over earlier approaches. Collectively, these studies underscore both the promise and limitations of text-guided editing, motivating our work on refining attention maps to achieve spatially consistent and localized modifications. 2.2 Training Free Image Editing Recent advances in text-to-image synthesis [ 7 , 26 , 31 , 33 , 34 ] have enabled high-quality photorealistic image generation from text prompts. Building on these advances, several studies have proposed dual-branch, training-free approaches that leverage rich feature and attention maps from pre-trained diffusion models for image editing. These methods exploit signals from the source image’s diffusion process to drive content modification, obviating the need for additional model training while achieving remarkable success in altering image content. Notably, PRedITOR [ 32 ] generates a target CLIP embedding via a diffusion prior model but struggles with fine detail and background consistency. Other methods enhance structural control: P2P [ 9 ] replaces cross-attention maps to maintain spatial alignment, and PnP [ 40 ] injects spatial features and self-attention maps into decoder layers. Approaches like MasaCtrl [ 4 ] preserve structure through mutual self-attention, while editing-area grounding techniques and attention regularization losses are employed in DPL [ 49 ] and refined further in ViMAEdit [ 43 ] . Despite these advances, challenges in achieving precise localization and consistent edits persist, motivating our work. 2.3 Graph Laplacian In optimization and semi-supervised learning, Laplacian regularization promotes smooth variation along a graph, similar to how Conditional Random Fields (CRFs) refine segmentation