PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal
Dinh-Khoi Vo, Van-Loc Nguyen, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le † · 2026
Read PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal by Dinh-Khoi Vo, Van-Loc Nguyen, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27555.txt.
Introduction
Vo and co-authors erase objects from photos with no extra training. Their PANDORA method works on stock text-to-image diffusion models. It needs no prompts, tuning, or slow optimization. One step silences attention keys tied to masked pixels. Background context then fills the gap on its own. A guidance step steers cleanup toward clean results. Many objects can vanish in a single pass. Tests show strong visual fidelity and plausible fills.
Catalog record
| CREATOR | Dinh-Khoi Vo, Van-Loc Nguyen, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le † |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26136 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27555
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal Dinh-Khoi Vo, Van-Loc Nguyen, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le † Abstract Removing objects from natural images is challenging due to difficulty of synthesizing semantically coherent content while preserving background integrity. Existing methods often rely on fine-tuning, prompt engineering, or inference-time optimization, yet still suffer from texture inconsistency, rigid artifacts, weak foreground–background disentanglement, and poor scalability for multi-object removal. We propose a novel zero-shot object removal framework, namely PANDORA, that operates directly on pre-trained text-to-image diffusion models, requiring no fine-tuning, prompts, or optimization. We propose Pixel-wise Attention Dissolution to remove object by nullifying the most correlated attention keys for masked pixels, effectively eliminating the object from self-attention flow and allowing background context to dominate reconstruction. We further introduce Localized Attentional Disentanglement Guidance to steer denoising toward latent manifolds favorable to clean object removal. Together, these components enable precise, non-rigid, prompt-free, and scalable multi-object erasure in a single pass. Experiments demonstrate superior visual fidelity and semantic plausibility compared to state-of-the-art methods. The project page is available at https://vdkhoi20.github.io/PANDORA . I Introduction Recent advances in diffusion models [ 3 , 8 ] have revolutionized generative image modeling, enabling high-quality image synthesis guided by textual prompts or other conditions. These models have opened new opportunities for semantic image editing [ 5 , 13 ] , particularly in longstanding and challenging object removal [ 19 , 18 , 4 ] that involves selectively erasing undesired regions from an image and reconstructing the void with content that blends naturally into the scene. This task goes beyond typical inpainting by requiring both precise object elimination and faithful restoration that aligns with the scene’s semantics and visual flow. Conventional object removal techniques have mainly utilized patch-based approaches [ 11 , 1 ] or GANs [ 19 , 12 ] . Patch-based methods often result in inconsistencies by filling in the masked areas with patches from other parts of the image, which can lead to unnatural blending with the surrounding regions. GANs, while enhancing realism, still face challenges with artifact generation and lack versatility in handling complex scenes. Fig. 1 : Our PANDORA enables prompt-free, fine-tuning-free object removal across various scenarios in a single forward. Without requiring training or textual prompts, our approach handles diverse and challenging removal settings, from a single object to multiple similar or distinct targets and even densely packed similar objects, while preserving background fidelity and structural consistency. Recent methods based on diffusion models [ 20 ] have significantly improved the realism of image synthesis. However, when adapted to object removal, these models often struggle to reliably eliminate target objects. One such adaptation, Stable Diffusion Inpainting [ 16 ] , extends the base model by conditioning on a binary mask to enable end-to-end inpainting. Despite its tailored design, the model frequently produces incomplete removals or introduces unexpected artifacts, even with considerable fine-tuning effort. Moreover, they require extensive prompt engineering [ 2 ] or fully fine-tuning such large-scale models [ 16 ] is often impractical in low-resource settings, limiting their accessibility and scalability in broader research applications. On the other hand, zero-shot methods [ 18 , 21 ] have been proposed but they often struggle to handle multi-object removal, leading to incomplete removals or noticeable artifacts. CPAM [ 21 ] can preserve background context but often fail to fully erase objects due to residual attention. Attentive Eraser [ 18 ] attempts to scale down attention weights but uses a uniform strategy that ignores pixel-wise correlations, often requiring sensitive manual tuning. To address these challenges, we propose PANDORA, a novel zero-shot object removal framework that operates directly on pre-trained diffusion models in a single pass, without any fine-tuning, prompt engineering, or inference-time optimization, thus fully leveraging their latent generative capacity for inpainting. The input image is inverted into the latent noise space via a preservation adaptation module [ 21 ] to ensure that the background regions remain unaffected during the object removal process, allowing targeted and artifact-free editing. We also introduce two synergistic core modules: Pixel-wise Attention Dissolution (PAD) dissolves object information at the attention level and Localized Attentional Disentanglement Guidance (LADG) complements it by reshaping the denoising trajectory in latent space, ensuring that masked regions are reconstructed coherently with their surrounding context. Particularly, PAD enables fine-grained control of self-attention by removing the top-k most correlated keys via percentile-based thresholding. This disconnects masked query pixels from dominant foreground regions, dissolving object information and enabling precise, natural reconstruction with coherent background content. Meanwhile, LADG steers latent noise away from object regions while preserving surrounding areas, refining the denoising trajectory in latent space to suppress residual artifacts and produce smoother, cleaner results. To evaluate multiple object removal methods, we build a new benchmark dataset, comprising 75 single-object samples, 17 multi-object samples, and 94 mass-similar object samples, with masks obtained through manual annotation or automatic extraction. Extensive experiments show that PANDORA excels in complex and cluttered scenes, supports multi-object erasure, and requires no prompts or training-time supervision. Our contributions are summarized as follows: • We propose PANDORA, a novel zero-shot object removal framework that directly leverages pre-trained diffusion models to remove objects in a single pass without fine-tuning, prompt engineering, or inference-time optimization. • We propose Pixel-wise Attention Dissolution (PAD) to dissolve object information by disconnecting masked query pixels from their most correlated keys in self-attention, enabling fine-grained removal and reconstruction with coherent background content. • We present Localized Attentional Disentanglement Guidance (LADG), which steers latent noise away from object regions while preserving unaffected areas, refining the denoising trajectory to remove residual artifacts and produce cleaner outputs. • We build a challenging benchmark dataset for evaluating object removal methods. Experiments demonstrate that our method achieves SOTA zero-shot object removal performance, excelling in both single- and multi-object scenarios. Fig. 2 : Overview of our proposed pipeline with an intuitive illustration of each module. The image is inverted into noise with intermediate latents stored and injected into BPA and PAD to preserve background and dissolve objects, respectively. Specifically, BPA restricts background queries to background regions, while PAD operates at the pixel level to constrain object queries to unrelated regions. Finally, LADG steers denoising away from masked object regions for seamless synthesis. Inputs: A mask M , the intermediate latent noises x^{i} , the target initial latent noise map x_{T} . Output: Erased Image I_{t} . 1 1. For t=T,T-1,\ldots,1 do: (a) \{\_,K_{i},V_{i}\}\overset{\text{get}}{\longleftarrow}\epsilon_{\theta}(x^{i}_{t},t) (b) \{Q,\_,\_\}\overset{\text{get}}{\longleftarrow}\epsilon_{\theta}(x_{t},t) (c) \text{self-attention}\overset{\text{inject}}{\longleftarrow}\text{BPA}(\text{ $Q,K_{i},V_{i},M$}) (d) \text{self-attention}\overset{\text{inject}}{\longleftarrow}\text{PAD}(\text{ $Q,K_{i},V_{i},M,top_{k}$}) (e) \epsilon\leftarrow\epsilon_{\theta}(x_{t},t,\text{self-attention}) (f) \epsilon\leftarrow\text{LADG}(\epsilon,\epsilon_{\theta}(x^{i}_{t},t),M,\alpha_{t}) (g) x_{t-1}\leftarrow\text{Sample}(x_{t},\epsilon) 2. End For. Return: VAE( x_{0} ). 2 Algorithm 1 PANDORA’s algorithm: Zero-Shot Object Removal II Proposed Method II-A Overview PANDORA performs zero-shot object removal directly on a pre-trained diffusion model. Given an input image I_{s} and a binary mask M specifying the target objects, the model produces an edited image I_{t} where the masked regions are erased and seamlessly reconstructed with contextually consistent background. The process begins with latent inversion to map the input image into the noise space while preserving unaffected regions in the denoising process. We then apply Pixel-wise Attention Dissolution (PAD) to disconnect masked query pixels from their most correlated keys, effectively dissolving object information at the attention level. Next, Localized Attentional Disentanglement Guidance (LADG) steers the denoising trajectory