Object-WIPER: Training-Free Object and Associated Effect Removal in Videos
Saksham Singh Kushwaha, Sayan Nag, Yapeng Tian, Kuldeep Kulkarni · 2026
Read Object-WIPER: Training-Free Object and Associated Effect Removal in Videos by Saksham Singh Kushwaha, Sayan Nag, Yapeng Tian, Kuldeep Kulkarni (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06391.txt.
Introduction
Removing a moving object from video also means removing its shadow and reflections. Saksham Singh Kushwaha and 3 co-authors present Object-WIPER in a 2026 paper, a framework that needs no training. It fills the erased region with content that stays consistent in meaning and stable across frames.
The method starts from a pretrained text-to-video diffusion model. Cross-attention between visual and text tokens locates the object and its associated effects, producing a mask that merges with the user's mask. The video is inverted to structured noise, masked tokens are re-randomized, and denoising copies saved background values to preserve the scene. A new metric rewards temporal and contextual coherence. On standard and new benchmarks, the method beat trained and training-free rivals.
Catalog record
| CREATOR | Saksham Singh Kushwaha, Sayan Nag, Yapeng Tian, Kuldeep Kulkarni |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-06391
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Object-WIPER: Training-Free Object and Associated Effect Removal in Videos Saksham Singh Kushwaha, Sayan Nag, Yapeng Tian, Kuldeep Kulkarni Abstract In this paper, we introduce Object-WIPER , a training-free framework for removing dynamic objects and their associated visual effects from videos, and inpainting them with semantically consistent and temporally coherent content. Our approach leverages a pre-trained text-to-video diffusion transformer (DiT). Given an input video, a user-provided object mask, and query tokens describing the target object and its effects, we localize relevant visual tokens via visual-text cross-attention and visual self-attention. This produces an intermediate effect mask that we fuse with the user mask to obtain a final foreground token mask to replace. We first invert the video through the DiT to obtain structured noise, then reinitialize the masked tokens with Gaussian noise while preserving background tokens. During denoising, we copy values for the background tokens saved during inversion to maintain scene fidelity. To address the lack of suitable evaluation, we introduce a new object removal metric that rewards temporal consistency among foreground tokens across consecutive frames, coherence between foreground and background tokens within each frame, and dissimilarity between the input and output foreground tokens. Experiments on DAVIS and a newly curated real-world associated effect benchmark ( WIPER-Bench ) show that Object-WIPER surpasses both training-based and training-free baselines in terms of the metric, achieving clean removal and temporally stable reconstruction without any retraining. Our new benchmark, source code, and pre-trained models will be publicly available. Figure 1 : Object-WIPER removes undesired objects and their associated effects without any training, thereby avoiding substantial training time and computational resources. † † * Work done during an internship at Adobe. 1 Introduction Object removal from videos is an extremely important problem that has widespread applications like film and video production that require boom mic or crew removal, surveillance and privacy protection and creative content generation. The history of this problem has its genesis in the classical non-parametric video inpainting techniques 1 ; 29 ; 15 ; 9 that use combinations of energy minimization, graph cuts and flow estimation techniques. The video inpainting approaches evolved with the explosion of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in the past decade yielding superior results 4 ; 14 ; 20 ; 43 ; 46 . The video inpainting approaches inherently focused only on filling the regions belonging to the object that is being removed through flow estimation techniques while completely ignoring the associated effects of the object like shadows and reflections. The very nature of these approaches lead to undue artifacts owing to the retention of the associated effects in the output videos. This drawback of retention of the associated effects is shared by the recent video inpainting methods 2 ; 47 that rely on modern architectures like diffusion models. Moreover, Miao et al. 28 proposed an approach to tackle the removal of the objects as well as their associated effects. However, their method requires the collection of a large amount of synthetic data using 3D engines followed by an expensive training with this data. The closest to our work is Omnimatte-zero 36 that attempts to remove the associated effects by identifying the corresponding tokens from the attention maps. Omnimatte-zero suffers from two major drawbacks. Firstly, Omnimatte-Zero constructs associated-effect masks by expanding from the user-provided object mask: it identifies regions that are strongly attended by tokens inside the object mask and adds them to the mask. However, this augmented mask is suboptimal, as it can miss the regions of associated effects with weaker activations and relies solely on the object mask as a seed. Secondly, it utilizes a heavy weight external model, TAP-Net 6 to track the foreground points and find associated background points in all the frames and further leverages these associations to compute the attention of the foreground points. This makes the attention computation for the foreground locations vulnerable to the inaccurate point tracking, especially in the cases of fast motion like a car speeding away, textureless areas or translucent objects. To overcome these drawbacks, we propose a two step approach to obtain the effects for the associated mask by first, leveraging the text-to-visual cross attention scores and identifying the visual tokens that are highly attentive to the query text tokens depicting the object to be removed and the associated effects. Given this set of seed visual tokens for the associated effects, we utilize the visual self attention scores to further refine this set and obtain the final mask that depicts the object and the associated effect. We relinquish the usage of any external model that may have introduced the erroneous computation of the attention for the foreground. Instead, we reinitialize the foreground region with Gaussian noise and, during the early denoising steps, when the global structure is formed, we bias the attention in the foreground region towards the background tokens using attention scaling. In the later steps, which mainly refine details, we just let the denoising process proceed normally, yielding an appropriate filling in the mask region. We show that the holistic nature of traditional metrics like peak-signal-to-noise-ratio (PSNR) or video quality scores used to evaluate the object removal in videos have several limitations in that it is easy to score high even when the object is not at all removed or only partially removed. In order to address this issue, we propose a novel metric, Tok en Sim ilarity ( TokSim ) that is designed for the problem of object removal from videos. Specifically, our metric rewards the similarity between foreground tokens in consecutive frames, similarity between foreground and background tokens in the same frame and dissimilarity between the foreground tokens in the input video and the output video. To summarize, the contributions of our work are the following. • We propose a training-free approach, Object-WIPER that removes objects and their associated effects by localizing the associated region by utilizing cross-attention and self-attention in MMDiT blocks (see Fig. 1 ). • We introduce a timestep-adaptive masking strategy with foreground reinitialisation and attention scaling, which prevents object leakage during denoising and enables effective object removal. • Further, given the paucity of evaluation metrics for object removal, we devise a new object removal metric ( TokSim ) that rewards high-quality object removal and heavily penalizes partial-to-no object removal. • We introduce a new real-world benchmark with associated effects and evaluate on it and DAVIS, showing Object-WIPER outperforms all baselines (including training-based) on TokSim while remaining competitive on traditional metrics. 2 Related Works Video Inpainting: Image and Video inpainting gained prominence due to the success of patchmatch, graphcuts, and energy minimization-based algorithms 1 ; 29 ; 9 ; 15 that operated at the pixel level. With the evolution of deep learning, a number of techniques 20 ; 43 ; 46 ; 4 ; 14 ; 47 were developed that cast the video inpainting as a pixel-to-pixel transformation. However, unlike our proposed approach, all the above video inpainting techniques are entirely focused on removing the objects and not the associated effects. Training-free method for Image and Video Editing: With the rise of diffusion models 35 ; 30 , training-free editing models 8 ; 3 ; 32 ; 5 ; 19 has gained prominence due to the inherent