DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing

Weitao Wang, Zichen Wang, Hongdeng Shen, Yulei Lu, Xirui Fan, Suhui Wu, Jun Zhang, Haoqian Wang, Hao Zhang · 2025

Read DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing by Weitao Wang, Zichen Wang, Hongdeng Shen, Yulei Lu, Xirui Fan, Suhui Wu, Jun Zhang, Haoqian Wang, Hao Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-14465.txt.

Introduction

Weitao Wang and co-authors present DreamSwapV, a method for swapping a subject in a video with a new one chosen by the user. A mask marks the subject to replace, and a reference image shows what should go in its place. Past methods worked only for narrow cases or leaned on vague text prompts.

DreamSwapV takes in several kinds of guidance and fuses them in a dedicated module. An adaptive mask plan handles subjects of unlike size and shape, so the new subject fits its scene better. The team built data and trained in two phases. Tests on VBench and their own DreamSwapV-Benchmark put the method ahead of prior work.

Catalog record

CREATORWeitao Wang, Zichen Wang, Hongdeng Shen, Yulei Lu, Xirui Fan, Suhui Wu, Jun Zhang, Haoqian Wang, Hao Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-14465

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing Weitao Wang, Zichen Wang, Hongdeng Shen, Yulei Lu, Xirui Fan, Suhui Wu, Jun Zhang, Haoqian Wang, Hao Zhang Abstract With the rapid progress of video generation, demand for customized video editing is surging, where subject swapping constitutes a key component yet remains under-explored. Prevailing swapping approaches either specialize in narrow domains—such as human-body animation or hand-object interaction—or rely on some indirect editing paradigm or ambiguous text prompts that compromise final fidelity. In this paper, we propose DreamSwapV, a mask-guided, subject-agnostic, end-to-end framework that swaps any subject in any video for customization with a user-specified mask and reference image. To inject fine-grained guidance, we introduce multiple conditions and a dedicated condition fusion module that integrates them efficiently. In addition, an adaptive mask strategy is designed to accommodate subjects of varying scales and attributes, further improving interactions between the swapped subject and its surrounding context. Through our elaborate two-phase dataset construction and training scheme, our DreamSwapV outperforms existing methods, as validated by comprehensive experiments on VBench indicators and our first introduced DreamSwapV-Benchmark. 1 Introduction The recent development of video generation technologies is enabling a broader audience to engage in customized content creation, also stimulating demand for editing real or generated videos. Users increasingly desire to introduce specific subjects into videos following the original motion trajectory, necessitating the video subject swapping task. Given a source subject in the video and a target subject to be inserted, the ideal outcome is that the target is seamlessly integrated into the original video, preserving its own appearance details, following the source motion, and interacting naturally with the surrounding context. Figure 1: Our DreamSwapV can swap any subject in any video with a user-specified mask and reference image. However, existing video editing methods exhibit critical limitations: On one hand, domain-specific approaches are confined to particular subjects: animation techniques like MagicAnimate ( Xu et al. 2024c ) and Animate Anyone 2 ( Hu et al. 2025a ) only support swapping human characters; Human-Object Interaction (HOI) methods like AnchorCrafter ( Xu et al. 2024b ) and DreamActor-H1 ( Wang et al. 2025 ) are limited to specific scenarios (e.g. livestreaming, E-commerce) for swapping handheld objects; thereby restricting their broader applicability. On the other hand, general-purpose video editing methods also face several challenges: (1) Tuning-free techniques ( Ku et al. 2024 ; Yang et al. 2025 ) manipulate attention features to swap subjects, but struggle to restore fine details and largely ignore interactions with subjects; (2) Tuning-based methods inject the target subject either via text prompts like VideoPainter ( Bian et al. 2025 ) or learned LoRA from several subject concepts like VideoSwap ( Gu et al. 2024 ) ; the former lacks detail fidelity, whereas the latter is computationally demanding and indirect in reference injection; (3) Recently, unified video customization frameworks ( Jiang et al. 2025 ; Hu et al. 2025b ) are emerging, aiming for versatility across multiple customization tasks, including video-driven subject swapping. Yet their all-in-one design sacrifices identity consistency and interaction realism in the subject swapping scenario. To address these limitations, we present DreamSwapV, a mask-guided, subject-agnostic framework for video subject swapping. Given a source video, a user-specified mask localizing the source subject, and a reference image of the target subject, we can perform end-to-end swapping of any subject in any video. Unlike conventional editing paradigms that focus on generating or editing an external object into the scene, we recast swapping as an inpainting task: we encourage the model to recover the appearance of the target subject as if it inherently belonged within the missing masked region of the source video, yielding a direct and intuitive training and inference procedure. Specifically, we first construct a task-oriented dataset based on HumanVID ( Wang et al. 2024b ) and introduce auxiliary conditions for better video control. A dedicated condition fusion module then effectively integrates these conditions for model’s better understanding of both video content and task objectives. Furthermore, an adaptive mask strategy accommodates subjects of varying scales and attributes, mitigating shape leakage while enhancing subject–context interaction. Lastly, we establish DreamSwapV-Benchmark—the first benchmark tailored to video subject swapping—and evaluate our method along with 5 indicators inherited from VBench ( Huang et al. 2024a ) , confirming the robustness and superior performance of DreamSwapV. Our contributions are summarized as follows: • We propose DreamSwapV, the first end-to-end framework dedicated to the video subject swapping, specifically focusing on generic subject-context interaction. • Our novel condition fusion module and adaptive mask strategy enable handling diverse subjects, enhancing contextual understanding, and improving video realism. • Through a two-stage dataset construction and training scheme, we outperform existing methods on VBench indicators and first introduced DreamSwapV-Benchmark. 2 Related Work 2.1 Video Generation and Editing Video generation is a pivotal component within the Artificial Intelligence Generated Content (AIGC) domain, realizing human creativity in the video medium. The field has undergone rapid evolution, from initial GAN-based methods ( Vondrick, Pirsiavash, and Torralba 2016 ; Tulyakov et al. 2018 ) , through subsequent diffusion models ( Ho et al. 2022 ; Singer et al. 2022 ) , to the current state-of-the-art Diffusion Transformers (DiT) ( Peebles and Xie 2023 ) . At present, leading commercial models ( Keling 2025 ; Hailuo 2025 ; Vidu 2025 ) and open-source models ( Yang et al. 2024 ; Kong et al. 2024 ) can leverage billion-scale architectures to generate high-fidelity videos at 720p-1080p resolutions. Concurrently, editing techniques for real or generated videos are also advancing vigorously. Tune-A-Video ( Wu et al. 2023 ) pioneers video editing with latent diffusion models via one-shot tuning, followed by ( Qi et al. 2023 ; Cong et al. 2023 ; Zhao et al. 2025 ) , enhancing capabilities in appearance detail, temporal consistency and motion control. With the advent of video foundation models, an increasing number of editing methods are based on video diffusion models ( Gu et al. 2024 ; Yang et al. 2025 ) or DiT ( Bian et al. 2025 ; Tu et al. 2025 ) , providing more precise and diverse video editing. Our DreamSwapV builds upon the cutting-edge Wan2.1 DiT model ( Wan et al. 2025 ) and adapts it specifically to the video subject swapping task, with our condition fusion module injecting additional conditioning information and a carefully designed training scheme. 2.2 Video Customization The freedom and creativity of personalized content creation facilitates the development of video customization, which can be broadly classified into two paradigms. Subject concepts-driven methods learn subject-specific adapters ( Houlsby et al. 2019 ) or LoRA modules ( Hu et al. 2022 ) from multiple reference images, embedding the subject concepts into the text space. DreamBooth ( Ruiz et al. 2023 ) and Textual Inversion ( Gal et al. 2022 ) pioneer this approach in the image domain and later extend to video customization by methods like ( Chefer et al. 2024 ; Wu et al. 2025 ; Jiang et al. 2024 ) . While effective for identity preservation, the requirement of per-subject finetuning hampers real-time use, motivating the emergence of end-to-end methods. End-to-end customization typically employs an additional conditioning network to inject user-specified information, similar to the idea of ControlNet ( Zhang, Rao, and Agrawala

Read the complete text