Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework
Xiao CaoYansong QuXiangzhen ChangWen XiaoJiakui HuHeyuan Li, Jialun Liu, Zhiyong HuangXuelong Li · 2026
Read Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework by Xiao CaoYansong QuXiangzhen ChangWen XiaoJiakui HuHeyuan Li, Jialun Liu, Zhiyong HuangXuelong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23891.txt.
Introduction
Xiao Cao and colleagues tackle mask-free video object insertion, placing a reference object into a scene so it looks native even when styles clash. Their Smart-Insertion-V runs insertion and image style transfer together in two streams, with a closed loop feedback step keeping the result honest. A dual world view rotary encoding separates mixed signals by space time offsets, and a decoupled guidance module adds vision language reasoning while keeping native text guidance. Tests show plausible placement with the most harmonious blends, plus a curated dataset release.
Catalog record
| CREATOR | Xiao CaoYansong QuXiangzhen ChangWen XiaoJiakui HuHeyuan Li, Jialun Liu, Zhiyong HuangXuelong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40711 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.23891
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework Xiao CaoYansong QuXiangzhen ChangWen XiaoJiakui HuHeyuan Li, Jialun Liu, Zhiyong HuangXuelong Li Abstract Mask-free video object insertion has emerged as a challenging task, requiring harmonious integration of reference objects into source videos. However, existing methods struggle when references exhibit severe stylistic domain gaps with the source scene. To overcome this, we propose Smart-Insertion-V , an end-to-end Dual-Stream framework that concurrently conducts video insertion and image style transfer. Within this framework, the image stream synchronously guides the video generation process, while a Closed-loop Feedback mechanism is further incorporated to ensure robust insertion. Inevitably, integrating these diverse conditioning signals results in feature entanglement and style leakage. To tackle this issue, we design Dual-World-View RoPE to distinguish different signals via spatial-temporal offsets without incurring heavy training overhead. Furthermore, to facilitate spatial grounding and stylistic adaptation, we introduce a Decoupled Guidance Module that leverages a Vision-Language Model for semantic reasoning while preserving original temporal guidance with native text encoder. To bridge data gap for harmonious reference insertion task, we propose a data curation pipeline and will release an open-source dataset . Experiments demonstrate that our method can insert objects into plausible positions while achieving the most harmonious results. Code and dataset will be released upon acceptance. 1 Introduction Recent advances in diffusion-based video generation diffusion1 ; diffusion2 ; diff1 ; diff2 ; diff3 ; diff4 ; diff5 ; diff6 have enabled increasingly powerful video editing capabilities dragvideo ; videoedit1 ; videoedit2 ; videoedit3 ; videoedit4 ; edit1 ; edit2 ; edit3 ; edit4 ; edit5 ; edit6 ; edit7 . Among them, video object insertion-seamlessly integrating a reference object into a source video-remains a challenging problem. Beyond simply placing an object into a scene, successful mask-free insertion requires jointly reasoning about spatial plausibility, adapting object appearance to match the scene style, and maintaining temporal consistency. This challenge is further amplified when the reference image is visually incompatible with the source video. Existing methods can be broadly categorized into three groups: explicit spatial guidance methods, mask-free methods, and cascaded methods. They remain limited in spatial reasoning, reference-scene harmonization, or robustness across multiple stages. Explicit spatial guidance methods getinvideo ; videoanydoor ; anything ; insert1 ; insert2 rely on user-specified priors, such as masks or control points, to localize the insertion region. Such dependence introduces additional annotation burden and limits practical applicability, while models’ insufficient scene understanding still leads to implausible object attributes, including inconsistent scale, perspective, or placement. Mask-free methods omniinsert ; univideo ; teleomni successfully eliminate manual spatial annotations. However, raw references frequently exhibit severe domain gaps with the target video, including discrepancies in lighting and style. By generating videos conditioned on unprocessed reference images without imposing explicit constraints, these methods often suffer from contextual disharmony between the inserted object and the source video. A straightforward remedy is to employ a cascaded pipeline, which first adapts the reference image to the scene via an image editing model before performing video insertion. However, this decoupled approach is inherently prone to error accumulation. Artifacts or inaccurate adaptations introduced during the initial editing phase are propagated and amplified throughout the video generation process, severely degrading both visual quality and temporal coherence. In this work, we propose Smart-Insertion-V, an end-to-end mask-free video insertion framework that seamlessly inserts raw reference objects into target videos under the guidance of coarse text prompts. To address spatial perception challenges, we introduce a Decoupled Guidance Module (DGM). Specifically, a Vision-Language Model (VLM) is incorporated to comprehend the scene’s spatial layout and resolve style discrepancies. Meanwhile, the native T5 text encoder is retained as a motion reasoner to preserve temporal dynamics. Furthermore, we propose an innovative dual-stream architecture consisting of a video stream and an image stream. While the video stream focuses on the core video insertion, the image stream concurrently performs reference image style transfer. Our core insight is that the image harmonization process can synchronously guide the video generation. Specifically, the image stream provides intermediate features that capture the stylistic transition trend from the raw reference toward the target video. This provides the video stream with high-quality, scene-adapted conditioning signals. However, integrating these diverse conditioning signals often leads to feature entanglement with the target latents. This entanglement causes style leakage, which severely degrades the unedited regions of the target video. To resolve this, we propose a novel Dual-World-View RoPE (Dual-RoPE). Existing methods typically process condition embeddings by adding task-indicator dimensions vace ; omniinsert or treating them as additional frames fulldit , both of which incur substantial training overhead. Dual-RoPE bypasses this limitation by applying distinct coordinate offsets. Specifically, target latents are anchored at zero-offset positions, whereas strong and weak conditions are offset spatially and temporally, respectively. This elegantly isolates all features, allowing the model to effectively process each component without heavy computational burdens. To address the scarcity of paired data capturing both raw and harmonized object states, we develop a comprehensive data curation pipeline coupled with a rigorous dual-verification mechanism. Specifically, our pipeline automatically synthesizes the requisite training quadruplets from massive open-source text-to-video datasets. Each quadruplet consists of a source video, a target video, an unprocessed reference image, and a harmonized reference image, accompanied by tailored instruction prompts and detailed video captions. Here, the unprocessed reference image serves as the raw object exhibiting severe domain gaps, while the harmonized reference represents the ideal object seamlessly integrated into the target environment. Leveraging this pipeline, we construct a large-scale, high-quality dataset that has undergone strict evaluation via our automatic dual-verification strategy. This continuously expanding dataset will be open-sourced in phases. In summary, the main contributions of our work are three-fold: • Decoupled Guidance Module (DGM): We introduce DGM to equip the framework with comprehensive spatial perception and style discrepancy understanding. It integrates a Vision-Language Model (VLM) for spatial-style reasoning, while independently retaining the native T5 encoder to preserve temporal dynamics and motion priors. • Dual-Stream Framework: We propose a dual-stream architecture with closed-loop feedback to concurrently perform video insertion and style transfer for adaptive refinement. Additionally, we design Dual-RoPE to effectively disentangle diverse conditioning signals via distinct spatial-temporal offsets, achieving high-fidelity results without heavy training overhead. • Scalable Data Curation and High-Quality Dataset: We develop an automated, highly scalable data curation pipeline capable of leveraging massive text-to-video datasets. Based on it, we construct a high-quality open-source dataset. 2 Related Work 2.1 Video Generation Driven by recent algorithmic breakthroughs, the landscape of video generation has undergone a profound paradigm shift. While proprietary systems like SeedDance seedance and Pika pika initially set the benchmark for visual fidelity and spatiotemporal consistency, the recent surge of high-performance open-source foundations—such as HunyuanVideo hunyuanvideo , CogVideo cogvideox , and the Wan series—has rapidly democratized high-fidelity video synthesis, bridging the performance gap with state-of-the-art commercial alternatives. Figure 2 : Overview of the data curation pipeline. We first identify the optimal object within a video that is most suitable for removal. Segmentation and video object removal techniques are then utilized to obtain the source video and the GT reference images. Next, we apply style transfer to the GT references to synthesize the raw, unharmonized references. Finally, all generated data undergo a strict filtering process driven by an automated dual-agent verification. 2.2 Video