NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing
Tianlin Pan, Jiayi Dai, Chenpu Yuan, Zhengyao Lv, Binxin Yang, Hubery Yin, Chen Li, Jing Lyu, Caifeng Shan, Chenyang Si · 2026
Read NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing by Tianlin Pan, Jiayi Dai, Chenpu Yuan, Zhengyao Lv, Binxin Yang, Hubery Yin, Chen Li, Jing Lyu, Caifeng Shan, Chenyang Si (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02802.txt.
Introduction
Tianlin Pan, Jiayi Dai, Chenpu Yuan, and colleagues edit videos without needing matched before-and-after training pairs. Their 2026 framework, NOVA, splits the job into sparse control and dense synthesis. Paired editing data are scarce, above all for local edits, and older pair-free tricks often broke backgrounds or timing.
In NOVA, a user edits a few keyframes, and those frames steer the meaning of the edit across the clip. A second branch keeps pulling motion and texture from the original video, preserving fidelity and smooth flow. Training uses artificially degraded videos, teaching the model to rebuild motion and timing without real pairs. In extensive tests, NOVA beat prior methods on edit accuracy, motion preservation, and coherence over time.
Catalog record
| CREATOR | Tianlin Pan, Jiayi Dai, Chenpu Yuan, Zhengyao Lv, Binxin Yang, Hubery Yin, Chen Li, Jing Lyu, Caifeng Shan, Chenyang Si |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31651 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.02802
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing Tianlin Pan, Jiayi Dai, Chenpu Yuan, Zhengyao Lv, Binxin Yang, Hubery Yin, Chen Li, Jing Lyu, Caifeng Shan, Chenyang Si Abstract Recent video editing models have achieved impressive results, but most still require large-scale paired datasets. Collecting such naturally aligned pairs at scale remains highly challenging and constitutes a critical bottleneck, especially for local video editing data. Existing workarounds transfer image editing to video through global motion control for pair-free video editing, but such designs struggle with background and temporal consistency. In this paper, we propose NOVA: Sparse Control & Dense Synthesis, a new framework for unpaired video editing. Specifically, the sparse branch provides semantic guidance through user-edited keyframes distributed across the video, and the dense branch continuously incorporates motion and texture information from the original video to maintain high fidelity and coherence. Moreover, we introduce a degradation-simulation training strategy that enables the model to learn motion reconstruction and temporal consistency by training on artificially degraded videos, thus eliminating the need for paired data. Our extensive experiments demonstrate that NOVA outperforms existing approaches in edit fidelity, motion preservation, and temporal coherence. Code and models will be released at https://github.com/WeChatCV/NovaEdit . Figure 1 : We propose Sparse Control, Dense Synthesis , a multi-task video editing framework using sparse user-provided keyframe edits and dense information derived from the original video. † † footnotetext: Work done at WeChat, Tencent Inc. † Corresponding Author. 1 Introduction Figure 2 : Chanllenges in Local Editing . Exisiting general video editing methods ( e.g . VACE Jiang et al. 2025 ) and datasets ( e.g . Senorita-2M Zi et al. 2025b ) perform well on global editing tasks but struggle with local editing, often producing artifacts and inconsistent edits in the targeted areas. Recent advances in diffusion models Ho et al. 2020 ; Song et al. 2020 ; Peebles and Xie 2022 ; Rombach et al. 2022 ; Ronneberger et al. 2015 ; Podell et al. 2023 ; Esser et al. 2024 ; Labs 2024 ; OpenAI 2024 ; DeepMind 2025 ; Kong et al. 2025 ; Wan et al. 2025 have fundamentally transformed the landscape of digital content creation. Building upon this progress, video editing Qin et al. 2023 ; Ku et al. 2024 ; Feng et al. 2023 ; Jiang et al. 2025 is crucial for controllable and practical video content creation, yet its progress is largely constrained by data limitations. While large-scale text-video datasets for generative modeling can be collected relatively easily from open sources, constructing datasets for video editing is far more challenging, as it requires precisely aligned pairs of original and edited videos that rarely occur naturally. To address this limitation, some methods synthesize editing pairs to construct training datasets Zi et al. 2025b ; Wu et al. 2025b ; Hu et al. 2024 , and then train diffusion-based video editing models on large-scale synthetic data. Other approaches Ku et al. 2024 ; Ouyang et al. 2024 ; Wang et al. 2025 ; Gao et al. 2025 ; Zi et al. 2025b attempt to leverage image editing models to modify the first frame and combine the edited keyframe with motion information extracted from the source video to generate edited video sequences. As illustrated in Figure 2 , video editing tasks can be broadly classified into two paradigms: global and local editing. We observe that existing frameworks perform well on global tasks, but face significant challenges with local video editing, which requires targeted modifications within specific spatial regions. This phenomenon can be attributed to two major bottlenecks. 1) The scarcity of high-quality paired data : synthesizing realistic “before” and “after” video pairs for local edits is notoriously difficult, as such edits often involve complex geometric and appearance changes. Consequently, large-scale synthetic datasets usually contain visual artifacts and inconsistencies, limiting the generalization of models trained on them. 2) The fragility of first-frame-based editing : approaches that rely solely on an edited first frame are prone to structural drift and motion misalignment, especially under significant camera or object movement. The discrepancy between the edited keyframe and the original motion dynamics often leads to degraded visual fidelity and poor temporal coherence in the resulting video. Although some techniques attempt to fine-tune a dedicated motion module for each video to improve temporal consistency, this process is labor-intensive and does not scale efficiently. In this paper, we propose a novel framework, NOVA, that leverages sparse control and dense synthesis to learn video editing models without paired supervision. Specifically, NOVA adopts a dual-branch architecture designed to balance semantic control and fidelity. We first employ image editing models to modify keyframes within a video, where the edited keyframes serve as temporal anchors that provide stronger and more semantically aligned constraints than using a single edited frame. The Sparse Branch serves as a controllable pathway that encodes multiple edited keyframes to guide spatial and semantic transformations, imposing localized constraints on where and how edits are applied. Meanwhile, the Dense Branch encodes the unedited source video to capture dense motion and texture information. Through multi-level cross-attention, the generator queries the Dense Branch to inject fine-grained motion cues and background details, mitigating hallucination and texture drift in non-keyframe regions. To ensure temporal coherence, NOVA integrates two complementary strategies: At inference time, all keyframes are edited sequentially with explicit reference to the first edited frame, effectively reducing flicker and temporal discontinuities caused by independent per-frame edits. During training , a degradation-simulation scheme is introduced to mimic real-world motion inconsistencies through interpolation and blurring of keyframes. This strategy enables the model to learn temporal restoration and texture propagation from unpaired data, thereby enhancing overall robustness and frame-to-frame consistency. Our comprehensive experiments demonstrate that NOVA quantitatively surpasses recent approaches across all editing-specific and general quality metrics without requiring per-video fine-tuning. Furthermore, rigorous ablation studies validate the necessity of our core components, confirming Dense Synthesis Branch is crucial for background preservation via guided generative reconstruction and that our consistency-aware inference pipeline is essential for maintaining temporal coherence. Our contributions are summarized as follows: • We introduce and formalize the ‘Sparse Control, Dense Synthesis’ paradigm for video editing, which first decouples dense and sparse signals for video editing, offering a new conceptual framework for the field. • We design a complete unpaired learning framework, including a novel degradation-simulation training strategy and a consistency-aware inference pipeline, which together enable learning temporal coherence and motion reconstruction without any paired data. • Extensive experiments demonstrate that our method outperforms state-of-the-art video editing approaches across multiple metrics. Figure 3 : The Limitations of Existing Schemes Previous methods are often limited by either requiring costly per-video finetuning Ouyang et al. 2024 ; Gao et al. 2025 or pre-training on large-scale paired video data Zi et al. 2025b ; Jiang et al. 2025 , which is difficult to acquire. Our method decouples control and synthesis signals, enabling a self-supervised framework that learns from unpaired data while maintaining high fidelity to the source video. Figure 4 : Inconsistent backgrounds in naive multi-keyframe approach We designate frames 0, 20, 40, 60, and 80 as edited keyframes (anchors). In non-keyframes reconstructed background exhibits inconsistent textures (e.g., on the building wall) and implausible motion (e.g., in the trees), as