DCDM : Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation
Haoyu Zhao 1,*, Yuang Zhang 2, Junqi Cheng 2, Jiaxi Gu 2, Zenghui Lu 2, Peng Shu 2, Zuxuan Wu 1, † \dagger, Yu-Gang Jiang 1, † \dagger · 2026
Read DCDM : Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation by Haoyu Zhao 1,*, Yuang Zhang 2, Junqi Cheng 2, Jiaxi Gu 2, Zenghui Lu 2, Peng Shu 2, Zuxuan Wu 1, † \dagger, Yu-Gang Jiang 1, † \dagger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13637.txt.
Introduction
Video generation often looks good frame by frame yet fails on consistency: worlds shift within a clip, cameras jump between clips, and elements change across shots. DCDM tackles all three with a system-level divide-and-conquer design sharing one video generation backbone. Within clips, a language model parses prompts into structured semantic representations that a diffusion transformer turns into coherent content. Between clips, a temporal camera representation in noise space enables precise stable camera motion control, aided by text-to-image initialization.
Across shots, a holistic scene generation approach with windowed cross-attention and sparse inter-shot self-attention preserves long-range narrative coherence efficiently. Validation on the CVM Competition test set at AAAI 2026 shows the strategies effectively address the three consistency challenges. Haoyu Zhao leads the work with seven co-authors.
Catalog record
| CREATOR | Haoyu Zhao 1,*, Yuang Zhang 2, Junqi Cheng 2, Jiaxi Gu 2, Zenghui Lu 2, Peng Shu 2, Zuxuan Wu 1, † \dagger, Yu-Gang Jiang 1, † \dagger |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13637
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DCDM : Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation Haoyu Zhao 1,*, Yuang Zhang 2, Junqi Cheng 2, Jiaxi Gu 2, Zenghui Lu 2, Peng Shu 2, Zuxuan Wu 1, † \dagger, Yu-Gang Jiang 1, † \dagger Abstract 1] Fudan University 2] Tencent \correspondence and \checkdata [Website] AAAI 2026 Consistency in Video Generative Models (CVM) Competition \checkdata [Team]Cinematrix \checkdata [Award]CVM @ AAAI’26 Top Team † † footnotetext: ∗ This work was completed during an internship at Tencent under the Qingyun Project. † Corresponding authors. 1 Introduction Recent advances in video generative models have significantly improved visual fidelity and short-term temporal coherence. However, achieving consistency across diverse generation scenarios remains a fundamental challenge. In practice, video consistency manifests along multiple dimensions, including intra-clip world knowledge consistency, inter-clip camera consistency, and inter-shot element consistency. These dimensions pose inherently different modeling requirements. Intra-clip consistency demands grounded semantic reasoning to ensure that objects, attributes, and physical interactions within a single clip remain plausible and non-contradictory. Inter-clip camera consistency requires explicit modeling of camera motion and view transitions, which are temporally structured and fundamentally geometric in nature. Inter-shot element consistency, in contrast, focuses on long-range narrative coherence, requiring persistent memory of characters, scenes, and styles across multiple shots and extended time horizons. Due to these fundamentally distinct sources of inconsistency, enforcing all three forms of consistency within a single monolithic video diffusion model is highly challenging. A unified model must simultaneously reason over semantics, geometry, and long-range temporal dependencies, often leading to representational entanglement and suboptimal trade-offs between controllability, coherence, and computational efficiency. As a result, existing approaches tend to excel in one dimension while compromising others. In this paper, we propose a system-level framework termed the Divide-and-Conquer Diffusion Model (DCDM). Rather than enforcing all consistency constraints within a single monolithic diffusion model, DCDM decomposes the problem into three dedicated components, each explicitly designed to address one key dimension of consistency, while sharing a common video diffusion backbone. Specifically, for intra-clip world knowledge consistency, our goal is to ensure that the generated video remains semantically and physically coherent within a single clip. To this end, we first employ a large language model to parse the input prompt, extracting the implicit subjects, attributes, and scene descriptions embedded in natural language. These structured semantic representations are then provided to a diffusion transformer (DiT) [ 6 ] , which translates the parsed world knowledge into visually coherent video content. This design allows DCDM to ground video generation in explicit semantic reasoning, reducing contradictions and implausible interactions within a clip. To achieve inter-clip camera consistency, DCDM explicitly models camera motion as a first-class control signal. We again leverage a language model to analyze the prompt and classify the described camera motion into predefined motion categories. Based on this analysis, we propose a temporally structured noise representation that is injected into the diffusion process to guide camera motion across frames. Furthermore, to enhance controllability and stability, we adopt a text-to-image initialization strategy, where a reference image is first generated from the textual description. The diffusion model then conditions on both the reference image and the camera representation, enabling camera-controlled video generation with improved temporal coherence and precise motion execution. For inter-shot element consistency, we focus on maintaining the identity and attributes of characters, scenes, and styles across long-form narratives. We employ a holistic scene generation model that produces the entire video sequence jointly, ensuring global coherence from the first shot to the last. To enable fine-grained directorial control, our architecture incorporates a window cross-attention mechanism to localize textual prompts to specific shots, while a sparse inter-shot self-attention pattern, which is dense within shots but sparse across shots, balances long-range consistency with the computational efficiency required for minute-scale generation. Beyond improving narrative coherence, this design exhibits strong emergent properties, including persistent character and scene memory, as well as an implicit understanding of cinematic structures and shot composition. In conclusion, DCDM demonstrates that video consistency is best addressed through a divide-and-conquer strategy, where semantic reasoning, camera control, and narrative coherence are modeled in a disentangled yet unified diffusion framework. The main contributions of this work are summarized as follows: • We propose the Divide-and-Conquer Diffusion Model (DCDM), a unified system framework that decomposes video consistency into three complementary dimensions, including intra-clip world knowledge consistency, inter-clip camera consistency, and inter-shot element consistency. • For intra-clip world knowledge consistency, we introduce an LLM-based prompt extension mechanism that enriches implicit semantics to ensure coherent video generation within each clip. • For inter-clip camera consistency, we propose to control the camera movement in noise space with a temporal structure that enables precise and stable camera motion control in diffusion models. • For inter-shot element consistency, we introduce sparse inter-shot self-attention that enables efficient long-range coherence while preserving dense intra-shot modeling. 2 Related Work Recent advances in diffusion models based on U-Net and Diffusion Transformer (DiT) [ 6 ] architectures have significantly improved video generation [ 9 , 2 , 13 , 11 , 8 , 4 , 12 , 14 ] . U-Net–based approaches employ an encoder–decoder structure that offers efficient inference and architectural flexibility, but their limited model capacity constrains performance gains under large-scale scaling. As a result, recent methods, such as CogVideoX [ 11 ] , Seedance [ 3 ] , Seaweed-7B [ 7 ] , and Wan-14B model [ 8 ] , increasingly adopt DiT as the core building block, leveraging stacked Transformer layers to jointly model spatial and temporal dependencies, with scalable depth that better supports large-scale training. These works encompass a broad spectrum of diffusion-based architectures, from U-Net variants to Transformer-centric designs, coupled with advances in temporal modeling, conditional control, sampling efficiency, and hierarchical synthesis. These developments collectively push the frontier toward more controllable and high-fidelity video generation. Despite these advances, modeling long-term dependencies and preserving consistency in complex scenes remains challenging due to the interplay of high-dimensional spatial detail and rich temporal dynamics. 3 Method Figure 1 : A high-level overview of our proposed Divide-and-Conquer Diffusion Model (DCDM) framework. We propose the Divide-and-Conquer Diffusion Model (DCDM) , a system-level framework that addresses video consistency by decomposing it into three complementary dimensions: (i) intra-clip world knowledge consistency, (ii) inter-clip camera consistency, and (iii) inter-shot element consistency. As illustrated in Fig. 1 , all components share a unified video diffusion backbone based on Diffusion Transformers [ 8 ] , while each consistency dimension is handled by a dedicated module with task-specific inductive biases. This design avoids forcing heterogeneous consistency constraints into a single monolithic model and enables scalable, controllable video generation across diverse scenarios. 3.1 Preliminaries We adopt a latent video diffusion formulation. Given a video latent \mathbf{x}_{0}\in\mathbb{R}^{T\times H\times W\times C} , a forward diffusion process gradually corrupts it with Gaussian noise. A DiT-based denoiser \epsilon_{\theta} is trained to predict the injected noise at diffusion step t : \mathcal{L}=\mathbb{E}_{\mathbf{x}_{0},t,\epsilon}\left[\|\epsilon-\epsilon_{\theta}(\mathbf{x}_{t},t\mid\mathbf{c})\|_{2}^{2}\right], (1) where \mathbf{c} denotes conditioning signals. In DCDM, \mathbf{c} is constructed differently depending on the target consistency dimension, while the DiT backbone remains shared. 3.2 Intra-clip World Knowledge Consistency Motivation. Intra-clip consistency requires that objects, attributes, actions, and physical interactions within a single clip remain semantically and physically coherent. However, user prompts are often under-specified, leaving critical world knowledge implicit. Relying on a