CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration
Tianyidan Xie 1 , Mingjie Wang 2 , Qiang Tang 3 , Feixuan Liu 4 , Rui Ma 5 , Lanjun Wang 6 , Zili Yi 1,* · 2026
Read CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration by Tianyidan Xie 1 , Mingjie Wang 2 , Qiang Tang 3 , Feixuan Liu 4 , Rui Ma 5 , Lanjun Wang 6 , Zili Yi 1,* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23579.txt.
Introduction
Xie and colleagues present CineAGI, a staged framework for auto movie making that keeps characters true across scenes. One module drafts film plans with character profiles, a character-first pipeline tracks identities across shots, and a sync layer lines up dialogue, looks, and music frame by frame. Overall consistency rises 40 percent over baselines in the team's tests.
Catalog record
| CREATOR | Tianyidan Xie 1 , Mingjie Wang 2 , Qiang Tang 3 , Feixuan Liu 4 , Rui Ma 5 , Lanjun Wang 6 , Zili Yi 1,* |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21788 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23579
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration Tianyidan Xie 1 , Mingjie Wang 2 , Qiang Tang 3 , Feixuan Liu 4 , Rui Ma 5 , Lanjun Wang 6 , Zili Yi 1,* Abstract Automated movie creation requires coordinating multiple characters, modalities, and narrative elements across extended sequences—a challenge that existing end-to-end approaches struggle to address effectively. We present CineAGI , a hierarchical movie generation framework that decomposes this complex task through specialized multi-agent orchestration. Our framework employs three key innovations: (1) a multi-agent narrative synthesis module where specialized LLM agents collaboratively generate comprehensive cinematic blueprints with character profiles, scene descriptions, and cross-modal specifications; (2) a decoupled character-centric pipeline that maintains identity consistency through instance-level tracking and integration while enabling flexible multi-character composition; and (3) a hierarchical audio-visual synchronization mechanism ensuring frame-level alignment of dialogue, expressions, and music. Extensive experiments demonstrate that CineAGI achieves 40% improvement in overall consistency, 4.4% gain in subject consistency, 5.4% enhancement in aesthetic quality, and 28.7% higher character consistency compared to baselines. Our work establishes a principled foundation for automated multi-scene video generation that preserves narrative coherence and character authenticity. I Introduction The automation of movie creation represents a fundamental challenge in generative AI, requiring the coordination of narrative structure, visual synthesis, and audio generation across extended temporal sequences. While recent advances in Large Language Models (LLMs) [ 18 , 15 ] and diffusion-based video synthesis [ 22 , 2 , 19 ] have shown impressive capabilities in individual content generation tasks, creating coherent long-form movies poses three critical challenges illustrated in Figure 1 . Fig. 1: Key challenges in automated movie generation. Existing text-to-video models face three fundamental limitations: (Top) Character identity inconsistency across scenes and contexts; (Middle) Scene inconsistency in visual styles and transitions; (Bottom) Cross-modal misalignment between visual elements, dialogue, and audio. Our hierarchical framework addresses these challenges through specialized agent coordination and decoupled processing pipelines. Character Identity Preservation. Quadratic computational complexity necessitates limited processing windows, causing loss of long-range identity information. Current approaches lack effective mechanisms to consistently preserve multi-scale identity features, from global attributes (facial structure, physique) to fine-grained details (expressions, lighting adaptation), across frames and scenes. Scene-Level Temporal Coherence. Memory constraints prevent global scene consistency enforcement, and existing architectures lack explicit mechanisms for decomposing scene elements at different temporal frequencies, making it difficult to maintain compositional coherence over extended durations. Cross-Modal Synchronization. Coordinating lip movements, emotional expressions, and musical timing at frame-level precision requires explicit synchronization mechanisms that existing end-to-end methods lack. To address these challenges, we present CineAGI, a hierarchical movie creation framework that orchestrates multi-modal generation through principled decomposition and specialized agent coordination. Our framework introduces three key technical innovations: Multi-Agent Narrative Synthesis. Specialized LLM agents (Character Designer, Script Writer, Storyteller, Composer, Quality Inspector) collaboratively generate cinematic blueprints. Unlike prior approaches that treat narrative elements independently, our coordinated agents maintain cross-modal consistency through structured information flow and validation mechanisms. Decoupled Character-Centric Pipeline. A three-stage approach of text-guided segmentation (Grounded-SAM2), identity-preserving face integration (SimSwap), and talking face synthesis (Wav2Lip) enables independent multi-character processing while maintaining global consistency across diverse scene contexts. Hierarchical Audio-Visual Synchronization. Explicit coordination at multiple levels, including frame-level lip alignment, emotion-aware music generation, and integrated scene assembly, addresses cross-modal alignment limitations in existing methods. This structured approach ensures synchronized audiovisual integration throughout the production. Our main contributions are: • A novel multi-agent movie generation framework with specialized LLM agents for coordinated narrative planning, enabling fine-grained control over character consistency and temporal coherence that surpasses existing approaches. • A systematic character-centric pipeline that maintains multi-level identity preservation through decoupled processing, achieving robust character consistency across diverse contexts while preserving creative flexibility. • Comprehensive evaluation demonstrating 40% improvement in overall consistency, 4.4% gain in subject consistency, 5.4% enhancement in aesthetic quality, and 28.7% higher character consistency compared to state-of-the-art baselines. II Related Work II-A Text-to-Video Generation Text-to-video synthesis has evolved rapidly through diffusion-based architectures. Recent systems such as Sora [ 3 ] , Lumiere [ 1 ] , and Stable Video Diffusion [ 2 ] have significantly advanced generation duration and visual fidelity. CogVideoX [ 24 ] introduced expert transformers with 3D VAE for spatial-temporal compression, while VideoCrafter [ 4 ] focused on temporal consistency through dedicated frame-level coherence modules. However, these methods face inherent limitations due to localized processing windows and lack of explicit narrative structure modeling. Recent advances have addressed specific challenges in video generation. For character consistency, ID-Animator [ 8 ] and ConsisID [ 25 ] introduced identity preservation techniques, while StoryDiffusion [ 26 ] and Animate Anyone [ 11 ] advanced long-range generation and character animation. For extended sequences, StreamingT2V [ 9 ] and FIFO-Diffusion [ 13 ] enabled multi-minute to theoretically infinite video generation. LLM integration has emerged as a powerful approach, with VideoDirectorGPT [ 17 ] pioneering LLM-guided planning, VideoPoet [ 14 ] demonstrating unified multimodal generation, and MovieGen [ 20 ] advancing joint video-audio synthesis. Despite these advances, maintaining multi-character consistency, cross-scene narrative coherence, and coordinated multi-modal production in long-form content remains challenging. Our work differs fundamentally by introducing hierarchical decomposition through multi-agent orchestration that decouples character generation from scene synthesis, enabling robust character consistency while preserving narrative flexibility across extended sequences. II-B Multi-Agent Systems Recent LLM advances have enabled sophisticated multi-agent systems for complex task automation. MetaGPT [ 10 ] and AgentVerse [ 7 ] demonstrated foundational principles for agent-based collaboration through specialized roles and structured coordination protocols. In movie production, agent-based approaches have explored automated content creation. AutoDirector [ 18 ] introduced multi-sensory composition through scheduled agents, while Anim-Director [ 15 ] focused on animation generation. AesopAgent [ 23 ] pioneered agent-driven evolutionary workflows combining RAG-based optimization with utility-layer execution, and StoryAgent [ 12 ] introduced specialized agents mirroring professional production roles. However, these systems struggle with temporal consistency and multi-modal synchronization over extended sequences due to insufficient mechanisms for managing complex inter-dependencies. Our hierarchical architecture advances beyond existing approaches through explicit coordination protocols inspired by professional pipelines, with specialized agents (Character Designer, Script Writer, Storyteller, Composer, Quality Inspector) ensuring both creative consistency and temporal coherence through structured information flow and validation mechanisms. Fig. 2: CineAGI framework overview. Given a story concept, our framework proceeds through three modules: (1) Narrative Synthesis : LLM agents collaboratively develop character profiles and production scripts ensuring narrative coherence; (2) Character Generation : creates visual and audio assets with consistent character appearances and voice profiles; (3) Cinematographic Synthesis : orchestrates video generation, character integration, and audio synchronization for complete movie production. III Method III-A Overview Our CineAGI framework introduces hierarchical multi-agent orchestration for automated movie creation, as illustrated in Figure 2 . Unlike end-to-end approaches that process entire sequences holistically, our hierarchical decomposition enables targeted optimization of individual components while maintaining global coherence through structured agent coordination. The framework leverages complementary strengths of different generative models: LLMs for high-level creative planning and narrative structure, specialized models for consistent character generation, and targeted synthesis models for precise audiovisual integration. The Narrative Synthesis Module serves as the creative foundation where LLM agents collaboratively develop rich, coherent story specifications. The Character Generation Module bridges abstract descriptions and concrete assets through specialized portrait and voice synthesis. The Cinematographic Synthesis Module coordinates these