Cut2Next: Generating Next Shot via In-Context Tuning
Jingwen He Hongbo Liu Jiajun Li Ziqi Huang Yu Qiao Wanli Ouyang Ziwei Liu · 2025
Read Cut2Next: Generating Next Shot via In-Context Tuning by Jingwen He Hongbo Liu Jiajun Li Ziqi Huang Yu Qiao Wanli Ouyang Ziwei Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08244.txt.
Introduction
Jingwen He and co-authors teach a video model the grammar of film editing in a 2025 paper. Anyone can generate a single pretty shot now; stringing shots into cinema is harder. Real editing follows patterns, like cutting between two speakers or slipping in a cutaway, and keeps strict continuity of place and motion. Current generators mostly chase surface consistency and miss that grammar.
Their framework, Cut2Next, defines the task as next shot generation and builds a diffusion transformer for it. A layered prompting scheme steers the model: broad prompts set the scene context and the editing relation between shots, while per-shot prompts fix content and camera choices. In-context tuning lets the model learn from the shot it must follow. The aim is generated film that an editor would recognize as editing.
Catalog record
| CREATOR | Jingwen He Hongbo Liu Jiajun Li Ziqi Huang Yu Qiao Wanli Ouyang Ziwei Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cut2Next: Generating Next Shot via In-Context Tuning Jingwen He Hongbo Liu Jiajun Li Ziqi Huang Yu Qiao Wanli Ouyang Ziwei Liu Abstract Effective multi-shot generation demands purposeful, film-like transitions and strict cinematic continuity. Current methods, however, often prioritize basic visual consistency, neglecting crucial editing patterns (e.g., shot/reverse shot, cutaways) that drive narrative flow for compelling storytelling. This yields outputs that may be visually coherent but lack narrative sophistication and true cinematic integrity. To bridge this, we introduce Next Shot Generation (NSG) : synthesizing a subsequent, high-quality shot that critically conforms to professional editing patterns while upholding rigorous cinematic continuity. Our framework, Cut2Next , leverages a Diffusion Transformer (DiT). It employs in-context tuning guided by a novel Hierarchical Multi-Prompting strategy. This strategy uses Relational Prompts to define overall context and inter-shot editing styles. Individual Prompts then specify per-shot content and cinematographic attributes. Together, these guide Cut2Next to generate cinematically appropriate next shots. Architectural innovations, Context-Aware Condition Injection (CACI) and Hierarchical Attention Mask (HAM) , further integrate these diverse signals without introducing new parameters. We construct RawCuts (large-scale) and CuratedCuts (refined) datasets, both with hierarchical prompts, and introduce CutBench for evaluation. Experiments show Cut2Next excels in visual consistency and text fidelity. Crucially, user studies reveal a strong preference for Cut2Next, particularly for its adherence to intended editing patterns and overall cinematic continuity, validating its ability to generate high-quality, narratively expressive, and cinematically coherent subsequent shots. Figure 1: Cut2Next demonstrating versatile Next Shot Generation . The model produces cinematically coherent subsequent shots (bottom) adhering to diverse editing patterns (e.g., Shot/Reverse Shot, Cut-Out, Cutaway) specified alongside the input shots (upper). 1 Introduction Video generation has advanced remarkably. Models like Sora sora and Kling kling synthesize photorealistic single-shot videos, built upon large-scale data and scalable architectures like Diffusion Transformers (DiT) dit . Following these successes, the academic community is increasingly focusing on narrative video generation lct ; mask2dit ; shotadapter ; xiao2025videoauteur . This aims to create videos composed of multiple interconnected shots, akin to films. Several approaches address multi-shot video generation. One prominent strategy is storyboard generation storydiffusion ; vgot ; one_prompt_one_story ; tewel2024training : text-to-image models first create keyframes, which image-to-video models then animate to construct the full video. Another approach is to directly tackle multi-shot video generation by training models on extensive video datasets lct ; mask2dit ; shotadapter . Both these research lines primarily aim for diverse content in long-form videos with rich plotlines. In contrast, a distinct research line prioritizes individual shot quality and basic consistency within sequences. For instance, IC-LoRA iclora leverages Flux flux ’s in-context generation ability to generate high-resolution storyboards with consistent environments and characters. CineVerse cineverse enhanced IC-LoRA with detailed annotations for user-controlled shot scale. SynCamMaster SynCamMaster aims to synthesize 3D-consistent multi-angle, multi-scale shots, which are then edited together to enrich visual expression. While significantly advancing shot quality and basic consistency, these approaches often overlook the explicit modeling and enforcement of complex editing patterns crucial to professional narrative filmmaking. Figure 2: Canonical cut-driven shot sequences (from CuratedCuts), their narrative functions, and the generation difficulties. Cut-In/Cut-Out : Emphasizes details or shifts focus; challenges models with drastic scale changes while maintaining subject consistency. Cutaway : Provides external or subjective context; demands generating novel yet semantically related content. Shot/Reverse Shot : Facilitates dialogue and reveals reactions; requires consistent character appearance and spatial logic across alternating viewpoints. Multi-Angle : Offers varied viewpoints; requires consistent rendering across significant visual transformations. Our work addresses this crucial gap. We align with the pursuit of high-fidelity individual shots but place an emphasis on the fundamental cinematic concept of the "cut." In professionally edited movies, cuts are not arbitrary; they serve a distinct narrative purpose moviecuts . Building on this, as illustrated in Figure 2 , we focus on canonical cut-driven sequences: Shot/Reverse Shot for dialogue, Cut-In/Cut-Out for emphasis, Cutaways for context, and Multi-Angle for perspective shifts. Such sequences are vital for enhancing narrative capability and inter-shot consistency. Traditionally, filming these sequences is difficult and expensive. It requires shooting different angles one by one, repeated actor performances, and careful resetting of the scene and lights. Based on these considerations, we define a new task: Next Shot Generation (NSG) . Given an existing shot, the goal is to generate a subsequent, highly coherent shot. This new shot must maintain character and environmental consistency while also adhering to established cinematic continuity principles and specific editing patterns. This task of Next Shot Generation presents significant challenges: First, it demands comprehensive cinematic continuity from complex inputs. Unlike typical subject-driven ominicontrol ; videobooth ; cao2023masactrl ; zhang2024flashface ; gal2024lcm tasks focused on isolated subject preservation, NSG requires consistency across numerous facets: character identity and appearance, spatial relationships, environmental details, lighting, color palettes, overall tone, and implied temporal progression. Achieving such multifaceted continuity from rich visual inputs is a foundational difficulty. Second, NSG aims to replicate diverse editing patterns, introducing significant visual diversity. Examples include Shot/Reverse Shot (shifting to a character’s converse view in dialogue), Cut-In (focusing on a detail for emphasis)/Cut-Out (returning to a wider context), Cutaway (providing external or subjective context), and Multi-Angle (cutting between different views of the same subject). The model must accurately synthesize these diverse visual outputs reflecting the specified editing pattern, while simultaneously upholding the comprehensive cinematic continuity. Next Shot Generation uniquely balances this required shot diversity with strict, multi-faceted continuity, making it more advanced than simpler consistency tasks. To achieve the demanding balance of Next Shot Generation (NSG), we first construct a comprehensive data foundation. This includes RawCuts, a large-scale dataset of adjacent shot pairs for foundational learning of visual transitions. Complementing this, CuratedCuts, a smaller, meticulously human-curated set, exemplifies the strong cinematic continuity and professional editing techniques crucial for NSG, serving fine-grained refinement. To guide our model in navigating the dual demands of continuity and diversity, we employ a Hierarchical Prompt Annotation scheme. This provides Relational Prompt to dictate inter-shot relationships and editing patterns, and Individual Prompts to detail per-shot content and cinematographic attributes. Methodologically, we propose Cut2Next, built upon FLUX.1-dev flux using in-context tuning. To specifically address NSG’s challenges, Cut2Next incorporates: (1) Our Context-Aware Condition Injection (CACI) mechanism for nuanced integration of diverse conditional inputs. (2) A Hierarchical Attention Mask (HAM) to orchestrate the intricate flow between visual and textual tokens. This approach allows Cut2Next to effectively utilize our rich annotations and generate high-quality, cinematically coherent next shots that masterfully balance diversity with continuity, all without introducing additional parameters to the base model. To rigorously evaluate Cut2Next’s capabilities in balancing these NSG demands, we introduce CutBench. This new benchmark features diverse movie shot images, each paired with our hierarchical prompts designed to elicit specific cinematic continuations that test both continuity and varied editing. Our quantitative results demonstrate Cut2Next significantly outperforms baselines in generating visually coherent and textually aligned subsequent shots. Crucially, to evaluate the critical balance—cinematic continuity and intended editing patterns—we conducted extensive user studies. These studies not only corroborate our quantitative findings but, more importantly, indicate a strong human preference for Cut2Next’s outputs in terms of overall cinematic continuity and faithful execution of specified editing. 2 Related Work 2.1 Multi-Shot Generation Multi-shot generation aims to create sequences of related visual outputs, either as video or sets of images, emphasizing inter-shot consistency