EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance
Zun Wang, Jaemin Cho, Jialu Li, Han Lin, Yue Zhang, Mohit Bansal · 2025
Read EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance by Zun Wang, Jaemin Cho, Jialu Li, Han Lin, Yue Zhang, Mohit Bansal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21876.txt.
Introduction
Zun Wang and a team of co-authors wrote this 2025 paper on teaching video generators to move their cameras on command. Current methods render a rough anchor video from estimated point clouds to show the desired motion, but estimation errors make the anchor misleading, and the model wastes capacity forgiving it. EPiC builds anchors differently, with no pose or point cloud estimation at all.
Masking a source video by what its first frame can see yields anchors aligned by construction, from any ordinary footage. A small add-on module, Anchor-ControlNet, feeds the anchor into a pretrained diffusion model using under one percent extra parameters. Training then needs fewer steps and less data, yet the model still obeys point-cloud anchors at test time. The authors claim precise, three-dimensional camera control at a fraction of the usual cost.
Catalog record
| CREATOR | Zun Wang, Jaemin Cho, Jialu Li, Han Lin, Yue Zhang, Mohit Bansal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21876
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance Zun Wang, Jaemin Cho, Jialu Li, Han Lin, Yue Zhang, Mohit Bansal Abstract Controllable 3D camera trajectories in video diffusion models are highly sought after for content creation, yet remain a significant challenge. Recent approaches often create anchor videos (i.e., rendered videos that approximate desired camera motions) to guide diffusion models as a structured prior, by rendering from estimated point clouds following annotated camera trajectories. However, errors inherent in point cloud estimation often lead to inaccurate anchor videos. Moreover, the requirement for extensive camera trajectory annotations further increases resource demands. To address these limitations, we introduce EPiC, an efficient and precise camera control learning framework that automatically constructs high-quality anchor videos without expensive camera trajectory annotations. Concretely, we create highly precise anchor videos for training by masking source videos based on first-frame visibility. This approach ensures high alignment, eliminates the need for camera trajectory annotations, and thus can be readily applied to any in-the-wild video to generate image-to-video (I2V) training pairs. Furthermore, we introduce Anchor-ControlNet, a lightweight conditioning module that integrates anchor video guidance in visible regions to pretrained video diffusion models, with less than 1% of backbone model parameters. By combining the proposed anchor video data and ControlNet module, EPiC achieves efficient training with substantially fewer parameters, training steps, and less data, without requiring modifications to the diffusion model backbone typically needed to mitigate rendering misalignments. Although being trained on masking-based anchor videos, our method generalizes robustly to anchor videos made with point clouds during inference, enabling precise 3D-informed camera control. EPiC achieves state-of-the-art performance on RealEstate10K and MiraData for I2V camera control task, demonstrating precise and robust camera control ability both quantitatively and qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to video-to-video (V2V) scenarios. This is compelling as it is trained exclusively on I2V data, where anchor videos are derived from source videos, using only their first frame for visibility referencing. 1 Introduction Recent advancements in video diffusion models (VDMs) bar2024lumiere ; girdhar2311emu ; hong2022cogvideo ; khachatryan2023text2video ; wang2023modelscope ; zhang2024show ; blattmann2023stable ; kondratyuk2023videopoet have dramatically enhanced the ability to generate dynamic and realistic videos. As video generation becomes increasingly practical and widespread, controllability has emerged as a crucial requirement for creating personalized and creative content. Previous works have explored various control signals to guide video generation, such as optical flow jin2025flovd ; koroglu2024onlyflow ; cong2023flatten , object trajectories yin2023dragnuwa ; wu2024draganything ; zhang2024tora ; shi2024motion ; chen2023motion ; wang2024dreamrunner , human poses ma2024follow ; lin2024ctrl , and depth maps lin2024ctrl ; chen2023control . In particular, controlling camera trajectories during the video generation process has emerged as a key research focus, facilitating precise spatio-temporal manipulation essential for downstream applications such as film recapturing bai2025recammaster ; yu2025trajectorycrafter , virtual cinematography ren2025gen3c , and augmented reality rendering shi2024stereocrafter . To achieve precise camera control, recent works ren2025gen3c ; yu2025trajectorycrafter ; cao2025uni3c ; zhang2024recapture ; ren2025gen3c ; yu2024viewcrafter have adopted explicit 3D-informed guidance for generation. The core idea is to construct an ‘anchor video’ (i.e., a video that approximates the desired camera motion to guide a diffusion model as a structured prior), by lifting a condition image into a 3D point cloud and rendering it along the camera trajectory. Training the camera control module typically requires anchor video and the corresponding full source video as input-output pairs, ideally with perfect geometric alignment. This assumes access to ground-truth 3D point clouds and camera trajectories, which are hard to obtain. As a workaround, existing methods synthesize training anchor-source video pairs by using source videos with high-quality camera annotations and estimating a point cloud from the first frame via off-the-shelf estimators wang2024dust3r ; yang2024depth , which is then rendered along the annotated trajectory as the anchor video. However, these estimators often introduce geometric inaccuracies, leading to misaligned regions in the rendered anchor videos (as illustrated in Fig. 1 (a)), making training more challenging, as the model must additionally learn to correct random misalignments beyond filling invisible regions. Moreover, the requirement of annotated camera trajectories from the source video restricts training data to multi-view video datasets such as RealEstate10K RealEstate10k and DL3DV ling2024dl3dv . These datasets mainly feature static scenes, thereby limiting the generalization ability of the trained camera control module to more dynamic or diverse real-world settings. To address the issues, we propose EPiC, for learning E fficient and P recise V i deo C amera control by crafting precisely-aligned training anchor videos with a lightweight ControlNet model design ( Section 4 ). Our key insight is that anchor videos should be well-aligned with the source videos to make learning both easier and more efficient, transforming the task from one of repairing misaligned content to the simpler task of copying visible regions. Thus, unlike previous approaches that render anchor videos from inaccurate 3D point clouds which often misaligned with the source video and reliant on annotated camera trajectories ( Fig. 1 (a) right), we directly synthesize anchor videos by masking the source video based on first-frame visibility ( Section 4.1 ), as described in Fig. 1 (b). Specifically, for each subsequent frame, we estimate its pixels trajectory with respect to the first frame from dense optical flow teed2020raft , preserving only those pixels that can be reliably traced back to the first frame. Pixels with no valid correspondence in the first frame are masked out. This process effectively mimics the key property of anchor videos—all new regions relative to the first frame are invisible—while ensuring precise alignment in visible regions ( Fig. 1 (b) right). Furthermore, our approach eliminates the need for camera trajectory annotations, allowing anchor videos to be created from any in-the-wild source. Furthermore, in contrast to prior methods that require extensive backbone modifications or heavy fine-tuning, we introduce a lightweight Anchor-ControlNet ( Section 4.2 ), which has only 30M parameters (less than 1% parameters of the CogVideoX 5B backbone) and injects anchor-video-based control signals into the generation process with the base model frozen. Unlike previous methods, such as ViewCrafter yu2024viewcrafter , which condition on the entire anchor video without visibility awareness, we apply visibility-aware masking to the outputs of our Anchor-ControlNet. Specifically, the ControlNet’s output is added to the latent representation only within the visible regions, leaving the unseen areas untouched. This design simplifies the ControlNet’s task to copying visible content, while delegating the synthesis of occluded or invisible regions entirely to the base diffusion model. This clear division of responsibility not only reduces learning difficulty but also improves overall generation quality. Combining these components, we demonstrate that anchor-video-based camera control can be learned in a highly efficient manner, achieving strong performance with just 5K in-the-wild training videos and 500 training steps, which is less than 10% of the data and iterations used in prior approaches. Figure 1: Comparison of anchor video creation methods for training camera control models. (a) Previous methods ( ren2025gen3c ; yu2024viewcrafter ) estimate the 3D point cloud (through depth estimation) using the first frame and render anchor videos with annotated camera trajectories, but suffer from region misalignment due to point-cloud estimation errors while limited to camera-pose annotated data, resulting in inefficient training. (b) Our method creates anchor videos via visibility masking based on first-frame