Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation
Min-Jung Kim, Jeongho KimHoiyeong JinJunha HyungJaegul ChooKAIST AI {emjay73,rlawjdghek,hy.jin,sharpeeee,jchoo}@kaist.ac.kr · 2025
Read Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation by Min-Jung Kim, Jeongho KimHoiyeong JinJunha HyungJaegul ChooKAIST AI {emjay73,rlawjdghek,hy.jin,sharpeeee,jchoo}@kaist.ac.kr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17040.txt.
Introduction
Min-Jung Kim and coauthors present InfCam, a depth-free method for camera-controlled video generation, where a creator steers the virtual camera after shooting. Keeping the output true to the set camera path, while filling hidden zones, is the core difficulty.
Past routes trained on path-video pairs with few path types, or warped frames by estimated depth and so inherited depth errors. InfCam instead encodes pure camera rotation as infinite homography warps inside the video diffusion latent space, a noise-free signal. A learned term then adds the parallax part. A data pipeline widens path and focal variety from synthetic multi-view sets. Tests beat baselines on pose accuracy and image quality, even on real footage.
Catalog record
| CREATOR | Min-Jung Kim, Jeongho KimHoiyeong JinJunha HyungJaegul ChooKAIST AI {emjay73,rlawjdghek,hy.jin,sharpeeee,jchoo}@kaist.ac.kr |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17040
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation Min-Jung Kim, Jeongho KimHoiyeong JinJunha HyungJaegul ChooKAIST AI {emjay73,rlawjdghek,hy.jin,sharpeeee,jchoo}@kaist.ac.kr Abstract Recent progress in video diffusion models has spurred growing interest in camera-controlled novel-view video generation for dynamic scenes, aiming to provide creators with cinematic camera control capabilities in post-production. A key challenge in camera-controlled video generation is ensuring fidelity to the specified camera pose, while maintaining view consistency and reasoning about occluded geometry from limited observations. To address this, existing methods either train trajectory-conditioned video generation model on trajectory–video pair dataset, or estimate depth from the input video to reproject it along a target trajectory and generate the unprojected regions. Nevertheless, existing methods struggle to generate camera-pose–faithful, high-quality videos for two main reasons: (1) reprojection-based approaches are highly susceptible to errors caused by inaccurate depth estimation; and (2) the limited diversity of camera trajectories in existing datasets restricts learned models. To address these limitations, we present InfCam, a depth-free, camera-controlled video-to-video generation framework with high pose fidelity. The framework integrates two key components: (1) infinite homography warping, which encodes 3D camera rotations directly within the 2D latent space of a video diffusion model. Conditioning on this noise-free rotational information, the residual parallax term is predicted through end-to-end training to achieve high camera-pose fidelity; and (2) a data augmentation pipeline that transforms existing synthetic multiview datasets into sequences with diverse trajectories and focal lengths. Experimental results demonstrate that InfCam outperforms baseline methods in camera-pose accuracy and visual fidelity, generalizing well from synthetic to real-world data. Link to our project page: https://emjay73.github.io/InfCam/ Figure 2 : (a) Infinite homography-based approach (ours). By conditioning on images warped by \mathbf{H}{\infty} , the model focuses on learning the parallax relative to the plane at infinity. This parallax is confined to the region between the epipole \mathbf{e}^{\prime} and \mathbf{x}_{\infty} , as visualized by the yellow segment on the epipolar line l^{\prime} . This spatial constraint helps the model to achieve higher camera pose fidelity with reduced search space. End-to-end training enables the network to implicitly refine the 3D geometry, correcting inaccuracies in \mathbf{X} . (b) Reprojection-based approach. Inaccuracies in depth estimation lead to unreliable conditions, causing artifacts in the generated image. Since no gradients flow through the depth estimation network, the incorrect reprojection position \mathbf{x^{\prime}} remains fixed during training, hindering error correction. 1 Introduction Changing the camera viewpoint in post-production is a highly sought-after video editing technique, as it eliminates the need for costly reshoots and enables creative visual effects. Recent advancements in large-scale video diffusion models have spurred research on controlling camera trajectories for novel-view video generation 2 ; 16 ; 13 ; 20 ; 1 . However, existing methods still suffer from several limitations. First, approaches that explicitly incorporate depth projection 20 ; 13 ; 21 suffer from inherent performance degradation due to reprojection errors arising from inaccurately estimated depths. While reprojection serves as an effective mechanism for aligning generated views with the target trajectory when accurate depth maps are available, it can also become a major source of error when the predicted depths are inaccurate. Consequently, the overall performance of such systems remains fundamentally constrained by the accuracy and reliability of the underlying depth predictor. Second, methods that learn the relationship between camera trajectory variations and novel-view video generation from trajectory–video pair datasets 1 ; 2 ; 16 tend to inherit dataset-specific biases. When the training data contain biased input–output video pairs, the network is likely to internalize these biases, leading to degraded generalization performance. To address these limitations, we propose a robust infinite homography-based conditioning method. Unlike existing depth reprojection-based methods that suffer from cascading errors due to their reliance on noisy depth estimates, our infinite homography warping module decomposes the reprojection process into known rotation and unknown translation components. We condition the model on the noise-free rotation component, while the residual translation component is learned end-to-end by leveraging strong prior knowledge from large video generation models. Additionally, we introduce a data augmentation strategy that transforms datasets with constrained viewpoints into flexible trajectory formats while simultaneously augmenting focal lengths. By synthesizing diverse camera movements and field-of-view variations, our model learns robust view synthesis. Extensive experiments demonstrate that the proposed infinite homography warping module and data augmentation enable superior trajectory fidelity compared to state-of-the-art methods. demonstrates the generalizability of our approach on challenging real-world videos. Our contributions are summarized as follows: • We propose a camera-controlled novel-view video generation framework that achieves high pose fidelity. • We introduce an infinite homography warping module that conditions on a noise-free rotation term, predicting the residual parallax term through end-to-end training. • We present a data augmentation strategy that transforms existing datasets with limited viewpoints into diverse training pairs with flexible camera trajectories and varying focal lengths, enabling robust generalization • Extensive experiments demonstrate that our infinite homography warping and data augmentation strategy significantly improve camera-controlled video generation, achieving state-of-the-art performance on both synthetic and real-world videos. 2 Related Work Depth Reprojection-Based Video Generation. This line of work 21 ; 13 ; 20 ; 11 estimates per-frame depth from an input and reprojects the resulting 3D structure along a target camera trajectory. The newly exposed or unprojected regions are subsequently predicted or inpainted to produce the final rendered video. For instance, GEN3C 13 proposes a method that builds a spatio-temporal 3D cache from videos via depth estimation and unprojection, renders this cache along the user-specified camera trajectory, and conditions a video diffusion model on these renderings to achieve precise camera control and temporal consistency. Meanwhile, TrajectoryCrafter 20 introduces a dual-stream conditional video diffusion framework that takes as input both the source video and reprojection results, which are subsequently processed by a video inpainting model fine-tuned on a dataset generated using a double reprojection scheme. Since this line of work relies on external depth estimation, its performance is ultimately limited by the accuracy of the estimated depth, which directly affects the magnitude of reprojection errors. Trajectory-Conditioned Video Generation. This line of work 16 ; 2 ; 1 ; 9 ; 17 ; 14 ; 19 trains a video generation model that is explicitly conditioned on camera trajectories, using paired datasets of trajectories and corresponding videos. SynCamMaster 2 builds upon a pre-trained text-to-video model with a multi-view synchronization module to produce synchronized videos from diverse camera viewpoints. Trained with a hybrid data scheme and the released SynCamVideo dataset, it demonstrates open-domain novel-view video generation for stationary cameras. ReCamMaster 1 further leverages the generative capabilities of pre-trained text-to-video models through frame-dimension conditioning, and is trained with the MultiCamVideo dataset, a large-scale multi-camera synchronized video dataset built with Unreal Engine 5. While effective, such trajectory-conditioned video generation methods trained on trajectory–video pair data often inherit biases from the underlying training data distribution. Figure 3 : Model Architecture Overview. Our model builds upon Wan2.1, training only newly introduced parameters while freezing pretrained weights. (a) DiT block with homography-guided self-attention layer. Homography-guided self-attention layer takes source, target, and warped latents combined with camera embeddings as input, and performs per-frame attention, ensuring temporal alignment. By conditioning on warped latents, the model enables rotation-aware reasoning and constrained parallax estimation. Only source and target latents proceed to the subsequent Wan2.1 layers. (b) Warping module . This module warps