Probing into Camera Control of Video Models

Chen Hou, Christian Rupprecht · 2026

Read Probing into Camera Control of Video Models by Chen Hou, Christian Rupprecht (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14815.txt.

Introduction

Hou and Rupprecht reframe camera control for video models as geometric guidance rather than an implicit mapping. Existing methods learn camera motion to video mappings with extra camera modules and paired data, which can bias models toward narrow outputs. The authors instead reformulate camera control into displacement fields applied by differentiable resampling of latent features during denoising. Effective camera control is claimed with minimal quality loss versus fine-tuned baselines, and the training-free method doubles as a probe revealing universal biases across video models.

Catalog record

CREATORChen Hou, Christian Rupprecht
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57199
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14815

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Probing into Camera Control of Video Models Chen Hou, Christian Rupprecht Abstract Video is a rich and scalable source of 3D/4D visual observations, and camera control is a key capability for video generation models to produce geometrically meaningful content. Existing approaches typically learn a mapping from camera motion to video using additional camera modules and paired data. However, such datasets are often limited in scale, diversity, and scene dynamics, which can bias the model toward a narrow output distribution and compromise the strong prior learned by the base model. These limitations motivate a different perspective on camera control. In this paper, we show that camera control need not be modeled as an implicit mapping problem, but can instead be treated as a form of geometric guidance that induces displacements across frames. Specifically, we reformulate camera control into a set of displacement fields and apply them via differentiable resampling of latent features during denoising. Our simple approach achieves effective camera control with minimal degradation across diverse quality metrics compared to fine-tuned baselines. Since our method is applicable to most video diffusion models without training, it can also serve as a probe to study the camera control capabilities of base models. Using this probe, we identify universal biases shared by representative video models, as well as disparities in their responses to camera control. Finally, we benchmark their performance in multi-view generation, offering insights into their potential for 3D/4D tasks. 1 1 1 Webpage: https://xrchitech.github.io/camprobe-page/ Code: https://github.com/xrchitech/CamProbe 1 Introduction As the most common data modalities for capturing the real world, video naturally encodes both spatial structure and temporal dynamics, making it one of the richest sources of 3D and 4D visual signals. A key mechanism for accessing such stereoscopic structure lies in camera motion, which defines how a scene is observed over time. Enabling controllable camera motion is therefore crucial for bridging scalable 2D foundation models with 3D/4D understanding and generation, and is essential for applications such as view synthesis, downstream 3D reconstruction, and building the world model. Recent approaches typically achieve camera control by fine-tuning video models on camera-pose-annotated datasets 4 ; 13 ; 55 . The model is trained to learn a direct mapping from camera trajectories to generated videos. However, such a paradigm presents several fundamental limitations. First, the relationship between camera motion and generated content is implicitly encoded in the model and remains largely uninterpretable. Since only few geometric constraints are imposed, models may learn dataset-specific correlations rather than the underlying camera geometry. Second, the available camera-annotated datasets are often limited in scale, diversity, and dynamics, as they are typically generated by rendering engines or Structure-from-Motion pipelines 4 ; 37 ; 58 . Such datasets usually lack realistic motion patterns, long temporal dynamics, and complex scene variations, which restricts the range of camera motions and scenarios the model can generalize to. Third, fine-tuning on such specialized datasets tends to bias the model towards a narrower output distribution. This adaptation can override the rich and diverse priors learned during large-scale pretraining, leading to reduced visual diversity and degraded generation quality. Some training-free camera control methods 16 avoid such problems by manipulating latent feature layouts but rely heavily on non-differentiable operations, such as point cloud reconstruction and inpainting, thereby rendering end-to-end optimization intractable. These limitations motivate a different perspective on camera control: since video diffusion models are already capable of generating realistic content, camera motion, and temporal dynamics, much of the complexity of video generation has been implicitly learned during pretraining. As a result, camera motion need not be modeled as a new generative process requiring training; it can instead be treated as a form of geometric guidance. Building on this insight, we reformulate camera control as a set of displacement fields integrated into the diffusion process. Given a camera trajectory, we first derive displacement fields that encode the spatial transformations induced by camera motion, then apply them via differentiable resampling of latent features during denoising. The displacement fields can either be heuristically defined or learned through end-to-end optimization. In this work, we show that depth-based geometry provides an effective means of constructing displacement fields. The approach enables effective camera control for video models with minimal degradation in quality across diverse evaluation metrics, even without training. Beyond camera control, our formulation also serves as a probe for analyzing the camera control capabilities of video diffusion models. Effective evaluation of camera control in video generation remains underexplored. Current benchmarks are either designed for video understanding tasks with VLMs 28 or rely on manually defined criteria for camera motion, which often fail to give correct assessments 57 . Moreover, in most cases, camera control is driven by text control. This setup not only entangles camera control with text-conditioned generation but, more importantly, does not faithfully reflect the model’s camera control capability, as prompts often fail to induce the intended motion. However, our experiments suggest that these prompt-induced failures do not necessarily reflect an inability of the models, but instead arise from limitations of the guidance mechanism (Section 4.2.1 ). Using our probe, we draw several interesting conclusions, including shared biases across models in motion types and directions, as well as notable disparities in their response to camera control. We further evaluate their performance in generating multi-view geometry, providing insights into their potential use for downstream 3D/4D tasks. In summary, our contributions are: • We reformulate camera control as a set of displacement fields applied via differentiable resampling during denoising, enabling camera control without any training or additional modules. • We propose using this formulation as a probe to study the camera control capability of video foundation models, thereby isolating model behavior from induction artifacts. • Using this probe, we identify several universal biases shared across representative video models, including underestimated camera control capability, asymmetric translation-rotation leakage, horizontal directional preference, as well as clear disparities in their sensitivity to camera control. • We introduce a multi-level evaluation protocol (2D, 2.5D, and 3D) for assessing multi-view consistency in video generation and benchmark five video models under orbital camera motions. 2 Related Work Camera-controllable video generation. Following the success of video models, camera-controllable video generation has attracted increasing attention 13 ; 16 ; 4 . Existing methods explore camera control from several directions. A primary line of research focuses on how to introduce camera signals into video models through various conditioning mechanisms, including adapter-based modules (e.g., LoRA 17 ) 9 ; 56 , camera-aware embeddings 13 ; 47 ; 3 ; 2 , and modifications to attention 4 ; 18 ; 22 ; 51 . These approaches are typically trained via fine-tuning to learn a direct mapping from input trajectories to output videos. In addition, some works focus on dataset construction, curating or synthesizing videos with camera annotations to support supervised training 5 ; 4 ; 37 ; 14 . Another line of work incorporates explicit structures 55 ; 38 ; 49 ; 7 ; 54 ; 16 or geometric constraints 52 ; 23 into the generation process, thereby providing inherent consistency in the generated visual signals. Generative novel view synthesis. Pretrained 2D models have been widely used for 3D/4D content generation. Existing approaches mainly follow several paradigms: fine-tuning feed-forward models for view synthesis 44 ;

Read the complete text