Stable Virtual Camera : Generative View Synthesis with Diffusion Models
Jensen (Jinghao) Zhou Hang Gao, Vikram Voleti Aaryaman Vasishta Chun-Han Yao Mark Boss, Philip Torr Christian Rupprecht Varun Jampani · 2025
Read Stable Virtual Camera : Generative View Synthesis with Diffusion Models by Jensen (Jinghao) Zhou Hang Gao, Vikram Voleti Aaryaman Vasishta Chun-Han Yao Mark Boss, Philip Torr Christian Rupprecht Varun Jampani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14489.txt.
Introduction
Given one photo of a room, a filmmaker might wish for a slow orbit around it; Jensen Zhou and co-authors built a diffusion model that grants the wish. Stable Virtual Camera accepts any number of input views and any target camera path, then generates the missing views. Earlier methods stumbled on wide swings of viewpoint or produced jittery sequences unless built for one narrow task.
Simple design choices, a tuned training recipe, and flexible sampling let one model cover many view-synthesis jobs without leaning on an explicit three-dimensional reconstruction. It can even spin out half-minute videos that loop seamlessly. Benchmarks across datasets put it ahead of prior methods. The authors released the model and code, aiming at virtual production and scene exploration from sparse captures.
Catalog record
| CREATOR | Jensen (Jinghao) Zhou Hang Gao, Vikram Voleti Aaryaman Vasishta Chun-Han Yao Mark Boss, Philip Torr Christian Rupprecht Varun Jampani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-14489
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stable Virtual Camera : Generative View Synthesis with Diffusion Models Jensen (Jinghao) Zhou Hang Gao, Vikram Voleti Aaryaman Vasishta Chun-Han Yao Mark Boss, Philip Torr Christian Rupprecht Varun Jampani Abstract We present S tabl e V irtual C a mera ( Seva ), a generalist diffusion model that creates novel views of a scene, given any number of input views and target cameras. Existing works struggle to generate either large viewpoint changes or temporally smooth samples, while relying on specific task configurations. Our approach overcomes these limitations through simple model design, optimized training recipe, and flexible sampling strategy that generalize across view synthesis tasks at test time. As a result, our samples maintain high consistency without requiring additional 3D representation-based distillation, thus streamlining view synthesis in the wild. Furthermore, we show that our method can generate high-quality videos lasting up to half a minute with seamless loop closure. Extensive benchmarking demonstrates that Seva outperforms existing methods across different datasets and settings. Figure 1 : Generative view synthesis. Stable Virtual Camera generates novel views from any number of input views and target cameras, which the user can specify anywhere. We show three examples: single view with simple orbit camera trajectory (top); two views with long camera trajectory (middle); and nine views with large spatial range (bottom). Please visit our website for video samples. † † footnotetext: ∗ Equal contribution. † † footnotetext: † Jensen and Hang did the work during internships at Stability AI. 1 Introduction Novel view synthesis (NVS) aims to generate realistic, 3D-consistent images of a scene from arbitrary camera viewpoints given any number of camera-posed input views. Traditional methods, which rely on dense input views, treat NVS as a 3D reconstruction and rendering problem [ 1 , 2 , 3 ] , but this approach fails with sparse inputs. Generative view synthesis addresses this limitation by leveraging modern deep network priors [ 4 , 5 ] , enabling immersive 3D interactions in uncontrolled environments without the need to capture large image sets per scene. In this work, we focus on generative view synthesis and, unless otherwise specified, refer to it simply as NVS for clarity. Despite recent progress [ 6 , 7 , 8 , 9 , 10 , 11 , 12 ] , NVS in the wild remains limited due to two key challenges: First, existing methods struggle to generate both large viewpoint changes [ 10 , 9 ] and temporally smooth samples [ 6 , 13 , 7 , 8 ] while being constrained by rigid task configurations, such as a fixed number of input and target views [ 7 , 11 , 9 , 12 ] , reviewed in Tab. 1 . Second, their sampling consistency is often insufficient, necessitating additional NeRF distillation to fuse inconsistent results into a coherent representation [ 13 , 7 , 8 ] . These limitations hinder their applicability across diverse NVS tasks, which we address in this work. We present S tabl e V irtual C a mera 1 1 1 We name this model in tribute to the Virtual Camera [ 14 ] cinematography technology, a pre-visualization technique to simulate real-world camera movements. ( Seva ), a diffusion-based NVS model that generalizes across a spectrum of view synthesis tasks without requiring NeRF distillation. With a single network, Seva generates high-quality novel views that strike both large viewpoint changes and temporal smoothness, while supporting any number of input and target views. Our approach simplifies the NVS pipeline without requiring distillation from a 3D representation, thus streamlining it for real-world applications. For the first time, we demonstrate high-quality videos lasting up to half a minute with precise camera control and seamless loop closure in 3D. We highlight these results in Fig. 1 and showcase more examples of camera control in Fig. 2 . To achieve this, we carefully design our pipeline in three key aspects: model design, training recipe, and sampling method at inference. First, Seva avoids explicit 3D representations within the network, allowing the model to inherit strong priors from pre-trained 2D models. Second, during training, we carefully craft our view selection strategy to cover both small and large viewpoint changes, ensuring strong generalization to diverse NVS tasks. Third, at inference, we introduce a two-pass procedural sampling approach that supports flexible input-target configurations. Together, these design choices create a versatile 3D “virtual camera simulation system” capable of synthesizing novel views along arbitrary camera trajectories with any number of input and target views, without using a 3D representation. We conducted a unified benchmark across 10 datasets and a variety of experimental settings, including both open-source and proprietary models. Our benchmark reflects the diversity of real-world NVS tasks across the board and systematically evaluates existing methods beyond their comfort zones. We find that Seva consistently outperforms previous works, achieving +1.5 dB PSNR over the state of the art CAT3D [ 8 ] in its own setup. Moreover, our method generalizes well to in-the-wild user captures, with input views ranging from 1 to 32. In summary, our key contributions with the Seva model include: (1) a training strategy for jointly modeling large viewpoint changes and temporal smoothness, (2) a two-pass procedural sampling method for smooth video generation along arbitrary long camera trajectories, (3) a comprehensive benchmark that evaluates NVS methods across different datasets and settings, and (4) an open-source release of model weights to support future research. model training generation interpolation input data capacity smoothness flexibility Regression-based pixelNeRF [ 4 ] ✗ ✓ sparse (1) pixelSplat [ 16 ] ✗ ✓ sparse (2) MVSplat [ 17 ] ✗ ✓ sparse (2) Long-LRM [ 18 ] ✗ ✓ semi-dense ({16, 32}) LVSM [ 11 ] / ✗ ✓ sparse ({2, 4}) Diffusion-based: image models Zero123 [ 6 ] ✓ ✗ sparse (1) ZeroNVS [ 13 ] ✓ ✗ sparse (1) ReconFusion [ 7 ] ✓ ✗ sparse (3) CAT3D [ 8 ] ✓ ✗ sparse ([1, 9]) Diffusion-based: video models SV3D [ 19 ] ✗ ✓ sparse (1) MotionCtrl [ 10 ] ✗ ✓ sparse (1) ViewCrafter [ 9 ] ✗ ✓ sparse (2) 4DiM [ 12 ] ✓ ✓ sparse ({1, 2, 8}) Seva ✓ ✓ sparse ([1, 8]), semi-dense ([9, 32 ∗ ]) Table 1 : Comparison of existing NVS models based on the source of training data and key attributes. Seva is trained on both object-level ( ) and scene-level ( ) data, offering flexible input conditioning, strong generation capacity, and smooth view interpolation. We define generation capacity and interpolation smoothness of each work based on their evaluation setting and our benchmark results. ∗ This upper-bound can be up to hundreds for dense captures, we test our model up to 32 views in practice. Figure 2 : Diverse camera control. Seva generates photorealistic novel views following diverse camera trajectories. This includes orbit, spiral, zoom out, dolly zooms, and any user-specified trajectories. Please visit our website for more visual results. 2 Background We consider the evaluation of an NVS model across three key criteria: (1) generation capacity—the ability to synthesize missing regions for large viewpoint changes; (2) interpolation smoothness—the ability to produce seamless transitions between views; and (3) input flexibility—the ability to handle a variable number of input and target views;