Bolt3D: Generating 3D Scenes in Seconds

Stanislaw Szymanowicz, Jason Y. Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Hołyński, Ricardo Martin-Brualla, Jonathan T. Barron, Philipp Henzler · 2025

Read Bolt3D: Generating 3D Scenes in Seconds by Stanislaw Szymanowicz, Jason Y. Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Hołyński, Ricardo Martin-Brualla, Jonathan T. Barron, Philipp Henzler (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14445.txt.

Introduction

Stanislaw Szymanowicz and co-authors present Bolt3D in a 2025 paper on fast three-dimensional scene generation. Given one or a few images, their model produces a full 3D scene representation in under seven seconds on a single graphics processor, where older generative methods needed slow per-scene optimization.

Bolt3D is a latent diffusion model built on proven two-dimensional diffusion designs, adapted to output consistent 3D geometry and appearance. To train it, the team converted existing multi-view photo collections into a large dataset of scenes using state-of-the-art reconstruction tools. The speedup over earlier approaches reaches into the hundreds of times.

Catalog record

CREATORStanislaw Szymanowicz, Jason Y. Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Hołyński, Ricardo Martin-Brualla, Jonathan T. Barron, Philipp Henzler
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-14445

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bolt3D: Generating 3D Scenes in Seconds Stanislaw Szymanowicz, Jason Y. Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Aleksander Hołyński, Ricardo Martin-Brualla, Jonathan T. Barron, Philipp Henzler Abstract We present a latent diffusion model for fast feed-forward 3D scene generation. Given one or more images, our model Bolt3D directly samples a 3D scene representation in less than seven seconds on a single GPU. We achieve this by leveraging powerful and scalable existing 2D diffusion network architectures to produce consistent high-fidelity 3D scene representations. To train this model, we create a large-scale multiview-consistent dataset of 3D geometry and appearance by applying state-of-the-art dense 3D reconstruction techniques to existing multiview image datasets. Compared to prior multiview generative models that require per-scene optimization for 3D reconstruction, Bolt3D reduces the inference cost by a factor of up to 300 × \times . Project website: szymanowiczs.github.io/bolt3d . Figure 1 : Given an arbitrary number of input images, Bolt3D directly outputs a 3D representation which can be rendered at interactive frame-rates. Operating in a feed-forward manner, generation takes mere seconds. Bolt3D features a latent diffusion model with a scalable 2D architecture, trained on large-scale appearance and geometry data, enabling generation of full 360 ∘ scenes from one or multiple input images. We encourage the reader to explore these scenes in the interactive viewer available on the project website. 1 Introduction Modern image and video generative models generate compelling high-quality visual content, but these models sample 2D images, rather than an underlying 3D scene. The ability to directly generate 3D content instead would enable numerous applications, such as interactive visualization and editing. However, scaling modern diffusion-based generative models to generate detailed 3D scenes remains a significant challenge for the research community, primarily due to two reasons. First, representing and structuring (possibly unbounded) 3D data to enable training a diffusion model that generates full scenes at high resolution is an unsolved problem. Second, “ground truth” 3D scenes are extremely scarce compared to the abundant 2D image and video data used to train state-of-the-art generative models. As a result, many recent 3D generative models are limited to synthetic objects [ 28 , 62 , 64 , 74 ] or partial “forward-facing” scenes [ 27 , 68 , 67 , 60 ] . Models that scale to real, full 360^{\circ} scenes use camera-conditioned multiview or video diffusion models to turn input image(s) into a large “dataset” of synthetic observations [ 12 , 34 ] , from which an explicit 3D representation (such as a neural [ 36 ] or 3D Gaussian [ 23 ] radiance field) is then recovered via test-time optimization. While this approach is capable of producing high-quality 3D content, it is impractical; both sampling hundreds of augmented images with the multiview diffusion model and optimizing a 3D representation to match these images are slow and compute-intensive. In this paper, we present a latent diffusion model for fast feed-forward 3D scene generation from one or more images. Our model, called Bolt3D, leverages the tremendous progress made in scalable 2D diffusion model architectures to generate an explicit 3D scene representation. We represent 3D scenes as sets of 3D Gaussians, stored in multiple 2D grids in which each cell stores the parameters of one pixel-aligned Gaussian [ 49 , 4 ] (“Splatter Images” [ 49 ] ). Crucially, unlike prior work, we use more Splatter Images than input views, and we generate Splatter Images using a diffusion model, which enables generating content for unobserved regions of the scene. Our generation process consists of two parts: denoising color and position of each Gaussian and subsequently regressing each Gaussian’s opacity and shape. Given a set of posed input images and target camera poses, our model jointly predicts the scene appearance (pixel colors) viewed by the target cameras as well as per-pixel 3D coordinates of scene points in all cameras (both input and target). We enable Bolt3D to predict accurate high-resolution per-pixel 3D geometry by designing and training a Geometry Variational Auto-Encoder (VAE [ 11 ] ) that resembles architectures used in image generation, but which we train from scratch using geometry data. Unlike 2D image datasets, real-world 3D scene datasets are small and limited. To address this, we create a large-scale geometry dataset by running a robust Structure-from-Motion framework [ 24 ] on large-scale multi-view image datasets. We use this new dataset to train our geometric VAE and diffusion model. Finally, to obtain a renderable 3D representation (which 3D point coordinates alone do not directly provide), we train a Gaussian head network that takes the full set of high-resolution images, geometry maps, and camera poses predicted by the denoising network as input and predicts the remaining properties of 3D Gaussians (opacities, shapes), and refined colors. The Gaussian head is supervised with rendering losses, which enables our full model’s output to render high-quality novel views. We demonstrate that Bolt3D outperforms prior single- and few-view feed-forward 3D regression methods and synthesizes detailed 3D scene content, even in ambiguous regions that are not observed in any input image. Furthermore, we show that Bolt3D reduces inference cost up to 300\times compared to multi-view image generation methods, which typically require per-scene optimization to reconstruct 3D models. Figure 2 : Method. Bolt3D takes as input one or more posed, observed images, and a set of target poses (a), and outputs a renderable 3D scene (e). First, we use a multi-view latent diffusion model to sample per-view latent appearance and geometry (b). The appearance and geometry latents are independently decoded to full-resolution images and pointmaps (c) using a pre-trained image VAE decoder and our trained geometry decoder, respectively. Next, a multi-view Gaussian head predicts the opacities and scales of pixel-aligned 3D Gaussians, and refines the predicted colors. Together with the pointmap from (c), these values form Splatter Images [ 49 ] (d), which can be combined to create a complete 3D Gaussian representation of the scene (e). 2 Related work Feed-forward 3D regression. Reconstructing a 3D scene from an input image (or from multiple images) is sometimes formulated as a feed-forward regression problem. Many different 3D representations have been developed towards this goal, including multi-plane images [ 54 , 25 ] , voxels [ 51 , 53 ] , meshes [ 15 , 13 ] , and radiance fields [ 66 , 20 ] . Most relevant to us are recent approaches that output 3D Gaussians [ 23 ] , a representation favored for its real-time rendering speed. Splatter Image [ 49 ] and pixelSplat [ 4 ] were the first to associate Gaussians with the pixels of one or two input images, respectively. Follow-up works iterated on this approach with improved architectures [ 73 ] , depth conditioning [ 50 , 63 ] , explicit feature matching [ 5 ] , and removing camera pose requirements [ 65 ] . Although these regression-based methods are capable of accurately reconstructing observed regions, they tend to produce blurry results in unseen regions. In contrast, Bolt3D is a generative approach and is thereby capable of generating unobserved regions of the scene. Reconstruction via 2D image generation. A common paradigm for addressing ambiguity in few-view 3D reconstruction is to generate multiple views of the scene, from which 3D can be recovered. Zero-1-to-3 [

Read the complete text