OrbitForge: Text-to-3D Scene Generation via Reconstruction-Anchored Video Synthesis
Chenrui Fan, Paolo Favaro · 2026
Read OrbitForge: Text-to-3D Scene Generation via Reconstruction-Anchored Video Synthesis by Chenrui Fan, Paolo Favaro (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24799.txt.
Introduction
Chenrui Fan and Paolo Favaro present OrbitForge, a system that turns a text prompt into a full orbit 3D scene. Text to video models hold rich knowledge of how scenes look. Their videos, though, do not directly give reliable 3D assets. OrbitForge adapts a frozen video model with per prompt Gaussian Splatting reconstruction. The reconstruction acts as an anchor that enforces 3D consistency.
A first video yields a preliminary reconstruction using a deformable splatting method with a robust proxy. Rendering that scene along a prescribed orbit reveals which viewpoints are missing. The video model then completes only those missing views. A final reconstruction turns the completed orbit into the finished scene. On a frozen audit of 300 prompts, the median measured orbit span reaches 359 degrees. Quality in previously unsupported views more than doubles by one reward measure. No task specific fine tuning is used anywhere in the pipeline.
Catalog record
| CREATOR | Chenrui Fan, Paolo Favaro |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24799
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OrbitForge: Text-to-3D Scene Generation via Reconstruction-Anchored Video Synthesis Chenrui Fan, Paolo Favaro Abstract Generic text-to-video models can be used as rich open-world scene priors. Despite the high quality of today’s generated videos, they do not directly yield reliable 3D assets: camera motion is difficult to control, view coverage is partial, and frames often contain inconsistencies across time. We introduce OrbitForge, an adapter built from frozen video priors and per-prompt Gaussian Splatting reconstruction optimization that converts a single text-generated video into a canonical closed-orbit 3D Gaussian Splatting scene. We use 3D reconstruction as an anchor to improve the 3D consistency of the generated video. We obtain a preliminary 3D reconstruction from a first generated video via Deformable Gaussian Splatting with a robust MedianGS proxy. We render views from a prescribed orbit to detect missing viewpoints. OrbitForge uses the text-to-video model to complete only the missing views, and reconstructs the completed orbit into a final Gaussian Splatting scene. This design requires no task-specific video or multiview fine-tuning, avoids per-prompt score-distillation optimization, and does not progressively generate views one step at a time. We further argue that this setting demands coverage-aware evaluation: local smoothness alone rewards methods that never attempt a full orbit. On a frozen 300-prompt T 3 Bench-derived audit, OrbitForge reconstruction attains a 359.0-degree measured median span, raises originally unsupported-bin Q10 ImageReward from 8.07 to 16.36 relative to MedianGS-only reconstruction, while remaining competitive with VideoMV on the coverage-quality. 1 Introduction Text-driven 3D generation has made remarkable progress by borrowing the visual knowledge of large 2D and video diffusion models. Yet there remains a sharp gap between a beautifully generated image or video and a usable 3D scene. A 3D scene is not merely a collection of plausible views: it must live in a coordinate system, remain renderable from requested cameras, keep the depicted object and scene recognizable around the orbit, and expose the back, sides, ground, and surrounding environment rather than only the most photogenic front view. Existing approaches address this challenge from one of three directions. Score-distillation methods optimize a 3D representation per prompt using a frozen image prior, often producing renderable object-centric assets but requiring long optimization and suffering from over-saturation, Janus artifacts, or weak scene context [ 13 , 17 , 8 , 18 ] . Specialized multiview generators learn a camera-conditioned prior and produce a fixed set of views that is convenient for reconstruction, but the camera format, elevation, and object-centric assumptions are usually built into the model [ 14 , 16 , 21 , 7 ] . Local novel-view methods can synthesize high-quality short trajectories from one or a few images, but their strength is local view completion, not necessarily closed 360-degree scene generation from an unconstrained text-generated video [ 3 , 20 , 9 ] . Figure 1: Example outputs from OrbitForge across six prompts. Each row renders the final Gaussian Splatting scene on the same canonical orbit at 30-degree intervals, showing complete orbit coverage while preserving object identity, ground contact, and surrounding scene context. We study a complementary question: can a frozen generic text-to-video model be used as a closed-orbit 3D scene generator without training a new multiview prior or running per-prompt Score Distillation Sampling (SDS) optimization? Generic text-to-video models are attractive because they already produce scene-level realism: objects rest on floors, reflect light, interact with backgrounds, and inherit open-world semantic detail from language prompts. However, their videos are not calibrated multiview captures. Camera motion is implicit and often covers only a partial arc. Object shape can deform, textures can be repainted, and backgrounds can drift. Directly fitting 3D Gaussians to such frames turns these contradictions into floaters, blurred geometry, stretched surfaces, and view-dependent artifacts. The key observation behind OrbitForge is that a failed first reconstruction is still useful. Even when it is imperfect, it has performed a crucial act: it has organized the generated frames into a shared 3D coordinate system. Once this preliminary reconstruction is rendered along a prescribed canonical orbit, the problem becomes structured. Some views are supported by the source trajectory; other views are visibly unsupported. The method can then complete the unsupported orbit interval by querying the text-to-video model again, while keeping the supported arc fixed, and reconstruct again under known cameras (Figure 1 shows several outputs generated by OrbitForge). Therefore, this paper makes a precise claim. OrbitForge is not designed to dominate every local aesthetic metric, and it should not be evaluated as if a narrow view-completion trajectory and a complete scene orbit were the same task. Its target is prompt-controllable, scene-level, closed-orbit 3D generation from generic text-to-video priors without task-specific generative-model training or per-prompt SDS optimization, while still using per-prompt Gaussian Splatting reconstruction optimization. A fair evaluation must measure coverage before rewarding local smoothness. Our contributions are fourfold. First, we formulate closed-orbit 3D scene generation from generic text-generated videos, a setting that leverages frozen text-to-video priors while explicitly handling uncalibrated cameras, partial coverage, and generated-video inconsistency. Second, we propose OrbitForge, a reconstruction-render-completion adapter that does not train a new generative prior and uses an imperfect first reconstruction as an anchor rather than as a final output. Third, we introduce MedianGS static scene consolidation and coverage-aware endpoint-window orbit completion, which convert a partial generated-video trajectory into a complete canonical-orbit reconstruction signal without progressive view expansion. Fourth, we introduce a coverage-aware evaluation protocol–measured angular span, monotone progress, closed-orbit success, lower-tail quality, and diagnostic appearance consistency–so that methods are compared on the task they actually solve rather than only on adjacent-frame smoothness. Figure 2: Qualitative comparison on three scene prompts and nine uniformly sampled orbit views. SDS-style text-to-3D methods can produce clean isolated turntables, while specialized multiview methods often simplify the surrounding scene. ViewCrafter preserves strong local image quality over a short trajectory, but does not expose a comparable closed orbit. OrbitForge targets a complete scene-level canonical orbit, retaining more of the ground, background, and object-environment relationship across the full view range. 2 Positioning and Related Work We summarize the positioning of OrbitForge in Table 1 . Score-distillation text-to-3D. DreamFusion optimizes a NeRF using a pretrained text-to-image diffusion prior through probability density distillation [ 13 ] ; SJC back-propagates diffusion scores through a differentiable renderer [ 17 ] ; Magic3D improves resolution and efficiency through a coarse-to-fine mesh optimization pipeline [ 8 ] ; and later methods such as ProlificDreamer and JSDDreamer revisit the distillation objective to improve diversity and stability [ 18 , 1 ] . These methods usually keep the diffusion prior frozen, but they still perform per-prompt SDS-style 3D optimization. OrbitForge instead samples a generic video prior and reconstructs from generated observations, avoiding per-prompt SDS optimization while retaining the scene-level appearance of modern video models. Specialized multiview generation. MVDream, SV3D, VideoMV, and CoSER show that diffusion and video architectures can be adapted into strong multiview generators [ 14 , 16 , 21 , 7 ] . Their outputs are valuable for object reconstruction because the camera grid is known and the views are designed to be mutually consistent. The trade-off is specialization: such systems typically assume a fixed multiview format, an object-centered orbit, a canonical elevation, or task-specific multiview training. OrbitForge takes the opposite route. It does not fine-tune the video model into a multiview generator; it