Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image

Ming Qian, Zimin Xia, Changkun Liu, Shuailei Ma, Wen Wang, Zeran Ke, Bin Tan, Hang Zhang, Gui-Song Xia, [0.5ex] LIESMARS & School of Artificial Intelligence, Wuhan University EPFL · 2026

Read Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image by Ming Qian, Zimin Xia, Changkun Liu, Shuailei Ma, Wen Wang, Zeran Ke, Bin Tan, Hang Zhang, Gui-Song Xia, [0.5ex] LIESMARS & School of Artificial Intelligence, Wuhan University EPFL (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14984.txt.

Introduction

Ming Qian and eight colleagues introduce Sat3DGen, which builds street-level 3D scenes from a single satellite image. Geometry-colorization methods stay building-focused, while feed-forward models yield rich but coarse geometry. The team puts geometry first, adding geometric constraints and perspective-view training to fight the viewpoint gap. On a new benchmark pairing VIGOR-OOD with elevation data, geometric error fell from 6.76 meters to 5.20 meters, and image realism improved sharply too.

Catalog record

CREATORMing Qian, Zimin Xia, Changkun Liu, Shuailei Ma, Wen Wang, Zeran Ke, Bin Tan, Hang Zhang, Gui-Song Xia, [0.5ex] LIESMARS & School of Artificial Intelligence, Wuhan University EPFL
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56481
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14984

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image Ming Qian, Zimin Xia, Changkun Liu, Shuailei Ma, Wen Wang, Zeran Ke, Bin Tan, Hang Zhang, Gui-Song Xia, [0.5ex] LIESMARS & School of Artificial Intelligence, Wuhan University EPFL Abstract Generating a street-level 3D scene from a single satellite image is a crucial yet challenging task. Current methods present a stark trade-off: geometry-colorization models achieve high geometric fidelity but are typically building-focused and lack semantic diversity. In contrast, proxy-based models use feed-forward image-to-3D frameworks to generate holistic scenes by jointly learning geometry and texture, a process that yields rich content but coarse and unstable geometry. We attribute these geometric failures to the extreme viewpoint gap and sparse, inconsistent supervision inherent in satellite-to-street data. We introduce Sat3DGen to address these fundamental challenges, which embodies a geometry-first methodology. This methodology enhances the feed-forward paradigm by integrating novel geometric constraints with a perspective-view training strategy, explicitly countering the primary sources of geometric error. This geometry-centric strategy yields a dramatic leap in both 3D accuracy and photorealism. For validation, we first constructed a new benchmark by pairing the VIGOR-OOD test set with high-resolution DSM data. On this benchmark, our method improves geometric RMSE from 6.76m to 5.20m. Crucially, this geometric leap also boosts photorealism, reducing the Fréchet Inception Distance (FID) from ∼ \sim 40 to 19 against the leading method, Sat2Density++, despite using no extra tailored image-quality modules. We demonstrate the versatility of our high-quality 3D assets through diverse downstream applications, including semantic-map-to-3D synthesis, multi-camera video generation, large-scale meshing, and unsupervised single-image Digital Surface Model (DSM) estimation. The code has been released on https://github.com/qianmingduowan/Sat3DGen . 1 Introduction Street-level 3D scenes are useful for mapping, robotics, simulation, and media creation ( 30 ; 29 ; 32 ; 42 ; 24 ; 13 ) . Ground-level capture is costly and uneven across regions ( 1 ) , whereas satellite imagery offers wide coverage, low cost, and frequent updates ( 3 ) . These characteristics motivate the generation of street-level 3D from overhead satellite images for large-scale, long-term applications. Our goal is to generate a 3D scene that faithfully preserves the semantics and appearance of an input satellite image and that can be rendered for street‑view images and videos. Existing methods for generating 3D from a single satellite image fall into two categories: 3D geometry colorization ( 9 ; 14 ) and 3D proxy for image rendering ( 20 ; 19 ) . 3D geometry colorization follows a two-stage pipeline to predict and then texture 3D building geometry. While producing clean building models, these methods fail to capture non-building elements (e.g., zebra crossings, trees), resulting in outputs weakly consistent with the input satellite image ( Fig. 1 (a,b)). 1 1 1 As of the submission deadline, the official implementations of Sat2Scene and Sat2City had not been fully released. We therefore report the 3D results shown in their papers and project pages. Extending them beyond buildings would require fine-grained geometry labels for many classes, which are scarce. 3D proxy for image rendering uses tailored feed-forward image-to-3D frameworks ( 8 ; 31 ; 35 ; 38 ) to learn a coarse, differentiable 3D proxy via joint optimization under 2D supervision. These methods are semantically faithful but yield poor geometry: boundaries are degraded, roofs are unrealistic, and floating artifacts are common ( Fig. 1 (c)). Our goal requires preserving the rich semantics of the input satellite image, making the proxy-based paradigm more suitable than geometry colorization. Encouragingly, recent object-level feed-forward image-to-3D works (e.g., InstantMesh ( 33 ) , LRM ( 8 ) ) have demonstrated that high-quality 3D can be learned from 2D supervision alone. This suggests that the poor geometry of existing scene-level proxy methods is not a fundamental flaw of the paradigm. Instead, we hypothesize it stems from insufficient geometric constraints to handle the unique challenges of outdoor scenes. Specifically, the supervision from only one satellite patch and a few ground-level panoramas is extremely sparse. This sparsity, coupled with the extreme viewpoint gap, leaves rooftop geometry underconstrained and induces artifacts like holes and floaters on vertical facades. Additionally, a footprint mismatch between the satellite and street views often destabilizes the geometry at scene boundaries. To solve these specific problems, we propose Sat3DGen, which embodies a holistic, geometry-first methodology. Our strategy is not to invent a new feedforward image-to-3D architecture from scratch, but to elevate a general framework by demonstrating how to effectively solve its core geometric failures. To enforce plausible vertical structures and suppress floating artifacts, we introduce a Gravity-based Density Variation Loss . To address boundary errors stemming from the footprint mismatch, a Spatial Token regularizes peripheral layouts. To resolve rooftop ambiguity, a Monocular Relative-Depth Prior constrains satellite-view depth. Furthermore, to mitigate the issue of sparse supervision, we employ Perspective View Training , jointly training on panoramas and their projected views to increase effective viewpoint coverage and photometric consistency. Image Satellite More than Supervision Synthetic Geometry Renderable Faithful Buildings Signal or Real Quality Sat2City ✗ ✗ ✗ 3D Synthetic ✓ Sat2Scene ✓ ✗ ✗ Imgae Real ✓ Sat2Density++ ✓ ✓ ✓ Imgae Real ✗ Ours ✓ ✓ ✓ Imgae Real ✓ Figure 1: Comparison of 3D scene generation methods (top: attribute table; bottom: visual results). Given an input satellite image, (a) Sat2Scene and (b) Sat2City generate only shells of buildings and roads and miss non‑building semantics; (c) Sat2Density++ and (d) Ours are faithful to satellite semantics and appearance, but Sat2Density++ is heavily distorted, whereas our Sat3DGen yields a more structured, higher‑quality 3D representation. In evaluation, this emphasis on geometry translates directly to substantial quantitative and perceptual gains. We first validate our geometric improvements against the leading method, Sat2Density++, on a new benchmark we constructed by pairing the VIGOR-OOD test set with 1-meter resolution DSM data. Sat3DGen achieves a geometric RMSE of 5.20m, a significant reduction from Sat2Density++’s 6.76m. Crucially, this leap in 3D accuracy directly fuels a dramatic improvement in photorealism. Even though it includes no components tailored to image quality, our framework reduces the Fréchet Inception Distance (FID) on the VIGOR-OOD unseen-city split from Sat2Density++’s \sim 40 to 19. The resulting assets support diverse downstream applications such as semantic-map-to-3D synthesis, surround-view video from satellite imagery, large-area mesh generation, and single-image Digital Surface Model (DSM) generation without ground-truth depth supervision. 2 Related Works Feed-Forward Image to 3D Works has gained popularity for producing high-quality 3D assets. Recently, large reconstruction models ( 8 ; 28 ; 33 ; 31 ) have focused on generating object-level 3D assets, leveraging larger datasets, more refined annotations, and more substantial models to improve the quality of the generated assets, achieving impressive results. However, existing works primarily focus on object-level generation, presenting additional challenges when applying these models in outdoor scenes. In our work, we focus on generating high-quality, comprehensive street-level 3D from a single input satellite image, thereby naturally enhancing the quality of generated videos and supporting various applications. Single Satellite to Street-view Synthesis. Early studies generate individual street‑view images from a single satellite patch ( 22 ; 23 ; 29 ; 24 ; 16 ; 27 ) , but they do not produce usable 3D or multi‑view consistency. Later works synthesize street‑view videos by learning a colored 3D asset from the satellite

Read the complete text