Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset
Clémentine Grethen, Simone Gasparini, Géraldine Morin, Jérémy Lebreton, Lucas Marti, Manuel Sanchez Gestido · 2025
Read Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset by Clémentine Grethen, Simone Gasparini, Géraldine Morin, Jérémy Lebreton, Lucas Marti, Manuel Sanchez Gestido (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18172.txt.
Introduction
Clementine Grethen, with five co-authors, adapts stereo vision to rebuild the Moon's surface in three dimensions. Lunar ground lacks texture, lighting swings hard, and orbit paths are unusual, so Earth-trained depth models fail there. The team built a photorealistic simulated stereo data set of the lunar south pole from detailed height and reflectance maps.
Fine-tuning a leading stereo model on that data moves it into the lunar domain. Varied heights, lighting, and viewing angles give grounded training signal. Space mission readers get both an open resource and proof that simulated Moon data can ready vision systems for real landings.
Catalog record
| CREATOR | Clémentine Grethen, Simone Gasparini, Géraldine Morin, Jérémy Lebreton, Lucas Marti, Manuel Sanchez Gestido |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18172
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset Clémentine Grethen, Simone Gasparini, Géraldine Morin, Jérémy Lebreton, Lucas Marti, Manuel Sanchez Gestido Abstract Accurate 3D reconstruction of lunar surfaces is essential for space exploration. However, existing stereo vision reconstruction methods struggle in this context due to the Moon’s lack of texture, difficult lighting variations, and atypical orbital trajectories. State-of-the-art deep learning models, trained on human-scale datasets, have rarely been tested on planetary imagery and cannot be transferred directly to lunar conditions. To address this issue, we introduce LunarStereo , the first open dataset of photorealistic stereo image pairs of the Moon, simulated using ray tracing based on high-resolution topography and reflectance models. It covers diverse altitudes, lighting conditions, and viewing angles around the lunar South Pole, offering physically grounded supervision for 3D reconstruction tasks. Based on this dataset, we adapt the MASt3R model to the lunar domain through fine-tuning on LunarStereo. We validate our approach through extensive qualitative and quantitative experiments on both synthetic and real lunar data, evaluating 3D surface reconstruction and relative pose estimation. Extensive experiments on synthetic and real lunar data validate the approach, demonstrating significant improvements over zero-shot baselines and paving the way for robust cross-scale generalization in extraterrestrial environments. Figure 1 : Top row : examples of lunar stereo image pairs of our dataset taken from three different trajectories, along with the corresponding ground-truth depth maps (viridis colormap) and 3D scene renderings. Middle row : for each view, it shows the ground truth for the depth map (viridis colormap) and the slope map (heat colormap), followed by the corresponding predictions of our method and the MASt3R baseline. Bottom row : the ground truth 3D scene (left) with the final 3D reconstructions for our method (center) and MASt3R (right), with View 1 in red and View 2 in green . 1 Introduction & Background Accurate 3D reconstruction of planetary surfaces is critically needed in space exploration missions, particularly for descent and landing, to perform Hazard Detection and Avoidance (HDA), trajectory planning, and autonomous navigation. For the Moon, the lack of an absolute Global Navigation Satellite System (GNSS), sparse visual features, and challenging lighting conditions make perception tasks very challenging. Thus, reliable and dense surface reconstruction is essential for landing a spacecraft safely on the Moon. In addition, reconstructing 3D models from images acquired during past or ongoing missions is key to building accurate topographic maps, refining prior knowledge of the terrain, and mission planning. Reconstruction during a landing sequence usually relies on classical computer vision pipelines, combining Structure-from-Motion (SfM) for sparse pose estimation with Multi-View Stereo (MVS) for dense depth reconstruction [ 17 , 13 ] . However, these methods are not well-suited to the constraints of spaceborne imagery. Their effectiveness is often limited in environments characterized by repetitive textures, low albedo variation, minimal stereo baseline, and strong illumination contrasts — all common in lunar imagery. Indeed, lunar imagery presents significant challenges. The surface is visually sparse, highly repetitive, and exhibits fractal-like patterns, with minimal color variation due to the absence of an atmosphere, resulting in low contrast at high altitude and significant difficulties for visual interpretation and image-based algorithms [ 13 , 29 , 16 ] . Furthermore, the descent trajectories are often near-nadir, thus introducing a degenerate configuration for monocular SfM due to the lack of lateral parallax [ 35 ] . Shadows and lighting gradients further affect feature detection and 3D estimation. As recent autonomous landing failures have shown [ 11 ] , the ability to perceive and understand the terrain in such conditions remains a critical open problem. In recent years, deep learning-based approaches have emerged as a means of overcoming some of the limitations of classical 3D reconstruction pipelines. These include learned feature matchers such as SuperGlue [ 32 ] , end-to-end stereo networks such as MVSNet [ 44 ] . More recently, unified architectures such as MASt3R [ 20 ] , DUSt3R [ 38 ] , and VGGT [ 37 ] has been introduced for the 3D reconstruction from general images: trained on large-scale terrestrial datasets like MegaDepth [ 22 ] and Co3Dv2 [ 30 ] , these models achieve state-of-the-art performance on human-scale imagery — including urban scenes, natural landscapes, indoor environments, and common objects. However, their effectiveness remains largely limited to the types of scenes represented in their training data. When applied to out-of-domain settings such as lunar imagery, their performance degrades significantly [ 36 ] . For example, applying MASt3R to raw lunar images often results in unreliable geometry, with flat reconstructions, inconsistent relief, or noisy outputs, as illustrated in Fig. 1 . This highlights a clear domain gap: deep models trained on Earth-like content do not generalize well to the low-texture, high-altitude, and structured viewpoints found in space applications — unless carefully adapted. To address this limitation, we introduce the first publicly available physically realistic lunar stereo dataset, LunarStereo , tailored for deep learning-based 3D reconstruction. Our dataset includes simulated stereo pairs generated through ray tracing over high-resolution Digital Elevation Models (DEMs), using accurate reflectance models (BRDFs), realistic Sun illumination, and varied camera trajectories. This enables us to provide dense ground-truth depth maps and accurate camera poses under a wide range of imaging conditions, including nadir and oblique views, low altitudes, and challenging illumination, closely mimicking real descent scenarios ( cf . first row of Fig. 1 for different samples of the dataset). We then used this dataset to fine-tune the MASt3R model for lunar stereo vision. We chose MASt3R due to its ability to output 3D correspondences, its superior performance compared to DUST3R, and its significantly lighter architecture compared to VGGT. Our fine-tuned version shows significant improvements in reliably reconstructing the surface and, in particular, the geometry of the reliefs, as can be observed in Fig. 1 . These findings demonstrate not only the potential of MASt3R in space imaging but also, more broadly, the adaptability of modern 3D vision networks to low-texture and out-of-distribution domains through targeted fine-tuning. The contributions of this work are twofold: (i) We release the first publicly available, high-fidelity, lunar stereo dataset with full geometric supervision, simulated from DEMs under physically-based rendering. 1 1 1 Data and code: https://clementinegrethen.github.io/publications/3D-Vast-ICCV2025.html While high-frequency texture details are not modeled, this enables wide coverage and flexible illumination control beyond real conditions at the South Pole. (ii) We fine-tune the MASt3R model on this dataset, demonstrating its successful adaptation to the lunar domain with dramatically improved 3D geometry estimation across all scenarios, achieving an average reduction of over 70\text{\,}\mathrm{\%} in slope estimation error and significantly enhancing overall relative accuracy by roughly 50\text{\,}\mathrm{\%} . The rest of the paper is organized as follows: Section 2 reviews prior work on lunar datasets and 3D reconstruction methods. Section 3 details our dataset creation pipeline. Section 4 describes the reconstruction and fine-tuning methodology. Section 5 presents our evaluation and results. Finally, Section 6 discusses limitations and future directions. 2 Related work 2.1 Lunar images datasets To evaluate and adapt deep learning–based 3D reconstruction methods to the lunar environment, we require datasets that combine imagery with accurate geometric supervision, stereo pairs or multi-view images with consistent camera metadata, dense geometry, and sufficient diversity in lighting,