GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing
Yuanhao Wang, Cheng Zhang, Gonçalo Frazão, Jinlong Yang, Alexandru-Eugen Ichim, [2pt] Thabo Beeler, Fernando De la Torre, [6pt] Carnegie Mellon University · 2025
Read GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing by Yuanhao Wang, Cheng Zhang, Gonçalo Frazão, Jinlong Yang, Alexandru-Eugen Ichim, [2pt] Thabo Beeler, Fernando De la Torre, [6pt] Carnegie Mellon University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08678.txt.
Introduction
Yuanhao Wang and co-authors present a tool for making three-dimensional clothes from one photo in a paper from two thousand twenty-five. Non-expert users can design garments in flat images, but turning a single view into a full garment that can be edited is hard. Views made by older methods often fail to match each other.
Their system, called GarmentCrafter, builds new views step by step. It first predicts depth and warps the image to guess new angles. A diffusion model then fills in hidden areas, guided by the changing camera view. By inferring color and depth together, the method keeps views consistent. Tests showed better detail and coherence than leading rival methods.
Catalog record
| CREATOR | Yuanhao Wang, Cheng Zhang, Gonçalo Frazão, Jinlong Yang, Alexandru-Eugen Ichim, [2pt] Thabo Beeler, Fernando De la Torre, [6pt] Carnegie Mellon University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08678
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing Yuanhao Wang, Cheng Zhang, Gonçalo Frazão, Jinlong Yang, Alexandru-Eugen Ichim, [2pt] Thabo Beeler, Fernando De la Torre, [6pt] Carnegie Mellon University Abstract We introduce GarmentCrafter, a new approach that enables non-professional users to create and modify 3D garments from a single-view image. While recent advances in image generation have facilitated 2D garment design, creating and editing 3D garments remains challenging for non-professional users. Existing methods for single-view 3D reconstruction often rely on pre-trained generative models to synthesize novel views conditioning on the reference image and camera pose, yet they lack cross-view consistency, failing to capture the internal relationships across different views. In this paper, we tackle this challenge through progressive depth prediction and image warping to approximate novel views. Subsequently, we train a multi-view diffusion model to complete occluded and unknown clothing regions, informed by the evolving camera pose. By jointly inferring RGB and depth, GarmentCrafter enforces inter-view coherence and reconstructs precise geometries and fine details. Extensive experiments demonstrate that our method achieves superior visual fidelity and inter-view coherence compared to state-of-the-art single-view 3D garment reconstruction methods. Figure 1 : From a real-world clothing image, GarmentCrafter synthesizes high-quality novel views, enabling the reconstruction of garment meshes with accurate geometry and rich detail. Additionally, users can easily apply 2D edits (e.g., modifying parts or surface details) using off-the-shelf tools on a single image, and GarmentCrafter seamlessly applies these edits across the 3D model with multi-view consistency. 1 Introduction Professional fashion designers use sophisticated software to create and edit garments in 3D, crafting highly detailed virtual apparels [ 1 , 3 , 4 , 2 ] . However, as digital garments become integral to virtual environments and personalized digital experiences [ 27 , 11 , 73 , 21 , 54 ] , there is a growing demand for intuitive tools that allow non-professional users to design and interact with 3D garments. For broader accessibility, such tools should allow users to work with 3D garments with minimal input, ideally from just a single image. This raises a key question: How can we create and edit 3D garments with simple manipulations in an image? Recent advancements in image generation models [ 53 , 51 , 55 , 66 ] and image editing techniques [ 84 , 9 , 48 , 87 , 67 , 50 ] have enabled high-quality garment design in 2D. Yet, achieving the same level of control and realism for 3D garments remains challenging for common users. Currently, state-of-the-art methods on single-view 3D garments rely either on 1) deforming, matching, and registration with the human body prior [ 43 ] and/or predefined garment templates [ 39 , 57 , 16 , 37 , 20 , 45 , 7 ] , or 2) novel view synthesis techniques [ 70 , 41 ] that use pre-trained 2D diffusion models conditioned on a reference image and target pose. However, they often fall short in capturing accurate, realistic geometry and appearance. Two characteristics of garments pose challenges. First, garments exhibit diverse shapes, complex geometries, and rich textures, making template-based methods limited in their ability to generalize across clothing styles. Most existing methods prioritize either geometry [ 44 , 16 ] or texture [ 80 , 52 ] , rarely balancing both [ 57 , 20 , 45 ] . Second, the fine details in garments demand stronger multi-view consistency. Existing novel view synthesis methods [ 74 , 42 ] , conditioned on a reference image and target pose, often neglect critical semantic connections across different views. How can we ensure that a pixel in one view corresponds to a point visible in another, with consistent appearance? In this paper, we propose a different approach, progressive novel view synthesis , to enhance cross-view coherence. Our method begins by estimating the depth of the input image and warping projected points to approximate unseen views. We then apply a multi-view diffusion model to complete missing and occluded regions based on the evolving camera pose. Furthermore, we incorporate a monocular depth estimation model to generate depth maps that remain consistent with the warped depths. Unlike existing novel view synthesis, our key insight is to use the depth-based warped image as an additional condition to guide cross-view alignment. By progressively synthesizing views and depths along a predefined camera trajectory, our method gradually refines the geometry and texture of the garment across viewpoints. We name our method GarmentCrafter , a novel solution for 3D garment creation and editing while users just need to operate on a single-view image, as shown in Figure 1 . Specifically, GarmentCrafter not only generates high-quality 3D garments but also extends garment editing from 2D to 3D. Thanks to our progressive novel view synthesis, users can make local edits (e.g., editing surface details) or perform part-based manipulations (e.g., modifying garment parts) directly on a single-view image, with precise effects reflected in 3D space — capabilities that are absent in the existing methods [ 57 ] . Trained on large-scale 3D garment datasets [ 8 , 88 , 18 ] , GarmentCrafter demonstrates superior performance on held-out 3D garment data as well as in-the-wild clothing images. Extensive experiments show that our method outperforms state-of-the-art 2D-to-3D garment reconstruction approaches in terms of geometric accuracy, visual fidelity, and cross-view consistency. Figure 2 : An illustration of progressive novel view synthesis in GarmentCrafter. Left: Given a garment image, our method performs depth-aware novel view synthesis along a predefined zigzag camera trajectory. Right: For each camera rotation from \pi_{i-1} to \pi_{i} , we project the current point cloud P_{i-1} into the image space based on camera pose \pi_{i} , resulting in incomplete RGB and depth images. Our diffusion model completes the RGB image using the warped view, input image, and camera pose as conditions, while a depth completion network refines the depth map based on the completed RGB, warped depth, and camera pose. The re-projected point cloud P^{\prime}_{i} is then merged with P_{i-1} to produce an updated point cloud P_{i} . This iterative process continues until a full 3D representation of the garment is achieved. 2 Related Work Single-View 3D Garment Reconstruction and Editing. Reconstructing 3D garments from a single image has been widely explored, with existing methods approaching the task from several perspectives. One line of work relies on parametric body templates, such as SMPL [ 7 , 16 , 47 , 29 ] , or employs 2D shape priors and keypoint-based techniques [ 83 ] to optimize garment structure. Another category of work uses explicit or implicit 3D parametric garment models [ 37 , 17 , 7 , 20 , 45 , 44 , 57 , 86 ] to capture garment shape and support pose-guided deformations. Additionally, some methods incorporate garment sewing patterns [ 6 , 28 , 76 , 14 , 12 , 39 , 88 ] , offering flexibility by reconstructing garments from 2D panels. However, these works often struggle to capture diverse garment styles and fine surface details (e.g., wrinkles), and lack support for intuitive garment manipulation, such as modifying surface details or garment parts. In contrast, GarmentCrafter prioritizes novel view synthesis for detailed geometry and texture reconstruction, without relying on garment templates or human