NGD: Neural Gradient Based Deformation for Monocular Garment Reconstruction

Soham Dasgupta, Shanthika Naik, Preet Savalia, Sujay Kumar Ingle, Avinash Sharma · 2025

Read NGD: Neural Gradient Based Deformation for Monocular Garment Reconstruction by Soham Dasgupta, Shanthika Naik, Preet Savalia, Sujay Kumar Ingle, Avinash Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17712.txt.

Introduction

Soham Dasgupta and co-authors present NGD, a method that rebuilds moving garments in 3D from a single camera video. Clothing is hard to capture because it swings, folds, and hides parts of itself. Volume-based neural methods tend to smooth away fine detail, while template methods that push mesh points around leave artifacts. NGD instead deforms the surface through learned gradient fields, a change aimed at cleaner geometry.

An adaptive remeshing step adds surface detail where wrinkles and pleats form, as on a skirt in motion. The system also learns texture maps that change over time, catching shifting light and shadow on the cloth. In side-by-side tests and numeric scores, the method beat the best existing approaches. The authors offer it as a step toward faithful capture of loose clothing from ordinary video.

Catalog record

CREATORSoham Dasgupta, Shanthika Naik, Preet Savalia, Sujay Kumar Ingle, Avinash Sharma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-17712

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NGD: Neural Gradient Based Deformation for Monocular Garment Reconstruction Soham Dasgupta, Shanthika Naik, Preet Savalia, Sujay Kumar Ingle, Avinash Sharma Abstract Dynamic garment reconstruction from monocular video is an important yet challenging task due to the complex dynamics and unconstrained nature of the garments. Recent advancements in neural rendering have enabled high-quality geometric reconstruction with image/video supervision. However, implicit representation methods that use volume rendering often provide smooth geometry and fail to model high-frequency details. While template reconstruction methods model explicit geometry, they use vertex displacement for deformation which results in artifacts. Addressing these limitations, we propose NGD, a N eural G radient-based D eformation method to reconstruct dynamically evolving textured garments from monocular videos. Additionally, we propose a novel adaptive remeshing strategy for modeling dynamically evolving surfaces like wrinkles and pleats of the skirt, leading to high-quality reconstruction. Finally, we learn dynamic texture maps to capture per-frame lighting and shadow effects. We provide extensive qualitative and quantitative evaluations to demonstrate significant improvements over existing SOTA methods and provide high-quality garment reconstructions. Figure 1 : Our method reconstructs high-fidelity garment geometry and appearance from input monocular video. 1 Introduction Recent advances in computer vision have enabled large-scale digitization of 3D garments for immersive AR/VR platforms, revolutionizing Social Media, E-Commerce, Gaming, and Entertainment industries. The sheer diversity, complex dynamics, and intricate articulations make garment digitization and modeling significantly challenging. Unlike conventional digital garment creation methods involving artists, which demanded expertise, time, and labor, deep learning has enabled garment digitization from images and videos [ 35 , 48 , 29 , 38 , 34 ] . Multi-view video inputs are used to obtain high-quality garment digitization, but they often require expensive calibrated multi-camera setups [ 51 , 35 , 48 , 28 , 19 ] , and hence difficult to scale. In comparison, monocular video inputs are easy to acquire and scalable with an abundance of “in the wild” videos available. Nevertheless, garment digitization from monocular video needs to reconstruct the dynamically evolving garment geometry and appearance while addressing the classical challenges like modeling varying garment sizes, non-rigid deformations due to body shapes and poses, and the diverse topology of garments. Advancements in differentiable rendering have made it possible to achieve high-quality geometry reconstruction from monocular videos. [ 35 , 9 , 34 , 27 , 14 , 4 ] . The existing approaches for garment reconstruction can be divided into implicit surface deformation methods [ 9 , 34 ] and explicit template deformation methods [ 27 , 4 ] . SCARF [ 9 ] is one of the first works to use implicit surface representation using Neural Radiance Fields (NeRF) [ 31 ] ; nevertheless, the geometric quality is limited by constraints inherent to volume rendering approaches. REC-MV [ 34 ] addresses this limitation by optimizing for both explicit feature curves and implicit garment surfaces. However, the use of implicit representations adds an overhead of surface extraction and the resulting surface is smooth, losing out high-fidelity surface details. Pergamo [ 4 ] and DGarments [ 27 ] deform garment templates using vertex displacements, with deformation supervised by differentiable rendering and SMPL-interpolated skinning weights. However, these methods rely on a fixed template constraining its ability to model dynamically varying topology, like the pleats of the skirt. Additionally, direct vertex displacement via differentiable rendering causes abrupt, sharp local changes and requires additional regularisers for smoothening undesirable local deformations. This often results in an over-smoothed surface and fails to capture high-frequency details. To address the above limitations, we develop a Neural Gradient Based Deformation method to reconstruct dynamic garments from input monocular video. Our method models appearance and geometry separately, as learning them together might result in appearance being corrected to compensate for geometric inaccuracies and vice versa. We propose a novel deformation parameterization that decomposes surface deformations into a frame-invariant component representing the global shape and a frame-dependent component modeling the pose-specific local surface deformations of the base garment mesh. Specifically, our deformation parameterization adopts NJF [ 1 ] to model garment reconstruction, which learns a local Jacobian field defined on the garment surface, followed by a Poisson solve to predict the global garment deformation in canonical space. This addresses the aforementioned limitation of existing template-based methods, which directly model deformation at the vertex level, often resulting in jagged surfaces when applied without any surface smoothness regularizer [ 20 ] . These canonical garments are skinned to model garment reconstruction to map to the corresponding input monocular views and optimized via a differentiable renderer [ 23 ] . While there are existing approaches that leverage NJF [ 1 ] to model deformation with a differentiable renderer, [ 10 ] , we develop a gradient-based deformation approach to model temporal deformations from the monocular input video. Unlike existing methods that directly optimize with colored images, which can result in inaccuracies due to ambiguities between shadows and textures, we use diffuse garment images . Additionally, we design an adaptive remeshing strategy to iteratively increase the mesh resolution in the regions of high-frequency geometrical details. This enables regions with fine details to be modeled by higher mesh resolutions and also freely deform the template to model extremely loose garments. Finally, we learn appearance via dynamic texture maps at each frame to capture lighting and shadow effects. Figure 1 visualizes the high-fidelity dynamic textured garment reconstructed by our method from an input monocular video. In summary, our key technical contributions are as follows: • We propose a novel method to reconstruct dynamically evolving textured garments from monocular videos. • Our novel deformation parameterization, combined with the novel adaptive remeshing, enables modeling extremely loose garments with high-frequency details. • We provide qualitative and quantitative comparisons with existing methods to show significant improvements, especially on loose garments. Code: https://github.com/astonishingwolf/NGD/ . Figure 2 : Method overview: Given an input video, we reconstruct dynamically evolving textured garment meshes using our Geometry and Appearance Reconstruction module. Figure 3 : In Geometry Reconstruction Module we introduce a novel deformation parameterization to deform a base mesh M^{B} to desired target mesh via learning a Jacobian Field guided by differentiable rendering supervision from input monocular video. 2 Related Works A large number of existing methods attempted clothed human reconstruction from single or multi-view images [ 36 , 49 , 44 , 17 , 15 , 16 , 47 ] or videos [ 3 , 42 , 43 , 37 , 33 , 19 , 12 , 13 ] , albeit cannot extract garment mesh separately. On the other hand, several existing garment reconstruction methods [ 18 , 50 , 32 , 5 , 25 , 30 , 29 , 26 , 6 , 7 ] recover garments from monocular image. However, these single-image reconstruction methods require supervised training on a large dataset. Please refer to the supplementary for a detailed discussion of these methods. Multiview images can recover garments in a self-supervised manner. Diffavatar [ 28 ] uses sewing patterns to represent garments and obtain simulation-ready garments from multiview images. Gaussian Garments [ 35 ] combines physics simulation with Gaussian splats [ 21 ] to obtain physically plausible garments from multiview inputs. The rendering captures fine details down to the level of furs. While multiview

Read the complete text