ClothHMR: 3D Mesh Recovery of Humans in Diverse Clothing from Single Image Conference: Proceedings of the 2025 International Conference on Multimedia Retrieval; June 30-July 3, 2025; Chicago, IL, USA Proceedings of the 2025 International Conference on Multimedia Retrieval (ICMR ’25), June 30-July 3,

Yunqi Gao, Leyuan Liu, Yuhan Li, Changxin Gao, Yuanyuan Liu, Jingying Chen · 2025

Read ClothHMR: 3D Mesh Recovery of Humans in Diverse Clothing from Single Image Conference: Proceedings of the 2025 International Conference on Multimedia Retrieval; June 30-July 3, 2025; Chicago, IL, USA Proceedings of the 2025 International Conference on Multimedia Retrieval (ICMR ’25), June 30-July 3, by Yunqi Gao, Leyuan Liu, Yuhan Li, Changxin Gao, Yuanyuan Liu, Jingying Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17545.txt.

Introduction

Yunqi Gao and coauthors present ClothHMR, a method that rebuilds a 3D mesh of a person from a single photo, even when loose or varied clothing hides the body shape. Standard mesh recovery tools lean on tight clothing cues, so draped garments often throw them off.

ClothHMR works in two stages. A clothing tailoring module first estimates body semantics and predicts body edges under the cloth, and a mesh recovery module built on the FHVM model then fits the full body mesh. The method beats prior state of the art systems on standard benchmarks and on photos taken in the wild. The team also built a web app on top of it for online fashion and shopping, where a shopper's body shape drives fit and size advice.

Catalog record

CREATORYunqi Gao, Leyuan Liu, Yuhan Li, Changxin Gao, Yuanyuan Liu, Jingying Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-17545

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ClothHMR: 3D Mesh Recovery of Humans in Diverse Clothing from Single Image Conference: Proceedings of the 2025 International Conference on Multimedia Retrieval; June 30-July 3, 2025; Chicago, IL, USA Proceedings of the 2025 International Conference on Multimedia Retrieval (ICMR ’25), June 30-July 3, 2025, Chicago, IL, USA DOI: 10.1145/3731715.3733288 ISBN: 979-8-4007-1877-9/2025/06 CCS: General and reference CCS: Computing methodologies Reconstruction Yunqi Gao, Leyuan Liu, Yuhan Li, Changxin Gao, Yuanyuan Liu, Jingying Chen Abstract . With 3D data rapidly emerging as an important form of multimedia information, 3D human mesh recovery technology has also advanced accordingly. However, current methods mainly focus on handling humans wearing tight clothing and perform poorly when estimating body shapes and poses under diverse clothing, especially loose garments. To this end, we make two key insights: (1) tailoring clothing to fit the human body can mitigate the adverse impact of clothing on 3D human mesh recovery, and (2) utilizing human visual information from large foundational models can enhance the generalization ability of the estimation. Based on these insights, we propose ClothHMR, to accurately recover 3D meshes of humans in diverse clothing. ClothHMR primarily consists of two modules: clothing tailoring (CT) and FHVM-based mesh recovering (MR). The CT module employs body semantic estimation and body edge prediction to tailor the clothing, ensuring it fits the body silhouette. The MR module optimizes the initial parameters of the 3D human mesh by continuously aligning the intermediate representations of the 3D mesh with those inferred from the foundational human visual model (FHVM). ClothHMR can accurately recover 3D meshes of humans wearing diverse clothing, precisely estimating their body shapes and poses. Experimental results demonstrate that ClothHMR significantly outperforms existing state-of-the-art methods across benchmark datasets and in-the-wild images. Additionally, a web application for online fashion and shopping powered by ClothHMR is developed, illustrating that ClothHMR can effectively serve real-world usage scenarios. The code and model for ClothHMR are available at: https://github.com/starVisionTeam/ClothHMR . Figure 1. 3D human meshes recovered by ClothHMR. ClothHMR can recover precise 3D meshes of humans wearing diverse clothing and in complex poses from a single image. Please zoom in to see the details. 1. Introduction The widespread application of 3D data in multimedia has driven the rapid development of 3D human mesh recovery technology. This technology provides more realistic interactive experiences in fields such as virtual fitting, online fitness, and immersive gaming. Although existing 3D human mesh recovery methods ( 45 ) have made promising progress, they still face challenges when dealing with individuals wearing diverse garments, especially loose clothing. Due to the scarcity of annotated datasets with diverse clothing, few 3D human mesh recovery methods adequately account for the impact of clothing occlusion. As shown in Fig. 2 , current SOTA methods ( 59 ; 58 ; 36 ; 13 ) often misestimate body shapes or fail to handle complex poses when dealing with individuals wearing loose clothing. Current SOTA methods may still estimate poses and shapes without sufficient precision even for individuals in non-loose clothing. Instead of recovering 3D human meshes from raw RGB images directly, most methods utilize intermediate representations such as heatmaps ( 47 ; 55 ; 41 ; 43 ; 61 ; 27 ; 11 ) , joints ( 20 ; 8 ; 59 ; 58 ; 4 ; 56 ) , silhouettes ( 41 ; 53 ; 56 ) , body segmentation ( 55 ; 38 ; 42 ) , and depth information ( 18 ) . Introducing intermediate representation distinctly contributes to overcoming data scarcity, thus guiding 3D human mesh recovery methods toward more accurate prediction ( 45 ) . However, inferring intermediate representations from raw RGB images remains an open problem that is not yet fully solved. Any failure in intermediate representation inferring may lead to 3D human mesh recovery failures. To address this, many methods ( 36 ; 54 ; 9 ; 60 ; 49 ; 28 ; 18 ) combine multiple intermediate representations. However, the intermediate representations they employed are predicted by multiple deep neural networks, and the inconsistencies among these representations can compromise the recovery of 3D human meshes. To improve the accuracy and robustness of 3D human mesh recovery, it is essential to utilize multiple intermediate representations that are both high-precision and consistent. In this paper, we propose ClothHMR to recover 3D meshes of individuals wearing diverse clothing. ClothHMR is based on two intuitive yet effective insights: (1) Tailoring clothing to fit the body can mitigate the adverse impact of clothing on 3D human mesh recovery; (2) Intermediate representations extracted through a single foundational human vision model (FHVM) are more consistent and of higher quality. To this end, ClothHMR primarily consists of two modules: clothing tailoring (CT) and FHVM-based mesh recovering (MR). Specifically, given a single RGB image, the CT module trims the clothing to fit the body, resulting in the tailored image. This module carries out body semantic estimation and body edge prediction to effectively trim excess clothing. The tailored image is then used to generate a 3D body mesh in the MR module. This module utilizes the foundational human visual model (FHVM) to extract three types of intermediate representations: joints, depth, and silhouette from the tailored image. It iteratively optimizes the 3D human mesh by minimizing the differences between the intermediate representations rendered from the estimated 3D human mesh and those extracted from the FHVM. As shown in Fig. 1 , ClothHMR can recover precise 3D meshes of humans wearing diverse clothing and in complex poses from a single image. Figure 2. Summary of SOTA. Current SOTA methods often misestimate body shapes or fail to handle complex poses when dealing with individuals wearing loose clothing. Even for individuals in non-loose clothing, current SOTA methods may still estimate poses without sufficient precision. ClothHMR can effectively handle challenges posed by diverse types of clothing and complex poses. Our contributions are summarized as follows: (1) We propose a clothing-tailoring scheme, enabling 3D human mesh recovery algorithms to tackle individuals wearing diverse clothing, especially those dressed in loose clothing. (2) We are the first to introduce a foundational human vision model into 3D human mesh recovery to enhance the precision of intermediate representations, thereby enabling the recovery of high-precision 3D human meshes using a simple iterative algorithm. (3) We propose ClothHMR, which can recover 3D meshes of individuals wearing diverse clothing, including extremely loose garments. ClothHMR significantly outperforms existing state-of-the-art methods by a substantial margin on both benchmark datasets and in-the-wild images. 2. Relate Work Figure 3. Overview. Given the image \mathcal{I} , ClothHMR first uses the clothing tailoring module to trim the clothing to fit the human silhouette, resulting in \mathcal{B} . Then, it initializes the mesh \mathcal{M}_{0} from \mathcal{B} and generates intermediate representations of \mathcal{B} using the foundational human vision model (FHVM), including joints \mathcal{G}_{j} , depth \mathcal{G}_{d} , and silhouette \mathcal{G}_{m} . The mesh recovery module aligns the intermediate representations generated by FHVM with those produced by the mesh model ( \mathcal{P}_{j} , \mathcal{P}_{d} , and \mathcal{P}_{m} ), and through iterative optimization, ultimately generates an accurate human mesh \mathcal{M}_{r} . 2.1. 3D Human Mesh Recovery 3D human mesh recovery is represented using parametric human models, including the SCAPE model ( 1 ) , the SMPL model ( 35

Read the complete text