Investigating Anthropometric Fidelity in SAM 3D Body
Aizierjiang Aiersilan, Ruting Cheng, James Hahn · 2026
Read Investigating Anthropometric Fidelity in SAM 3D Body by Aizierjiang Aiersilan, Ruting Cheng, James Hahn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06035.txt.
Introduction
Aizierjiang Aiersilan and 2 co-authors probe a blind spot in SAM 3D Body in a 2026 evaluation paper. The model rebuilds a person's three-dimensional body mesh from a single photo. It handles blocked views and varied poses well, producing clean, coherent meshes.
Bodies that differ from average, though, lose their distinctive shape. Muscle loss in older adults, curved spines, and pregnancy were smoothed away even when clear in the photo. The authors blame a regression to the mean caused by the model's compact body representation and its training alignment. They propose hybrid representations and medical expert checks as constructive paths toward the accuracy that medical uses demand.
Catalog record
| CREATOR | Aizierjiang Aiersilan, Ruting Cheng, James Hahn |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-06035
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Investigating Anthropometric Fidelity in SAM 3D Body Aizierjiang Aiersilan, Ruting Cheng, James Hahn Abstract The recent release of SAM 3D Body 38 marks a significant milestone in human mesh recovery, demonstrating state-of-the-art performance in producing clean, topologically coherent meshes from single images. By leveraging the novel Momentum Human Rig (MHR), it achieves remarkable robustness to occlusion and diverse poses. However, our evaluation reveals a specific and consistent limitation: the model struggles to reconstruct detailed anthropometric deviations, especially on populations with special body shape alters such as geriatric muscle atrophy, scoliosis, or pregnancy, even when these features are prominent in the input image. In this paper, we investigate this phenomenon not as a failure of the model’s capacity, but as a byproduct of the perception-distortion trade-off . We posit that the architectural reliance on the low-dimensional parametric MHR representation, combined with semantic-invariant conditioning (DINOv3) and annotation-based alignment, creates a “regression to the mean” effect. We analyze these mechanisms to understand why individual biological details are smoothed out and propose specific, constructive pathways for future work to extend the impressive baseline performance of SAM 3D Body into the medical domain. 1 Introduction 3D foundation models ( 1 ) have historically trended toward universality, among which SAM 3D Body 38 continues this trend by effectively solving the “2D-to-3D lifting” problem for human subjects. Its capacity for human body generation not only facilitates human pose learning and unlocks new capabilities in robotics and interactive mixed reality applications, but also paves a promising road toward 3D body shape analysis in the medical domain. Recent clinical research has established that high-fidelity 3D anthropometry serves as a superior digital biomarker compared to traditional metrics like BMI, enabling early detection of metabolic risks, cardiovascular pathologies, and hormonal irregularities through precise morphological analysis 30 ; 9 ; 13 ; 20 . Unlike its sibling model SAM 3D Objects 29 , which targets general rigid objects, SAM 3D Body employs a specialized encoder-decoder architecture based on the Momentum Human Rig (MHR) 11 to decouple skeletal structure from surface shape, achieving a level of robustness that previous methods 36 ; 42 struggled to match. Table 1: An overview of recent 3D foundation models, landscape bifurcates into generative models (creating assets from text/images) and geometric understanding models (reconstructing scenes and motion). Model Primary Task Example Applications Key Features Generative 3D Models (Text/Image \to 3D Asset) SAM 3D Objects 29 Single-Image to Mesh Robotics, AR/VR Scalable synthetic training; occluded object recovery SAM 3D Body 38 Human Mesh Recovery Sports, Medical Analysis Momentum Human Rig (MHR); decoupled skeleton/shape TRELLIS 36 Text/Image to 3D Game Dev, VFX Structured Latent (SLAT); multi-format decode Hunyuan3D 2.0 42 High-Fidelity Generation AAA Gaming, Retail Sparse-view DiT; PBR textures MeshLLM 10 Text to Mesh CAD, Design LLM-based mesh tokens; editable topology CAT3D 12 Multi-View Generation Photorealistic Rendering Diffusion-based NVS; high consistency LGM 28 Fast Reconstruction Real-time Previews Large Geometry Model; asymmetric U-Net; outputs mesh or 3D Gaussians SF3D 4 Textured Mesh Generation E-commerce UV-unwrapping; illumination disentanglement AssetGen 2.0 26 Text/Image to 3D Asset Content Creation, Gaming Single-stage 3D diffusion; improved view consistency; optional TextureGen module Deep CAD 34 Generative CAD Geometry AEC, Manufacturing, Design Precision CAD generation; trained on pro design data; design autocompletion DreamFusion 25 Text-to-3D (NeRF) Concept Art, Prototyping Score Distillation Sampling (SDS); uses 2D diffusion models for 3D generation Geometric & Scene Understanding (Structure / Motion) DUSt3R 32 Stereo Reconstruction 3D Scanning Dense unconstrained stereo; pose-free camera estimation MASt3R 15 3D Matching & Metric Rec. Visual Localization, SfM Pixel-wise 3D correspondences; metric pointmaps; cross-screen matching MonST3R 40 Dynamic Reconstruction (4D) Autonomous Driving, Motion Capture Extends DUSt3R/MASt3R to dynamics; 4D pointmaps; no pose supervision LoRA3D 17 Foundation Adaptation Scene Fine-tuning Low-rank self-calibration for geometric foundation models MUSt3R 6 Multi-View Reconstruction Visual Odometry, Mapping Multi-view aggregation; global alignment-free; metric 3D output PanSt3R 43 3D Panoptic Segmentation Robotics, Scene Analysis Joint 3D geometry and panoptic segmentation; multi-view consistent Copilot4D 41 4D (3D + Time) Forecasting Autonomous Driving LiDAR tokenization; future state prediction; agent-conditioned forecasting However, the transition from general-purpose computer vision to medical application requires a shift in evaluation criteria—from perceptual plausibility to metric fidelity. Despite its capabilities, our internal testing uncovered a curious limitation. When tasked with reconstructing human bodies exhibiting specific medical or physiological conditions (e.g., third-trimester pregnancy, scoliosis, unilateral limb amputation, etc.), as shown in Figure 1 , the model consistently produced “standardized” body shapes. Not only were textures like skin wrinkles and clothing smoothed out, but the overall geometry was also rectified to a generic mean. The typical scoliosis shape of the spine’s sideways curve and the abdominal region of the pregnant population were neglected during 3D figure generation. Driven by these observations, we investigate why a model capable of reconstructing complex poses paradoxically struggles with organic biological variance. We argue that this behavior is a feature, rather than a bug, of current alignment strategies 7 , and explore its critical implications for medical simulation. This work serves as a conceptual critique and a roadmap for bridging the gap between artistic generation and clinical reconstruction. Figure 1: The limitations of SAM 3D Body when generating 3D figures for pregnant women (left, under license from Pressmaster - stock.adobe.com), people with scoliosis 31 (middle left), people with limb loss (middle right, under license from Kawee - stock.adobe.com), and obese people (right, under license from DenisProduction.com - stock.adobe.com). Missing biological signature of each population is marked by a red box. 2 Related Work 2.1 Parametric Human Models and Phenotypic Diversity The field of 3D human body modeling has long been dominated by SMPL 16 and its variants. While revolutionary, SMPL’s global shape space often suffers from spurious long-range correlations. To address this, STAR 22 introduced sparse, spatially-local blend shapes, and SUPR 23 further refined this with a unified part-based representation to allow for more localized control. Most recently, AnnyBody 5 has challenged the limited phenotypic range of standard models by explicitly modeling a diverse range of body shapes and medically meaningful morphs, highlighting the “standardization” bias inherent in older datasets. SAM 3D Body’s MHR, while robust, appears to regress to a similar global mean as early SMPL models, lacking the localized expressivity of SUPR or the population inclusivity of AnnyBody. 2.2 High-Fidelity and Implicit Reconstruction To capture details beyond the parametric mesh, recent approaches have diverged from pure regression. HiLo 39 explicitly separates high-frequency details (clothing, wrinkles) from low-frequency shape, ensuring that fine-grained geometry is not smoothed out. Similarly, CHROME 8 focuses on occlusion resilience and view consistency without over-relying on smooth priors. Implicit function approaches like SelfPIFu 37 abandon the mesh template entirely to learn topology-agnostic representations from in-the-wild data. In contrast, SAM 3D Body’s reliance on a fixed topology and low-dimensional MHR latent space inherently limits its ability to represent the high-frequency anomalies critical for medical diagnosis. 2.3 Biomechanical Fidelity Beyond surface shape, the underlying skeletal accuracy is crucial. HSMR 35 integrates a biomechanically accurate skeleton into the reconstruction pipeline to prevent biologically implausible joint locations. While SAM 3D Body decouples skeleton and shape for robustness, our analysis suggests it prioritizes pose plausibility over the metric accuracy required for biomechanical analysis, often “correcting” pathological skeletal deviations (e.g., scoliosis) to a healthy mean. 3 The Architectural Smoothing of