FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images

Rong Wang, Fabian Prada, Ziyan Wang, Zhongshi Jiang, Chengxiang Yin, Junxuan Li, Shunsuke Saito, Igor Santesteban, Javier Romero, Rohan Joshi, Hongdong Li, Jason Saragih, Yaser Sheikh, Australian National University · 2025

Read FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images by Rong Wang, Fabian Prada, Ziyan Wang, Zhongshi Jiang, Chengxiang Yin, Junxuan Li, Shunsuke Saito, Igor Santesteban, Javier Romero, Rohan Joshi, Hongdong Li, Jason Saragih, Yaser Sheikh, Australian National University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19207.txt.

Introduction

Wang and co-authors present FRESA, a method that builds a personal, animatable 3D avatar of a person from just a few photos. Past methods often need hours of per person tuning, which limits real use. Their 2025 method learns a shared prior from more than a thousand clothed people, so new avatars come out at once.

Instead of borrowing standard skin weights, the system infers each person's shape, skin weights, and pose linked bending together. A canonicalization step aligns poses and gives a clean starting point for fine detail. Features from several frames are merged to cut artifacts and keep identity. Tests show more faithful rebuilds and motion than prior top methods, even from casual phone photos.

Catalog record

CREATORRong Wang, Fabian Prada, Ziyan Wang, Zhongshi Jiang, Chengxiang Yin, Junxuan Li, Shunsuke Saito, Igor Santesteban, Javier Romero, Rohan Joshi, Hongdong Li, Jason Saragih, Yaser Sheikh, Australian National University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-19207

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images Rong Wang, Fabian Prada, Ziyan Wang, Zhongshi Jiang, Chengxiang Yin, Junxuan Li, Shunsuke Saito, Igor Santesteban, Javier Romero, Rohan Joshi, Hongdong Li, Jason Saragih, Yaser Sheikh, Australian National University Abstract We present a novel method for reconstructing personalized 3D human avatars with realistic animation from only a few images. Due to the large variations in body shapes, poses, and cloth types, existing methods mostly require hours of per-subject optimization during inference, which limits their practical applications. In contrast, we learn a universal prior from over a thousand clothed humans to achieve instant feedforward generation and zero-shot generalization. Specifically, instead of rigging the avatar with shared skinning weights, we jointly infer personalized avatar shape, skinning weights, and pose-dependent deformations, which effectively improves overall geometric fidelity and reduces deformation artifacts. Moreover, to normalize pose variations and resolve coupled ambiguity between canonical shapes and skinning weights, we design a 3D canonicalization process to produce pixel-aligned initial conditions, which helps to reconstruct fine-grained geometric details. We then propose a multi-frame feature aggregation to robustly reduce artifacts introduced in canonicalization and fuse a plausible avatar preserving person-specific identities. Finally, we train the model in an end-to-end framework on a large-scale capture dataset, which contains diverse human subjects paired with high-quality 3D scans. Extensive experiments show that our method generates more authentic reconstruction and animation than state-of-the-arts, and can be directly generalized to inputs from casually taken phone photos. Project page and code is available at https://github.com/rongakowang/FRESA . Figure 1 : FRESA. We present a novel method to reconstruct personalized skinned avatars with realistic pose-dependent animation all in a feed-forward approach, which generalizes to causally taken phone photos without any fine-tuning. We visualize the predicted skinning weights associated with the most important joints in (b) and colormaps of per-vertex displacement magnitudes 2 2 2 Scales normalized across all vertices to highlight large deformation. during animation in (c). * * footnotetext: Work done while Rong Wang was an intern at Reality Labs Research. 1 Introduction 3D digitization for clothed humans is highly demanded in many vision and graphics applications, such as extended reality (XR), virtual try-on, and telepresence [ 27 ] . Since humans vary largely in body shapes and cloth types, the reconstructed avatars need to faithfully preserve these personalized details, and can be realistically animated robust to the underlying topology. To this end, traditional pipelines often require laborious artist designs [ 14 , 54 ] or controlled environments [ 16 , 39 ] , which limits their use. In this work, we aim to automatically reconstruct animatable avatars from easily accessible sources, e.g. casually taken phone photos, to better facilitate downstream applications. Existing works in 3D clothed human reconstructions [ 59 , 60 , 47 , 48 , 66 , 67 ] have demonstrated impressive quality and generalizability. While they mostly focus on reconstructing a static frame for a given pose, [ 20 , 17 , 31 ] further extend to produce avatars more suitable for animation. This is achieved by reconstructing the avatar in a canonical space associated with a rigged body template, and then animating it by target joint motions via the linear blend skinning (LBS) [ 37 ] method. However, all these works skin the avatar by querying the skinning weights from nearest template body vertices, which inevitably produces deformation artifacts in challenging poses and extreme body shapes. Recently, [ 49 , 12 , 62 , 56 ] propose to jointly optimize personalized canonical shapes and skinning weights from posed 3D scans or images. However, due to the lack of a unified prior from different body shapes and cloth types [ 55 ] , they are restricted to a per-subject setting and require hours of test-time optimization, which is computationally expensive. In this paper, we present a novel method to reconstruct personalized skinned avatars for realistic animation in an efficient feed-forward approach. The key to our method is a universal clothed human model, which is learned from a large collection of human subjects while capable of jointly inferring personalized canonical avatar shapes, skinning weights, and pose-dependent deformation in animation. Specifically, to normalize pose variations in input images and ensure model generalizability, we first develop a 3D canonicalization process to produce pixel-aligned initial conditions for geometry and semantics references, which helps to reconstruct canonical avatars with fine-grained geometric details. While the canonicalization can produce artifacts due to noisy unposing, we propose to robustly reduce the artifacts and generate a plausible avatar by aggregating from multi-frame references. In this way, the fused feature can preserve information intrinsic to the person, e.g. body shapes and cloth types, to better represent the person identity. Finally, to resolve the coupled ambiguity between the canonical geometry and skinning weights, i.e. to avoid generating incorrect components that can be accidentally warped to a correct posed shape [ 32 ] , we adopt a multi-stage training process to jointly supervise the model with posed-space ground truths and canonical-space regularization. Leveraging the rich prior from the universal model, we achieve feed-forward reconstruction of avatars and animation without requiring expensive per-subject optimization. Our key contributions can be summarized as follows. ( i ) We present a novel method FRESA , which enables instant feed-forward reconstruction of 3D human avatars with personalized canonical shapes, skinning weights, and pose-dependent animation via a universal clothed human model, while achieving zero-shot generalization to inputs of only a few casually taken photos. ( ii ) We propose a novel pipeline with explicit canonicalization and multi-frame aggregation, which robustly improves the fidelity and realism of resulting avatars. ( iii ) We develop a large-scale clothed human dataset with diverse subjects and high-quality 3D scans to facilitate learning an effective universal prior. Extensive experiments show that our method outperforms state-of-the-arts in both reconstruction fidelity and animation quality. 2 Related Works 3D Clothed Human Reconstruction. Early works of 3D human reconstruction often utilize parametric human models [ 34 , 42 ] as a prior for the underlying body shapes. For instance, [ 36 , 5 ] adopt a SMPL+D model to express cloth geometry as per-vertex displacements of the SMPL [ 34 ] body template, while [ 26 , 3 ] use UV maps to further improve the mesh resolutions. However, these approaches assume a fixed topology and can not model loose clothes with large shape variation to the body template. In contrast, [ 47 , 48 , 59 , 60 , 4 ] propose to leverage neural implicit fields [ 40 , 38 ] for continuous surface representation, which can flexibly model diverse body shapes and clothes types. Such representation also supports fast mesh extraction via Marching Cubes [ 35 ] to enable explicit geometry supervision and real-time rendering on mobile devices, which are difficult to achieve with other representations such as neural radiance field [ 57 , 18 , 68 ] or 3D primitives [ 30 , 69 , 25 ] as they require more time-demanding rendering pipelines. However, above methods mostly focus on reconstructing static humans for the given poses, while lacking dedicated canonicalization and skinning for reconstructed avatars, thus are not suitable for animation-based applications [ 56 ] . Figure 2 : Method Overview. We propose

Read the complete text