Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance

Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Stefanos Zafeiriou · 2025

Read Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance by Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Stefanos Zafeiriou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05379.txt.

Introduction

Gerogiannis and co-authors present Arc2Avatar, a method that builds an expressive three-dimensional avatar head from a single photo. Past methods needed many views of a person to build a good avatar. New large models trained on human faces, though, carry strong knowledge of what heads look like from any angle.

Arc2Avatar fine-tunes such a face model on synthetic data so it can imagine the head from new viewpoints. It then shapes a fast three-dimensional representation that stays tied to a standard face mesh. That tie lets the avatar change expression the way digital puppets do, through blend shapes. Extra steps clean up the fit. The result is a lifelike avatar from one image, ready for games and virtual meetings.

Catalog record

CREATORDimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Stefanos Zafeiriou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Stefanos Zafeiriou Abstract Inspired by the effectiveness of 3D Gaussian Splatting (3DGS) in reconstructing detailed 3D scenes within multi-view setups and the emergence of large 2D human foundation models, we introduce Arc2Avatar, the first SDS-based method utilizing a human face foundation model as guidance with just a single image as input. To achieve that, we extend such a model for diverse-view human head generation by fine-tuning on synthetic data and modifying its conditioning. Our avatars maintain a dense correspondence with a human face mesh template, allowing blendshape-based expression generation. This is achieved through a modified 3DGS approach, connectivity regularizers, and a strategic initialization tailored for our task. Additionally, we propose an optional efficient SDS-based correction step to refine the blendshape expressions, enhancing realism and diversity. Experiments demonstrate that Arc2Avatar achieves state-of-the-art realism and identity preservation, effectively addressing color issues by allowing the use of very low guidance, enabled by our strong identity prior and initialization strategy, without compromising detail. Please visit our project page for more resources. 1 Introduction Recent advances in 3D representations have significantly transformed the field of 3D avatar generation. For many years, 3D Morphable Models (3DMMs) [ 5 , 6 , 40 , 57 , 58 ] have been the standard solution, offering precise control over avatars through manipulation of the underlying meshes and a decent degree of identity representation through PCA. However, 3DMMs often struggle to capture fine details, such as the complexity of hair, limiting their ability to represent the entirety of the head. The emergence of Neural Radiance Fields (NeRFs) [ 51 ] , opened new possibilities by implicitly modeling complex geometries and textures present in human heads [ 34 , 32 , 18 , 89 ] via MLPs, however their representation is computationally intensive and difficult to manipulate. Recently, 3D Gaussian Splatting (3DGS) [ 33 ] has emerged as a groundbreaking solution for 3D scene representation. Characterized by real-time performance and enhanced detail capturing abilities, 3DGS has been applied to represent 3D head avatars. Many methods aim to leverage the controllability of 3DMMs by embedding them into the splatting process [ 61 , 86 , 68 , 84 ] , allowing flexible control over the representation. Although these approaches have shown impressive results, they rely on multi-view input data of a subject, which limits their applicability compared to single-image reconstruction approaches. Parallel to these developments, the field of generative modeling has witnessed significant advancements, especially with Latent Diffusion Models [ 28 , 64 ] . Large foundation text-to-image models have been proposed, that can additionally serve as robust priors for various tasks. However, their utilization in the 3D space is challenging. The seminal work of [ 59 ] bridged this gap by introducing Score Distillation Sampling (SDS), allowing 3D assets to be optimized to match textual descriptions. Since then, SDS has become a potent approach in conditional 3D asset generation, primarily from text, and has been applied to avatar generation as well [ 31 , 25 , 45 , 11 , 94 , 44 , 8 ] . Some of these methods even extend to animatable avatars but often suffer from low fidelity [ 44 , 94 ] . A major issue shared among SDS-based approaches is oversaturation and unnatural colors, stemming from unstable SDS gradients. This instability results in avatars that appear overly saturated and unrealistic. Additionally, relying on text as guidance for avatar generation presents its own challenges, as textual descriptions are often vague and lack sufficient identity-specific information. This can lead to distortions and artifacts beyond color issues, affecting the fidelity and accuracy of the generated avatars. Motivated by the scarcity of methods in the SDS literature that generate highly realistic, expressive 3D avatars from a single image, we present our approach, termed Arc2Avatar, which addresses these challenges directly by leveraging robust priors from human face foundation models, the representational power of 3DGS, and the controllability of 3DMMs. We use Arc2Face [ 54 ] , a state-of-the-art human face foundation model conditioned on identity embeddings rather than text embeddings. However, to overcome Arc2Face’s predominantly frontal focus, we extend it for diverse-view generation using synthetic data and Low Rank Adaptation [ 30 ] (LoRA) for efficient fine-tuning. Our method employs a modified SDS approach, where splats are strategically initialized on a densely sampled FLAME 3DMM [ 40 ] with a masked 3DGS setup. In this framework, splats in the facial area are protected from densification, pruning, and opacity resets, ensuring they maintain consistent alignment with the template. This alignment is further reinforced by template-based regularizers, which preserve proximity to the template mesh throughout optimization. The proposed approach overcomes the limitation of SDS approaches to produce detailed 3D assets with natural colors due to the inevitable uncertainty in text prompts, by leveraging the precise ID-guidance of Arc2Face, along with template-based regularizers, targeted initialization, and a masked strategy for facial splats. Additionally, this allows for the use of reduced diffusion guidance scale during distillation, in contrast to previous Stable Diffusion-based [ 64 ] distillation methods. Arc2Avatar generates complete 3D heads with unprecedented realism, detail, and natural color fidelity. By leveraging the dense template correspondence, our method further enables realistic expressions through 3DMM blendshapes (see ). Finally, we introduce an optional SDS-based refinement step to correct exaggerated expressions and enhance expression fidelity. In summary, the contributions of this paper are: • The first SDS-based method to successfully generate realistic 3D head avatars from a single image using a face foundation model as guidance. • A carefully designed, task-specific SDS process effective for ID-driven 3D avatar generation. • Blendshape-enabled 3DGS avatars powered by a 3DMM that allows expression generation and refinement using the same framework. 2 Related Work 2.1 ID-Conditioned Diffusion Models Following the remarkable success of text-to-image diffusion models, recent research has focused on adapting foundation models, such as Stable Diffusion (SD) [ 64 ] , for subject-specific image generation. Early methods, including Textual Inversion [ 20 ] and DreamBooth [ 65 ] , leveraged model or embedding optimization to customize SD based on a few images of a target object. However, these methods often encounter challenges due to lengthy optimization times. Subsequent works [ 21 , 93 ] introduced encoders for direct subject conditioning, while [ 35 ] reduced optimization demands by fine-tuning only a subset of the model’s parameters. Although these methods can efficiently preserve the characteristics of arbitrary objects, their ability to adapt to human subjects is limited by the lack of identity-specific features in their conditioning inputs. To achieve consistent identity preservation in human image generation, more recent approaches incorporate facial features to condition text-to-image models. Within this framework, the CLIP image encoder [ 62 ] has become a popular choice for extracting features from facial images, as demonstrated in FastComposer [ 85 ] , PhotoVerse [ 10 ] , MoA [ 77 ] , and PhotoMaker [ 42 ] . These features are then introduced to the pre-trained SD model as a supplement to text embeddings through cross-attention layers. Alternatively, Celeb-Basis [ 90 ] and StableIdentity [ 80 ] condition SD using an embedding basis derived from a

Read the complete text