Geometry-Aware Texture Generation for 3D Head Modeling with Artist-driven Control
Amin Fadaeinejad Abdallah Dib Luiz Gustavo Hafemann Emeline Got Trevor Anderson Amaury Depierre Nikolaus F. Troje Marcus A. Brubaker Marc-André Carbonneau · 2025
Read Geometry-Aware Texture Generation for 3D Head Modeling with Artist-driven Control by Amin Fadaeinejad Abdallah Dib Luiz Gustavo Hafemann Emeline Got Trevor Anderson Amaury Depierre Nikolaus F. Troje Marcus A. Brubaker Marc-André Carbonneau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04387.txt.
Introduction
Amin Fadaeinejad and co-authors present a framework for generating 3D head textures under artist control in a 2025 paper. Building a digital head that matches an exact artistic vision takes long hours of skilled labor. Their pipeline learns how head shape and skin texture relate across many demographic groups, then lets artists steer the result.
Control comes at three levels. Artists can reshape the overall head, shift skin tone while keeping the person's character, and edit fine details like wrinkles or facial hair. An edit made on one texture map spreads on its own to the other maps that realistic rendering needs. Tests show varied, clean results, and demos cover tone changes and adding age details or cleaning scanned models.
Catalog record
| CREATOR | Amin Fadaeinejad Abdallah Dib Luiz Gustavo Hafemann Emeline Got Trevor Anderson Amaury Depierre Nikolaus F. Troje Marcus A. Brubaker Marc-André Carbonneau |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04387
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geometry-Aware Texture Generation for 3D Head Modeling with Artist-driven Control Amin Fadaeinejad Abdallah Dib Luiz Gustavo Hafemann Emeline Got Trevor Anderson Amaury Depierre Nikolaus F. Troje Marcus A. Brubaker Marc-André Carbonneau Abstract Creating realistic 3D head assets for virtual characters that match a precise artistic vision remains labor-intensive. We present a novel framework that streamlines this process by providing artists with intuitive control over generated 3D heads. Our approach uses a geometry-aware texture synthesis pipeline that learns correlations between head geometry and skin texture maps across different demographics. The framework offers three levels of artistic control: manipulation of overall head geometry, adjustment of skin tone while preserving facial characteristics, and fine-grained editing of details such as wrinkles or facial hair. Our pipeline allows artists to make edits to a single texture map using familiar tools, with our system automatically propagating these changes coherently across the remaining texture maps needed for realistic rendering. Experiments demonstrate that our method produces diverse results with clean geometries. We showcase practical applications focusing on intuitive control for artists, including skin tone adjustments and simplified editing workflows for adding age-related details or removing unwanted features from scanned models. This integrated approach aims to streamline the artistic workflow in virtual character creation. 1 Introduction Creating immersive virtual experiences relies on realistic character heads with diverse appearances across video games, films, and virtual reality applications. This requires generating both detailed geometry and high-quality texture maps for convincing rendering. Traditional approaches include capturing real subjects in light stages [ 8 ] or manually sculpting in specialized software like ZBrush 1 1 1 https://www.maxon.net/en/zbrush , but these methods remain labor-intensive, requiring significant time investment and specialized skills from artists. Alternative approaches to 3D head generation rely on 3D Morphable Models (3DMMs) [ 4 ] , which create statistical models from head scan collections [ 11 ] . While these linear models offer convenient generation through basis combination, they typically produce overly smooth results lacking the fine details necessary for realism. More recent non-linear approaches using Generative Adversarial Networks [ 16 ] can generate more detailed heads [ 27 , 38 ] with corresponding texture maps. However, these methods often require additional post-processing steps to achieve satisfactory results. Other generative based methods, whether image-conditioned [ 26 , 25 , 36 ] or text-prompted [ 54 , 49 ] , offer limited artistic control. They typically only allow global modifications rather than precisely placed details like scars or wrinkles, and often require multiple iterations to achieve a specific artistic vision, particularly when trying to realize precise facial attributes or skin tones through text descriptions. We present a novel framework that provides artists with intuitive control at three levels while streamlining the creation process. First, our approach generates high-quality head geometry with corresponding textures that maintain consistency across different demographics. Second, we enable precise skin tone manipulation without altering other facial characteristics, which helps both artistic expression and representation of diverse populations. Third, we facilitate detailed editing of specific features like wrinkles or facial hair. All of these controls are integrated into a cohesive system that automatically propagates edits across all necessary texture maps. Our contributions include: • A multi-level control framework for 3D head generation that enables artists to manipulate geometry, skin tone, and fine details independently. • A geometry-aware texture generation approach that maintains consistency between facial structure and appearance. • A novel approach for editing fine-grained details in intrinsic texture maps. This allows artists to modify a single texture map using any image editing tool, with the changes coherently propagated to the intrinsic texture maps. • A data-driven method for precise manipulation of skin tones while preserving other facial attributes, enabling greater diversity in virtual character creation. 2 Related works Generative head models The generation of faces and full heads has traditionally relied on statistical morphable models (3DMM) [ 4 , 28 , 14 , 6 ] . These linear models are constructed by applying Principal Component Analysis (PCA) to a collection of head scans. The resulting orthogonal basis enables the sampling of new heads through linear combinations of basis vectors. However, due to the lack of semantic interpretation in this basis, controlling or editing the generation process remains unintuitive. Moreover, the generated heads lack fine details, as high-frequency information is not captured by PCA. Non-linear morphable models [ 27 , 38 , 13 ] have been shown to produce more detailed heads compared to linear models [ 13 ] . These non-linear models can be categorized into two main types: the first leverages Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) [ 16 ] operating in UV space [ 27 ] , while the second utilizes Graph Neural Networks (GNNs) that operate directly on vertex coordinates [ 2 ] . The approaches using CNNs consider the generation of geometry and color within the 2D UV space as in [ 27 ] . This requires flattening the 3D geometry into UV space. However, flattening the geometry onto a 2D space without making cuts isn’t feasible. As a result, some areas that are close together in 3D end up being far in the 2D representation. Conversely, some positions that are close on the 2D UV map can be far apart in the 3D mesh, such as the eyes and mouth. This creates problems for using CNNs in UV space, as it violates the inductive bias of locality in CNNs. This leads to visible artifacts on the geometry. Post-processing is generally applied to fix these artifacts as in [ 27 ] . The second approach for generative head models uses Graph Neural network (GNN) and auto-encoders to generate a complete head without requiring additional post-processing [ 39 , 2 ] . However, as they operate on a vertex-level basis, they are unable to generate sharp and detailed color texture maps. In this work, we propose a geometry-aware texture synthesis network, where the geometry is driven by a GNN-based auto-encoder, and the color texture generation is based on CNNs, for a complete and high-quality head generation. Controllable generative models Controlling generative models has achieved remarkable advances for 2D portrait image editing, mainly because of the availability of a large corpus of 2D portrait image datasets such as [ 30 , 22 ] . Such methods [ 1 , 55 , 34 , 53 , 52 , 41 , 42 , 19 , 29 , 5 ] , allow for semantic editing of the face attributes on 2D portrait images. This is achieved by projecting the image onto the latent space of StyleGAN [ 23 ] and finding orthogonal directions in that space that allow for controlling various face attributes (such as hair, age, expression, and eyeglasses). Recently, [ 46 ] uses a pre-trained StyleGAN [ 24 ] to control the skin tone of a 2D portrait image. Although numerous works tackled the problem of controlling generative models for 2D portrait images, there is a scarcity of work addressing this problem for 3D facial asset generation. When referring to 3D facial assets, we consider both the geometry (coordinates of vertices in 3D space) and the associated texture maps (including diffuse, specular, and normal maps) used for realistic rendering. StyleRig [ 45 ] drives