Express4D: Expressive, Friendly, and Extensible 4D Facial Motion Generation Benchmark
Yaron Aloni, Rotem Shalev-Arkushin, Yonatan Shafir, Guy Tevet, Ohad Fried, Amit Haim Bermano · 2025
Read Express4D: Expressive, Friendly, and Extensible 4D Facial Motion Generation Benchmark by Yaron Aloni, Rotem Shalev-Arkushin, Yonatan Shafir, Guy Tevet, Ohad Fried, Amit Haim Bermano (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12438.txt.
Introduction
Yaron Aloni and co-authors present a 2025 computing paper with a new data set for generating facial motion from text. Animation and virtual avatars need faces that act out nuanced descriptions, not just smile or frown on cue. Existing sets are either driven by speech or labeled with coarse emotions, and were captured on costly rigs.
Express4D was recorded with ordinary equipment, using language-model-written prompts to elicit fine-grained performances. The motion ships in the standard ARKit blendshape format, so it drops straight into animation tools. Two baseline models trained on the set learned the tangled two-way mapping between words and expressions. The authors release the data, code, and examples as a benchmark. Future systems can now be graded on expressive range, not just lip sync.
Catalog record
| CREATOR | Yaron Aloni, Rotem Shalev-Arkushin, Yonatan Shafir, Guy Tevet, Ohad Fried, Amit Haim Bermano |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-12438
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Express4D: Expressive, Friendly, and Extensible 4D Facial Motion Generation Benchmark Yaron Aloni, Rotem Shalev-Arkushin, Yonatan Shafir, Guy Tevet, Ohad Fried, Amit Haim Bermano Abstract Dynamic facial expression generation from natural language is a crucial task in Computer Graphics, with applications in Animation, Virtual Avatars, and Human-Computer Interaction. However, current generative models suffer from datasets that are either speech-driven or limited to coarse emotion labels, lacking the nuanced, expressive descriptions needed for fine-grained control, and were captured using elaborate and expensive equipment. We hence present a new dataset of facial motion sequences featuring nuanced performances and semantic annotation. The data is easily collected using commodity equipment and LLM-generated natural language instructions, in the popular ARKit blendshape format. This provides riggable motion, rich with expressive performances and labels. We accordingly train two baseline models, and evaluate their performance for future benchmarking. Using our Express4D dataset, the trained models can learn meaningful text-to-expression motion generation and capture the many-to-many mapping of the two modalities. The dataset, code, and video examples are available on our webpage: https://jaron1990.github.io/Express4D/ 1 Introduction \begin{overpic}[assets/teaser_2.pdf] \end{overpic} Figure 1 : (Right) We construct our dataset, Express4D, by prompting a Large Language Model to generate a wide range of text instructions for facial motions. These instructions are then enacted by real users and recorded, resulting in an expressive, dynamic facial expression dataset. The dataset integrates seamlessly with animation tools, enabling rich and flexible character animation. (Left) Three examples from Express4D are presented on different face ARKit rigs from the MetaHuman project by Epic Games [ 1 ] . Facial animation, and specifically dynamic facial expression generation, is one of the most sought-after disciplines of computer graphics since its emergence, with broad applications in entertainment, gaming, virtual reality, video conferencing, and human-computer interaction [ 37 , 23 , 45 ] . This elusive task demands subtle and nuanced precision, while perpetually carrying the risk of falling into the uncanny valley [ 28 ] . In spite of its prevalence, automatically generating the subtle nuances and emotions has received limited attention in recent literature, where the main focus was put on text-to-speech applications [ 13 , 38 , 11 ] . This is surprising since text-driven generation demonstrates impressive capabilities across various other domains, from high-quality images [ 33 , 18 , 22 ] and videos [ 35 , 8 , 27 ] to human body motion and dancing [ 36 , 46 , 31 ] . Like many other fields employing machine learning, this stems largely from the scarcity of data. Existing facial motion datasets suffer from several critical limitations that hinder rich and semantic text-driven generation. Most datasets are either dedicated to speech [ 39 , 13 ] , or continue the traditional animation approach of considering only a limited subset of categorical actions (e.g. happiness, anger, and disgust) [ 32 , 9 ] . While these subsets are commonly considered to span the range of human expressions, they do not capture subtle nuances. They hence might serve a human animator well through convenient decomposition, but they fail to represent richness of the facial distribution and to provide corresponding semantics (see Fig. 5 b). For example, ‘Happiness’ could be expressed in many ways, and without semantic description, it is challenging to control the fine-grained details for generation of smiling softly while looking at a loved one, or winking and laughing. Moreover, some facial motions express a combination of emotions, e.g. “A person is transitioning from sadness to laughter as if remembering something funny”, or do not fit into a specific category, e.g. “A person is looking up as if thinking hard”. Hence, a dataset with a variety of facial motions derived from diverse detailed descriptions is needed. A recent work in this direction [ 40 ] estimates motion and expressions from monocular videos; however, this approach relies on automatic reconstructions, which again tends to fail in extracting exactly the same nuances, leading to critical information loss. To address these limitations, we introduce Express4D , a facial motion dataset and baseline benchmark, comprising 1205 diverse sequences performed by 18 participants, featuring LLM-based natural language instruction labels which are performed and recorded; accompanied by two baseline models for benchmark comparisons. Our data is captured using off-the-shelf mobile apps [ 6 ] , leveraging commodity mobile camera and depth sensors in the popular ARKit [ 5 ] blendshape format. This collection method provides precise 3D facial motion, but requires no additional equipment or complex lab settings. The used representation is compatible with common animation pipelines, facilitating the integration of generated animations in downstream tasks, in contrast to the common mesh-based datasets captured in elaborate lab settings. Together, this creates a comprehensive resource for text-to-motion research while maintaining compatibility with industry-standard animation pipelines. Due to the flexibility of our data representation, we can easily adapt the generated facial expressions to diverse characters (see Fig. 3 ). Our simple instruction generation pipeline and minimal equipment requirements further allow the easy extension of our dataset using personal devices. We train two baseline text-to-facial motion generation models over our dataset that demonstrate the generation abilities of models trained on this data, and serve as a baseline for future reference. Additionally, we train a facial motion feature extractor for automatic evaluations of facial motion generation, based on the common architecture used for evaluating human motion generation [ 20 ] . We publicly release our code and dataset for further usage and exploration. 2 Related Work 2.1 Face Representations The 3D Morphable Model (3DMM) [ 7 ] , was a seminal work in modeling 3D faces using PCA, modeling both shape and texture as linear combinations of principal components derived from 3D face scans. This foundational work enabled parametric control over facial identity and expression and laid the groundwork for many subsequent models [ 14 ] . Building on this idea, FLAME [ 25 ] introduced a parametric 3D face model that also employs PCA to represent facial shape and expressions using components learned from 3D scans of different identities and expressive performances. These components are also known as blendshapes. Another prominent representation that emerged around the same time is Apple’s ARKit blendshape model [ 5 ] , which simplifies the approach by leveraging a fixed set of manually designed expression coefficients. ARKit enables efficient and robust real-time facial animation, making it especially suitable for deployment in consumer devices and virtual avatar systems. 2.2 Face Motion Datasets Table 1 summarizes prominent facial expression datasets and compares them. While many datasets aim to capture the nature of human facial expressions, few include fine-grained annotations of the performed actions. Among the compared datasets, only MMHead [ 40 ] and our Express4D provide free-text descriptions. However, MMHead estimates these captions automatically based on recorded videos, whereas in Express4D, the expressions were performed in response to free-text prompts and captured with a depth camera. This makes Express4D the only dataset where expressions are both described in free language and intentionally enacted to match those descriptions. Although ARKit is highly adopted by the industry, current facial motion datasets typically use either the FLAME facial representation [ 9 , 13 , 40 , 32 , 12 ] or the 3DMM mesh representation [ 11 , 39