MotivNet: Evolving Meta-Sapiens into an Emotionally Intelligent Foundation Model

Rahul Medicharla, Alper Yilmaz · 2025

Read MotivNet: Evolving Meta-Sapiens into an Emotionally Intelligent Foundation Model by Rahul Medicharla, Alper Yilmaz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24231.txt.

Introduction

Rahul Medicharla and Alper Yilmaz build a facial emotion recognition model meant for the real world. State-of-the-art models often generalize weakly. Tested on diverse data, their performance drops. That drop has held the whole field back. Complex fixes usually demand cross-domain training. Such training contradicts real-world use, since target domains are unknown in advance.

Their model, MotivNet, avoids cross-domain training entirely. It uses Meta-Sapiens as its backbone. Sapiens is a human-vision foundation model with strong real-world generalization. Huge masked-autoencoder pretraining gives it that strength. MotivNet adds emotion recognition as a downstream task for Sapiens. Three criteria judge its viability as such a task. They cover benchmark scores, model similarity, and data similarity. Results meet the criteria across datasets. The code is publicly available.

Catalog record

CREATORRahul Medicharla, Alper Yilmaz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-24231

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MotivNet: Evolving Meta-Sapiens into an Emotionally Intelligent Foundation Model Rahul Medicharla, Alper Yilmaz Abstract In this paper, we introduce MotivNet, a generalizable facial emotion recognition model for robust real-world application. Current state-of-the-art FER models tend to have weak generalization when tested on diverse data, leading to deteriorated performance in the real world and hindering FER as a research domain. Though researchers have proposed complex architectures to address this generalization issue, they require training cross-domain to obtain generalizable results, which is inherently contradictory for real-world application. Our model, MotivNet, achieves competitive performance across datasets without cross-domain training by using Meta-Sapiens as a backbone. Sapiens is a human vision foundational model with state-of-the-art generalization in the real world through large-scale pretraining of a Masked Autoencoder. We propose MotivNet as an additional downstream task for Sapiens and define three criteria to evaluate MotivNet’s viability as a Sapiens task: benchmark performance, model similarity, and data similarity. Throughout this paper, we describe the components of MotivNet, our training approach, and our results showing MotivNet is generalizable across domains. We demonstrate that MotivNet can be benchmarked against existing SOTA models and meets the listed criteria, validating MotivNet as a Sapiens downstream task, and making FER more incentivizing for in-the-wild application. The code is available at github.com/OSUPCVLab/EmotionFromFaceImages 1 Introduction As humans, we are inherently emotive and express ourselves through language and pose, using tools such as gestures, body postures, and facial expressions to visually enhance our message. Experts across various fields have recognized the importance of these visual cues, prompting researchers to analyze them systematically. One such research domain that has formed as a result is Facial Expression Recognition (FER) [ 30 , 25 ] . FER is a computer vision discipline focused on developing models that categorize static facial images as universally recognizable emotions. It has grown significantly in recent years and has become a key focus area in computer vision due to its widespread applicability in healthcare, education, and public safety [ 5 , 25 , 12 , 30 ] . Current state-of-the-art FER models can classify facial images on intra-dataset samples with satisfactory performance [ 30 ] . However, these models tend to have poor generalization on diverse data, leading to severely deteriorated performance in a real-world context [ 30 , 31 ] . This is a result of the lack of publicly available FER datasets that are large, diverse, and balanced enough to mimic the real world [ 25 , 7 ] . Most FER datasets today are either collected in-lab or sourced from the Internet [ 30 ] . In-lab datasets generally do not have enough samples to capture the racial and cultural variance of humans, and large-scale FER datasets collected from the Internet are heavily imbalanced against certain emotions, such as disgust or contempt, leading to limited data on the intra-class variance for these labels [ 30 , 7 ] . The use of either type of dataset causes models to have weak generalization when tested cross-domain, making FER less incentivizing for use in the real world and hindering FER as a research domain. To address this, researchers have proposed complex architectures that join data across multiple domains to improve generalization [ 30 ] ; however, these architectures require training on multiple domains in order to be generalizable on them, which is inherently contradictory for real-world applicability. FER models should be generalizable on unseen data to be useful in the wild. To address these issues, we introduce MotivNet, a transfer-learning-based facial emotion recognition model that is generalizable without cross-domain training for robust real-world application. We sample a uniform distribution of classes to comprise our dataset, minimizing the impact of class imbalance, and capture the native diversity of humans by using Meta-Sapiens [ 13 ] as a backbone. Sapiens is a state-of-the-art human vision foundational model with four main downstream tasks: pose estimation, body part segmentation, depth estimation, and surface normal estimation [ 13 ] . Built on a pretrained Masked Autoencoder [ 10 ] , Sapiens achieved SOTA generalization on diverse real-world human data and upholds this generalization when adapted for various tasks, incentivizing us to experiment with the model’s capabilities as a feature extractor. In this paper, we propose MotivNet as an additional Sapiens downstream task. To evaluate our model’s viability, we introduce three criteria. 1) The model architecture should not vary significantly from the Sapiens architecture; 2) The data used to fine-tune the model should be similar to the data Sapiens is trained on; 3) The model should achieve results competitive with current cross-domain FER benchmarks. Our model, MotivNet, satisfies all three criteria. MotivNet employs an encoder-decoder architecture with Sapiens as the backbone and ML-Decoder [ 26 ] as the classification head. It is fine-tuned on the AffectNet dataset [ 22 ] . Throughout the paper, we will discuss related work in FER, the Meta-Sapiens architecture, our methods for unbiasedly training a model, and our results showing that MotivNet is generalizable. We validate MotivNet as a Sapiens task, pushing forward FER’s applicability in the real world. Our contributions are summarized below. 1. We introduce MotivNet, an FER model that is generalizable without cross-domain training and finetuned in an unbiased manner for robust application in the real world. 2. MotivNet is proposed and validated as a novel Sapiens downstream task, achieving performance competitive with current cross-domain FER models. 2 Related Work Datasets . A few large-scale datasets published in recent years have become the norm for training and benchmarking FER models. These include AffectNet [ 22 ] , FER-2013 [ 8 ] , EmotioNet [ 2 ] , and RAF-DB [ 16 ] [ 30 ] . These datasets contain, at a minimum, tens of thousands of static facial images for use in FER tasks, with AffectNet and EmotioNet superseding this with around one million images [ 30 ] . To accumulate such a large number of samples, these datasets are sourced from the Internet. They are collected with the goal of providing robust real-world context that captures the native diversity of humans. However, as previously described, these large in-the-wild FER datasets are affected by heavy class imbalance [ 7 ] . Since the images are pulled from the Internet, it is likely that a primary source for them is social media. This brings about an uneven distribution of class frequencies, since people generally do not post pictures of themselves disgusted or fearful. Models solely trained with these datasets become affected by this class imbalance, showing a significant bias toward majority labels such as happy or neutral when tested [ 7 ] . To counteract this imbalance, we randomly select a uniform distribution of samples from each class to comprise our dataset, thus limiting the impact of this native imbalance. Figure 1 : An example of FER class imbalance by graphing the frequency of each label in the AffectNet dataset. [ 13 ] Common Models. Deep Learning FER models today typically adhere to a common pipeline: a backbone that extracts features and a classification head that outputs predictions [ 30 ] . The scope of relevant FER work can be broadly categorized into two variants based on the model’s backbone architecture: Convolutional Neural Networks (CNN) [ 24 ] or attention-based

Read the complete text