7ABAW-Compound Expression Recognition via Curriculum Learning * Equal Contribution, † \dagger Corresponding authors. This work is done at Netease.
Chen Liu *, Feng Qiu *, Wei Zhang, Lincheng Li † \dagger, Dadong Wang, Xin Yu · 2025
Read 7ABAW-Compound Expression Recognition via Curriculum Learning * Equal Contribution, † \dagger Corresponding authors. This work is done at Netease. by Chen Liu *, Feng Qiu *, Wei Zhang, Lincheng Li † \dagger, Dadong Wang, Xin Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07969.txt.
Introduction
Chen Liu and co-authors present a 2025 computing paper on recognizing compound facial expressions. A compound expression blends two basic emotions, like happily surprised. Labeled data for such blends is scarce, and the differences between blends are subtle.
Their framework learns in stages, following a curriculum. First, the model trains on single expressions to master the basic facial features of each emotion. Next, mixing techniques blend single-expression images to fake compound ones, filling the data gap. Finally, real multi-expression data is folded in once the basics are firm. The work was built for a workshop challenge on affective behavior analysis in the wild. The staged design helps the model climb from simple to hard cases in order.
Catalog record
| CREATOR | Chen Liu *, Feng Qiu *, Wei Zhang, Lincheng Li † \dagger, Dadong Wang, Xin Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-07969
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
7ABAW-Compound Expression Recognition via Curriculum Learning * Equal Contribution, † \dagger Corresponding authors. This work is done at Netease. Chen Liu *, Feng Qiu *, Wei Zhang, Lincheng Li † \dagger, Dadong Wang, Xin Yu Abstract With the advent of deep learning, expression recognition has made significant advancements. However, due to the limited availability of annotated compound expression datasets and the subtle variations of compound expressions, Compound Emotion Recognition (CE) still holds considerable potential for exploration. To advance this task, the 7th Affective Behavior Analysis in-the-wild (ABAW) competition introduces the Compound Expression Challenge based on C-EXPR-DB, a limited dataset without labels. In this paper, we present a curriculum learning-based framework that initially trains the model on single-expression tasks and subsequently incorporates multi-expression data. This design ensures that our model first masters the fundamental features of basic expressions before being exposed to the complexities of compound emotions. Specifically, our designs can be summarized as follows: 1) Single-Expression Pre-training: The model is first trained on datasets containing single expressions to learn the foundational facial features associated with basic emotions. 2) Dynamic Compound Expression Generation: Given the scarcity of annotated compound expression datasets, we employ CutMix and Mixup techniques on the original single-expression images to create hybrid images exhibiting characteristics of multiple basic emotions. 3) Incremental Multi-Expression Integration: After performing well on single-expression tasks, the model is progressively exposed to multi-expression data, allowing the model to adapt to the complexity and variability of compound expressions. The official results indicate that our method achieves the best performance in this competition track with an F-score of 0.6063. Our code is released at this repository . 1 Introduction The Compound Expression (CE) Recognition task focuses on identifying complex emotional states conveyed by facial expressions that are combinations of basic emotions. Unlike recognizing single, basic emotions such as happiness, sadness, or anger, compound expressions involve more nuanced and mixed emotions, such as happily surprised, sadly angry, or fearfully disgusted. These compound expressions provide a richer and more accurate representation of human affective states [ 15 , 10 , 45 , 18 , 56 , 31 , 37 , 13 , 14 , 12 , 17 , 16 , 11 , 53 , 57 , 58 , 26 ] . Single-expression recognition has seen significant advancements with the advent of deep learning, which enables more accurate and efficient recognition of basic emotional states. However, understanding compound expressions still faces great challenges. We categorize several challenges inherently of this task as follows: a) Limited datasets. The annotated compound expression data is very rare, posing a challenge for training robust models. b) Subtle Differences of Compound Expressions. The compound expression usually contains subtle variations in facial features. Hence, they are more difficult to distinguish than basic emotions. To facilitate the development of this field, the 7th Affective Behavior Analysis competition (ABAW7) [ 19 ] set the Compound Expression (CE) [ 3 , 7 , 42 , 43 ] Challenge based on the C-EXPR-DB [ 9 ] dataset. Participations are required to achieve compound expression recognition in videos with limited amounts and unknown labels [ 3 , 7 , 42 , 43 ] . To solve the above challenges, we introduce a curriculum learning framework, which enhances the generalization ability of our model in compound expression recognition tasks by gradually transitioning from basic expression to multi-expression learning. More specifically, we first train our model on the datasets just containing single expressions. In this fashion, our model learns the foundational facial features associated with basic emotions. After achieving satisfactory performance on the single-expression recognition task, we expose the model to data with multiple expressions progressively. This stepwise introduction allows the model to adapt to the increased complexity and variability inherent in compound expressions. To address the scarcity of annotated compound expression datasets, we apply CutMix and Mixup to the original single-expression images, creating hybrid images that exhibit characteristics of multiple basic emotions. The CutMix technique involves cutting and pasting patches from different images, while Mixup generates linear interpolations between pairs of images. These augmented images are then used to train the model, providing it with a rich and varied training set that includes a wide range of compound expressions. We conduct comprehensive evaluations on the officially provided dataset and our self-curated validation dataset. The results demonstrate the effectiveness of our method. 2 Related Work Compared to compound expression recognition, basic expression recognition has achieved many advancements in recent years [ 4 , 8 , 1 , 51 ] . Single-expression recognition has been a foundational area of research in the field of affective computing and computer vision [ 59 , 25 , 24 , 35 , 34 ] . Early methods rely heavily on handcrafted features such as Local Binary Patterns (LBP) [ 32 ] , Histogram of Oriented Gradients (HOG) [ 2 ] , and Gabor filters [ 30 ] to capture the distinct characteristics of facial expressions. These traditional techniques, though effective to some extent, faced limitations in handling variations in lighting, pose, and occlusion. With the advent of deep learning, particularly convolutional neural networks (CNNs), the performance of single expression recognition systems has seen significant improvements [ 49 , 48 , 46 , 50 , 47 ] . CNNs automatically learn hierarchical feature representations from raw pixel data, leading to more robust and accurate recognition. Hasani and Mahoor [ 5 ] employ deep residual networks for CER, while Kosti et al. [ 20 ] use LSTM networks to capture temporal dynamics in video sequences. Multi-task learning (MTL) frameworks, as explored by Zhang et al. [ 55 ] , have also enhanced CER by leveraging shared feature extractors with task-specific heads. Further advancements were made by incorporating transfer learning and fine-tuning pre-trained models on large-scale face datasets. This approach leveraged the generalization capabilities of models trained on extensive datasets, such as VGG-Face [ 33 ] and ResNet [ 21 ] , to enhance the performance of expression recognition tasks. Additionally, researchers have explored the integration of attention mechanisms and ensemble learning to refine the focus on critical facial regions and combine the strengths of multiple models [ 39 , 41 , 59 ] , respectively. However, given the complexity of human emotions in real-world situations, detecting a single expression is inadequate. Compound Expression Recognition (CER) extends traditional emotion recognition by identifying complex, blended expressions. Early efforts by [ 3 , 7 ] systematically categorized compound emotions, laying the groundwork for further studies [ 49 , 48 , 46 , 50 , 47 ] . Dimitrios [ 9 ] curate the Multi-Label Compound Expression dataset, C-EXPR, and introduced C-EXPR-NET, which simultaneously tackles Compound Expression Recognition (CER) and Action Unit (AU) detection tasks. Despite progress, challenges remain due to the subtlety and variability of compound expressions, highlighting the requirement for more sophisticated models and diverse datasets. 3 Method In this work, we employ Masked Autoencoder (MAE) [ 6 ] as our feature extractor. In this section, we present our approach for the two competition tracks in three parts. First, we will describe the construction process of the feature extractor utilized in this challenge. Then we detail our framework flow (as depicted in Fig. 1 ) for the Compound Expression Recognition (CER) challenge.