A Multiscale Network with Supervised Contrastive Learning for Real-Time Facial Emotion Recognition
Rejoy Chakraborty, Archisman Adhikary, Chayan Halder, Payel Rakshit, Sanchita Ghosh, Kaushik Roy · 2026
Read A Multiscale Network with Supervised Contrastive Learning for Real-Time Facial Emotion Recognition by Rejoy Chakraborty, Archisman Adhikary, Chayan Halder, Payel Rakshit, Sanchita Ghosh, Kaushik Roy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01069.txt.
Introduction
Chakraborty and co-authors read emotions from faces in live video in this 2026 paper. A face can pass through many states in one clip. Those states vary a lot from person to person. Expression change is smooth, not a set of hard jumps, which makes it hard to model. A tool that tracks the drift could aid counselors as they work.
Their system is a deep network built at many scales. Supervised contrastive learning shapes its features, so like expressions pull close and unlike ones part. The design targets real time use on standard video. Training runs on a standard dataset for this task. The authors report very satisfactory outcomes from the system. They present it as a support for psychological counseling, where a second read on a subject's state has value. The paper adds to work on affect aware computing, with stress on change over time.
Catalog record
| CREATOR | Rejoy Chakraborty, Archisman Adhikary, Chayan Halder, Payel Rakshit, Sanchita Ghosh, Kaushik Roy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01069
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Multiscale Network with Supervised Contrastive Learning for Real-Time Facial Emotion Recognition Rejoy Chakraborty, Archisman Adhikary, Chayan Halder, Payel Rakshit, Sanchita Ghosh, Kaushik Roy Abstract Real-time emotion recognition from facial expressions is a challenging task, particularly in video-based scenarios where multiple emotional states may occur over time. The difficulty increases further due to the fact that each emotional state is associated with facial expressions that vary significantly across individuals. The change of facial expressions portraying emotional state is not discrete, but rather continuous, which is very challenging to represent through computational aids. A system with the ability to detect variations in facial expressions can have a significant impact on determining the emotional state of an individual. Such a system can be very beneficial for psychologists during counseling by providing additional insights into the emotional state of a subject. In this paper, a deep learning-based system is presented to detect emotional changes in real-time video of a person by modeling the change in facial expressions. The current study is conducted on a standard dataset for training of the deep learning system and the system has provided very satisfactory outcomes in this respect. 1 Introduction Emotion Recognition is one of the currently active research topics that deals with human emotions through an artificial recognition system. Emotion recognition technology can provide benefits to various fields like healthcare, education, and marketing by providing insights into individuals’ emotional states. The facial expression [ 25 ] , vocal tone [ 1 ] , psychological signals [ 26 ] , etc., can be used to get insight into one’s emotional state. Mehrabian et. al. [ 17 ] have analyzed that emotion recognition through facial expression can be one of the most impactful approaches. Emotions and psychological processes are deeply connected [ 9 , 16 ] . The relationship between emotional and psychological changes is bidirectional, which indicates that changes in emotional state can also influence psychological processes. For example, experiencing positive emotions, such as happiness or joy, can lead to an increase in motivation and engagement in activities, which can have a positive impact on psychological functioning, such as improved self-esteem or sense of purpose. To recognize and understand the emotional and psychological state of a person, we have experts in that domain who can understand the significance of the changes in emotions. Now, in day-to-day life, the ratio between psychological experts and people suffering from psychological issues is dreadful. As a consequence, very few people are getting the help of the experts. Here, an automated AI-based system to determine the emotional changes from facial expressions will be crucial for society. Facial expressions can often be ambiguous, with different emotions sometimes being conveyed by similar-looking expressions. For example, a smile naturally indicates happiness, but it can also refer to hiding negative emotions or showing politeness. Other than that, a facial expression can be heavily influenced by the context in which it is observed. For example, a smile in one situation may indicate happiness, while in another it may signal sarcasm or insincerity. Hence, emotions are not static and can change rapidly, often with subtle shifts in facial expression. Changes in emotion can occur over different time scales. Some changes may be relatively short-lived, and some may continue for a day or longer. Due to these factors, it is challenging to accurately recognize emotions in real-time, especially when dealing with video or live-streaming data. Several researchers have proposed their models regarding emotion recognition through facial expression, which are mostly Convolutional Neural Network (CNN)-based architectures [ 20 , 12 , 19 , 34 ] . To the best of our knowledge, none of the previous research works have focused on analyzing and monitoring emotional changes over an extended period, which is crucial for comprehensively understanding a person’s psychological state. Simply recognizing emotions in a single image cannot provide conclusive information about someone’s mental health. Thus, our research addresses this gap by providing a detailed analysis of emotional changes over time. The key contributions of the study are mentioned below: 1. Proposed MSFERNet, a multi-scale Facial Emotion recognition architecture, outperformed standard datasets FER13 [ 3 ] and CK+ [ 14 ] . 2. Developed RT-FER, a user-friendly application for real-time emotion monitoring. 3. The GUI has incorporated a video player, which will provide the user with an opportunity to capture changes in emotion due to the effect of the video. The rest of the paper is organized into different sections. The recent study of literature is presented in Section 2 . The dataset used for the proposed study is described in Section 3 . An extensive description of the methodology is provided in Section 4 . Section 5 provides the experimental results and analysis of the study. The paper is concluded in section 6 . 2 Literature Review Numerous researchers have proposed their architectures to achieve high accuracy in recognizing emotions through facial expressions, trained using standard datasets like FER13 [ 3 ] , MMI [ 23 ] , CK+ [ 14 ] , RAFD-DB [ 11 ] etc. In particular, many of them have also focused on the preprocessing stage to enhance the feature extraction process for facial expressions. Mollahosseini et al. [ 20 ] proposed a CNN model for facial recognition using facial landmark extraction and applied data augmentation techniques for regularization. The architecture stacks two modules similar to Inception module [ 28 ] including pooling layers and it achieves high recognition rates of 94.7\% , 77.9\% , 55\% , 76.7\% , 47.7\% , 93,2\% , and 61.1\% on standard datasets MultiPie [ 4 ] , MMI [ 23 ] , DISFA [ 18 ] , FERA [ 31 ] , SFEW [ 2 ] , CK+ [ 14 ] , FER13 [ 3 ] respectively. Li et al. [ 12 ] proposed a VGGNet [ 27 ] based network, followed by applying the ACNN technique with an attention mechanism while achieving an accuracy of 80.54\% and 54.84\% on RaFD [ 10 ] and AffectNet [ 21 ] respectively. Mohammadpour et al. [ 19 ] introduced a new approach for identifying the facial Action Units (AU) using shallow CNN architecture with only two convolutional layers followed by max-pooling layers and two dense layers identifying the number of AUs. Preprocessing has been applied before feeding to the network. The network architecture consisted of two convolution layers, each followed by max pooling, and two fully connected layers at the end to predict the number of activated AUs and achieved an accuracy of 97.01\% on CK+ [ 14 ] dataset. Jain et al. [ 5 ] proposed a ConvoCNN architecture that integrates two residual blocks, each consisting of four convolution layers. The model was trained on CK+ [ 14 ] and JAFFE [ 15 ] databases, performing cropping and intensity normalization as preprocessing steps, and achieved 93.24\% and 95.23\% , recognition rates, respectively. Kim et al. [ 8 ] proposed a spatio-temporal architecture that combines CNN and LSTM to address the variation of facial expressions, where CNN was first used to learn the spatial features of the facial expressions in all frames of the emotional state, and an LSTM was used to preserve the sequence of these spatial features. The proposed