Modeling Face Emotion Perception from Naturalistic Face Viewing: Insights from Fixational Events and Gaze Strategies
Meisam J. Seikavandi a , Maria J. Barrett b , and Paolo Burelli a · 2025
Read Modeling Face Emotion Perception from Naturalistic Face Viewing: Insights from Fixational Events and Gaze Strategies by Meisam J. Seikavandi a , Maria J. Barrett b , and Paolo Burelli a (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15926.txt.
Introduction
Meisam Seikavandi, Maria Barrett, and Paolo Burelli model how people read emotions from faces in a 2025 study. Recognizing feelings from a face is basic to social life, and eye tracking shows where viewers look while judging. The authors collected eye movements from a small group of participants under an instruction-free design, covering free viewing and a grounded recognition task.
Fixation, pupil, and tiny involuntary eye movements were checked against emotion judgments and task performance. Regions of interest on the face structured the analysis. Understanding natural gaze strategies can guide both psychology and emotion-aware interfaces that respond to how people actually look.
Catalog record
| CREATOR | Meisam J. Seikavandi a , Maria J. Barrett b , and Paolo Burelli a |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15926
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Modeling Face Emotion Perception from Naturalistic Face Viewing: Insights from Fixational Events and Gaze Strategies Meisam J. Seikavandi a , Maria J. Barrett b , and Paolo Burelli a Abstract Face Emotion Recognition (FER) is essential for social interactions and understanding others’ mental states. Utilizing eye tracking to investigate FER has yielded insights into cognitive processes. In this study, we utilized an instructionless paradigm to collect eye movement data from 21 participants, examining two FER processes: free viewing and grounded FER. We analyzed fixational, pupillary, and microsaccadic events from eye movements, establishing their correlation with emotion perception and performance in the grounded task. By identifying regions of interest on the face, we explored the impact of eye-gaze strategies on face processing, their connection to emotions, and performance in emotion perception. During free viewing, participants displayed specific attention patterns for various emotions. In grounded tasks, where emotions were interpreted based on words, we assessed performance and contextual understanding. Notably, gaze patterns during free viewing predicted success in grounded FER tasks, underscoring the significance of initial gaze behavior. We also employed features from pre-trained deep-learning models for face recognition to enhance the scalability and comparability of attention analysis during free viewing across different datasets and populations. This method facilitated the prediction and modeling of individual emotion perception performance from minimal observations. Our findings advance the understanding of the link between eye movements and emotion perception, with implications for psychology, human-computer interaction, and affective computing, and pave the way for developing precise emotion recognition systems. keywords Face Emotion Perception, Emotional Gaze Strategies, Free Face viewing, Microsaccade † † affiliation: a brAIn lab - IT University of Copenhagen,Denmark † † affiliation: b IT University of Copenhagen, Denmark 1 Introduction Facial emotion recognition (FER) is fundamental to human social interactions, enabling individuals to decipher emotions from facial expressions [ 1 , 2 ] . While methodologies like brain imaging and physiological signals have been employed to probe this complex process [ 3 , 4 , 5 ] , eye tracking emerges as a non-invasive yet insightful tool, offering deep insights into visual attention and emotional processing [ 6 ] . Previous research has leveraged eye movements to understand emotion perception (EP) in adults [ 7 , 8 ] , shedding light on atypical EP associated with conditions such as autism, Attention-Deficit/Hyperactivity Disorder (ADHD), schizophrenia, and certain types of dementia [ 9 , 10 ] . Diagnostic eye-tracking tasks, such as the antisaccade task, play a crucial role in identifying diseases like dementia and Alzheimer’s [ 11 , 12 ] . The shift towards naturalistic tasks, mirroring real-life scenarios, gains attraction due to their relaxed environment, making them suitable for lightweight EP assessments. Eye-tracking research extends its applications to clinical diagnosis and human-robot interaction (HRI), with potential implications for designing socially intelligent robots [ 13 ] . Figure 1 : The network structure designed for predicting fixations in Task 2. The model employs a three-layer MLP to analyze spatial, temporal, and spatiotemporal features to predict the dwell time percentage for each face. In this chapter, we extend the work we have done in [ 14 ] and delve more into FER using an instructionless paradigm, exploring two FER processes: free viewing and grounded FER. We aim to unravel the intricate relationship between eye movements and emotion perception by scrutinizing fixational, pupillary, and microsaccadic events extracted from eye movement data. By integrating these gaze features with deep-learning models, our study uniquely combines the analysis of microsaccadic events and advanced computational techniques to enhance the understanding of emotional processing. By delineating regions of interest in the face and harnessing deep-learning models for face recognition, we investigate the role of eye-gaze strategies in face processing and their link to presented emotions and emotion perception performance. Leveraging features extracted from pre-trained deep-learning models, we analyze attention during free viewing, enhancing scalability and comparability across datasets and populations. Furthermore, we employ a sequential model with bidirectional LSTM layers to capture the temporal aspects of fixation between regions of interest, providing insights into the dynamic nature of gaze behavior during face viewing. Our study contributes to the efficiency of EP assessments by predicting FER success based on gaze features during face viewing. This approach not only enhances the precision and ecological validity of emotion recognition systems but also has significant implications for clinical diagnostics and human-computer interaction. By extending the understanding of the eye movements-emotion perception relationship, our work impacts psychology, HCI, and affective computing domains, offering potential advancements in FER research and applications. Figure 2 : The FER task unfolds in three steps: (1) overlaying areas of interest, (2) presenting faces with emotions, and (3) randomizing locations to study gaze patterns. Figure 3 : The heatmaps display FER trial with no instructions. Different emotions have distinct fixation distributions, showcasing varied attention patterns across emotions. 2 Background 2.1 Instructionless FER Task This section outlines the instructionless FER task initially developed by Russell et al. [ 10 ] and our modifications to enhance the understanding of the FER process. Russell et al. designed the task to identify early-stage frontotemporal dementia, comprising: 1. Displaying four faces with distinct emotions for 10 seconds 2. Presenting an emotion word for 2 seconds 3. Showing the word and faces together for 3 seconds This setup aimed to intuitively direct participants’ gaze to the face matching the emotion if they remembered its position. The constant positions of the faces simulated a memory retrieval task, which helped analyze free-viewing and retrieval phases and considered working memory as a potential confounder. 2.2 Modifications to the Instructionless FER Task To deepen our insight into the FER process and mitigate the effect of working memory, we adjusted Russell et al.’s design by randomizing face positions in Step 3. This required participants to recognize rather than recall face positions, effectively isolating the role of memory. This alteration helped distinguish between the free-viewing (Step 1) and grounded FER (Step 3) phases. Through these changes, we analyzed gaze behavior and performance variations between these phases, generating quantitative data on cognitive processes in FER. These adjustments better simulate real-world scenarios, enhancing our study of the links among eye movements, emotion perception, and attention in FER tasks. 2.3 Microsaccades Microsaccades are small, involuntary eye movements that occur during visual fixation, typically lasting 6-30 milliseconds and with amplitudes under 0.1 degree of visual angle. Interspersed with slow drifts, they prevent retinal image fading and are vital for tasks requiring sustained visual attention, like reading [ 15 ] . Studies suggest that microsaccadic activity responds to emotional stimuli, affecting attention and emotion-related cognitive processes. Emotional arousal, fatigue, and saccade preparation can influence microsaccades, altering their rate and magnitude depending on the emotional context [ 16 ] . While some research reports significant changes in microsaccadic behavior in response to different emotions, others find no noticeable differences [ 17 ] . Further studies link microsaccades with cognitive effort, affective priming, and arousal, highlighting their sensitivity to these factors and their role in emotional and cognitive processing [ 17 ] . This underlines the importance of microsaccades in understanding visual attention and emotion perception. ((a)) ((b)) ((c)) Figure 4 : Microsaccades across different steps. Step 1 shows baseline microsaccadic activity, Step