Exploring the Temporal Dynamics of Facial Mimicry in Emotion Processing Using Action Units

Meisam J. Seikavandi, Jostein Fimland, Maria Barrett, and Paolo Burelli brAIn lab, IT University of Copenhagen, Denmark IT University of Copenhagen, Denmark · 2025

Read Exploring the Temporal Dynamics of Facial Mimicry in Emotion Processing Using Action Units by Meisam J. Seikavandi, Jostein Fimland, Maria Barrett, and Paolo Burelli brAIn lab, IT University of Copenhagen, Denmark IT University of Copenhagen, Denmark (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17306.txt.

Introduction

Meisam Seikavandi and co-authors study how people copy the facial expressions of others over time, and how that copying varies by emotion, in a paper from 2025. This copying, called facial mimicry, is automatic and helps people grasp what others feel. Most past work treated mimicry as a single snapshot rather than a moving pattern.

The team coded videos into small facial action units for both the shown faces and the watchers' faces. A warping method then measured how closely the two time patterns lined up. Fear drew stronger copying than happiness, while anger drew less copying than fear. Copying also tracked modestly with traits such as extraversion and agreeableness. The authors link the findings to emotion software that reads and answers human expression.

Catalog record

CREATORMeisam J. Seikavandi, Jostein Fimland, Maria Barrett, and Paolo Burelli brAIn lab, IT University of Copenhagen, Denmark IT University of Copenhagen, Denmark
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17306

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring the Temporal Dynamics of Facial Mimicry in Emotion Processing Using Action Units Meisam J. Seikavandi, Jostein Fimland, Maria Barrett, and Paolo Burelli brAIn lab, IT University of Copenhagen, Denmark IT University of Copenhagen, Denmark Abstract Facial mimicry—the automatic, unconscious imitation of others’ expressions—is vital for emotional understanding. This study investigates how mimicry differs across emotions using Face Action Units from videos and participants’ responses. Dynamic Time Warping quantified the temporal alignment between participants’ and stimuli’s facial expressions, revealing significant emotional variations. Post-hoc tests indicated greater mimicry for ’Fear’ than ’Happy’ and reduced mimicry for ’Anger’ compared to ’Fear’. The mimicry correlations with personality traits like Extraversion and Agreeableness were significant, showcasing subtle yet meaningful connections. These findings suggest specific emotions evoke stronger mimicry, with personality traits playing a secondary role in emotional alignment. Notably, our results highlight how personality-linked mimicry mechanisms extend beyond interpersonal communication to affective computing applications, such as remote human-human interactions and human-virtual-agent scenarios. Insights from temporal facial mimicry—e.g., designing digital agents that adaptively mirror user expressions—enable developers to create empathetic, personalized systems, enhancing emotional resonance and user engagement. I Introduction Facial mimicry, the automatic imitation of others’ expressions, plays a crucial role in empathy and social interaction [ 15 , 10 ] . Often termed the ‘social glue’ [ 24 ] , it fosters emotional connection and social cohesion. This subtle, unconscious reaction is central to emotion processing [ 29 , 31 ] and offers valuable insights for enhancing human-computer interaction [ 23 ] . While facial expression research is extensive [ 21 , 36 , 46 , 37 ] , the temporal dynamics of mimicry remain underexplored. Studies have primarily focused on static aspects, overlooking how expressions align and evolve over time—a key factor in emotional contagion, where individuals unconsciously synchronize expressions. Personality traits like Extraversion and Agreeableness influence mimicry responses [ 16 ] , yet their impact on temporal dynamics needs further study. To bridge these gaps, this study examines the temporal dynamics of facial mimicry during emotion processing using Action Units (AUs) [ 13 ] . Dynamic Time Warping (DTW) enables us to analyze temporal alignment, revealing nuanced patterns missed in static analyses [ 33 ] . These findings can inform affective computing systems—such as virtual agents or telepresence applications—by leveraging personality-linked mimicry patterns to mirror users’ expressions in real time. This approach enhances emotional resonance and engagement, particularly in remote interactions without full physical presence. Our research questions are: 1. How do personality traits influence the temporal dynamics of facial mimicry and emotion recognition? 2. What is the relationship between the temporal alignment of facial mimicry and the congruence between perceived and felt emotions? 3. How do different emotions affect the temporal patterns of facial mimicry and recognition accuracy? By addressing these questions, we aim to advance the understanding of facial mimicry in emotion processing, with implications for social cognition, clinical interventions, and technology design. II Background Facial mimicry plays a vital role in human communication as a nonverbal channel for emotional exchange. It involves the automatic imitation of others’ facial expressions, enhancing emotion recognition and empathy [ 29 , 31 , 2 ] . Studying facial mimicry provides insights into social cognition, affective processes, and neural mechanisms. II-A Facial Mimicry and Emotion Recognition The matched motor hypothesis suggests that observing facial expressions activates corresponding motor representations, aiding emotion recognition through embodied simulation [ 28 ] . This supports the action-perception loop , where perceiving expressions trigger motor responses that refine emotional understanding [ 7 ] . Mimicry deficits are linked to impaired recognition, especially in negative moods [ 15 , 7 ] . The facial feedback hypothesis proposes that facial actions amplify emotional experiences, forming a feedback loop between expression and emotion [ 44 , 20 ] . Mimicry is largely automatic, integrating cues like voice and movement for complete emotional representation [ 3 ] . II-B Neural Mechanisms of Facial Mimicry Facial mimicry involves the mirror neuron system , which activates during both observation and execution of expressions, facilitating emotional understanding [ 18 ] . Brain imaging links emotional expression observation to empathy and emotion processing areas [ 35 ] , though the underlying neural pathways remain partially understood. II-C Personality Traits and Emotion Processing Personality traits from the Big Five Inventory—Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism—affect emotion recognition and mimicry. Extraverts, attuned to social cues, exhibit greater mimicry [ 38 , 26 ] , while Neuroticism is associated with a bias toward negative emotions, impairing accuracy [ 11 ] . II-D Action Units in Emotion Recognition Facial mimicry influences emotional communication, with AUs from the Facial Action Coding System (FACS) [ 13 ] indicating facial expressions [ 43 ] . AUs, which categorize muscle movements linked to emotions, are detectable via electromyography (EMG) or image-based techniques. They capture subtle changes, revealing micro-expressions that expose concealed emotions [ 40 ] .Recent advances in AU detection, like Zhi et al. [ 49 ] , highlight deep learning models. Automatic AU imitation enhances empathy and recognition [ 17 ] . Research by Pfister et al. [ 30 ] and Song et al. [ 39 ] underscores micro-expression recognition’s importance, with technological advances improving AU classification [ 41 , 47 , 12 ] . II-E Dynamic Time Warping and Facial Expressions DTW aligns time-dependent sequences, such as facial expressions and AUs. Raducanu and Dornaika [ 32 ] showed that dynamic recognition of facial expressions using DTW outperforms static image analysis. Khan et al. [ 22 ] applied DTW to measure AU similarities, enhancing emotion classification. Zhao et al. [ 48 ] optimized K-Nearest Neighbor classification with DTW, improving facial expression analysis. Yang et al. [ 45 ] demonstrated DTW’s efficacy in emotion recognition, boosting classification accuracy. II-F Research Gap Despite extensive research on static mimicry, temporal dynamics remain underexplored. AUs provide a granular approach to mimicry analysis. This study bridges gaps by using AUs and DTW to examine the temporal alignment of expressions. We analyze AU divergence and its correlation with personality traits and discrepancies between felt and perceived emotions, offering deeper insights into individual differences in emotion recognition. III Methods III-A Participants We recruited 73 participants (52 males, 21 females; mean age 27.4\pm 6 years) with normal or corrected-to-normal vision and neurotypicality ( diagnosed with neurodivergent conditions such as autism, ADHD, or dyslexia). Informed consent was obtained per university ethical guidelines. III-B Experimental Design and Procedure Participants completed 88 trials (4 practice, 84 main) using videos from the CREMA-D dataset [ 6 ] , which features actors expressing six emotions at varying intensities. The CREMA-D dataset provides demographic diversity with 91 actors (48 male, 43 female) aged 20–74, covering six basic emotions (Anger, Disgust, Fear, Happy, Neutral, Sad) at four intensity levels. We selected 84 clips representing diverse combinations to ensure a balanced emotional and demographic context, supporting generalizability. To simulate face-to-face dialogue, a written scenario was provided before each video to enhance engagement and emotional alignment. Participants imagined conversing with the individuals in the videos, and responding to their emotions. Facial expressions were recorded at up to 40 fps using a webcam, synchronized with stimuli via Lab Streaming Layer (LSL is a framework for real-time data synchronization and integration across

Read the complete text