The Emotion-Memory Link: Do Memorability Annotations Matter for Intelligent Systems? PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE

Maria Tsfasman, Ramin Ghorbani, Catholijn M. Jonker, Bernd Dudzik · 2025

Read The Emotion-Memory Link: Do Memorability Annotations Matter for Intelligent Systems? PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Maria Tsfasman, Ramin Ghorbani, Catholijn M. Jonker, Bernd Dudzik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-14084.txt.

Introduction

Maria Tsfasman, Ramin Ghorbani, Catholijn Jonker, and Bernd Dudzik test whether felt emotion predicts what people remember from a group chat. Their paper is from two thousand twenty-five. Smart systems that support meetings could do better if they knew which moments users will recall. Emotion is often taken as a sign of a memorable moment.

Annotators marked both emotion and memorability second by second in live group talks. The emotion marks came from outside observers, not from the speakers themselves. The link between the two sets of marks was no stronger than chance. That result questions the habit of using third-party emotion labels as a stand-in for personal relevance. The authors say first-person data may be needed for memory-aware systems.

Catalog record

CREATORMaria Tsfasman, Ramin Ghorbani, Catholijn M. Jonker, Bernd Dudzik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-14084

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Emotion-Memory Link: Do Memorability Annotations Matter for Intelligent Systems? PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE Maria Tsfasman, Ramin Ghorbani, Catholijn M. Jonker, Bernd Dudzik Abstract Humans have a selective memory, remembering relevant episodes and forgetting the less relevant information. Possessing awareness of event memorability for a user could help intelligent systems in more accurate user modelling, especially for such applications as meeting support systems, memory augmentation, and meeting summarisation. Emotion recognition has been widely studied, since emotions are thought to signal moments of high personal relevance to users. The emotional experience of situations and their memorability have traditionally been considered to be closely tied to one another: moments that are experienced as highly emotional are considered to also be highly memorable. This relationship suggests that emotional annotations could serve as proxies for memorability. However, existing emotion recognition systems rely heavily on third-party annotations, which may not accurately represent the first-person experience of emotional relevance and memorability. This is why, in this study, we empirically examine the relationship between perceived group emotions (Pleasure-Arousal) and group memorability in the context of conversational interactions. Our investigation involves continuous time-based annotations of both emotions and memorability in dynamic, unstructured group settings, approximating conditions of real-world conversational AI applications such as online meeting support systems. Our results show that the observed relationship between affect and memorability annotations cannot be reliably distinguished from what might be expected under random chance. We discuss the implications of this surprising finding for the development and applications of Affective Computing technology. In addition, we contextualise our findings in broader discourses in the Affective Computing and point out important targets for future research efforts. I Introduction Memory for conversations and other social interactions plays a crucial role in shaping social bonds and fostering relationship building [ 1 ] . Considering human conversational memory in intelligent systems is, thus, essential for explaining and predicting human behaviour in conversations including their affective responses. Conversational memory can be defined as a sub-type of episodic memory, which manages the encoding, storage, and retrieval of personally experienced events [ 2 ] , particularly within conversational settings. Affective Computing (AC) has long focused on recognising and interpreting human emotions to enhance interactions between users and intelligent systems [ 3 ] . Emotions are considered to be central to human experience, shaping decision-making, social interactions, and memory. Their automatic detection is valuable for intelligent systems because emotional responses often signal moments of high personal relevance to users. To capture these signals, Multimodal Emotion Recognition (MER) commonly uses human behavioural cues, such as facial expressions, speech patterns, and physiological signals, to infer emotional states. While MER has made significant strides in detecting momentary affective states, its potential to model longer-term cognitive processes, such as memory, remains under-explored. Both theoretical and empirical research suggests that the way we emotionally experience events is strongly linked to how well we remember them [ 4 ] . Emotional arousal enhances memory encoding and retrieval, with emotionally charged events being remembered better than neutral ones [ 5 ] . The effect is linked to hormone release during arousing experiences, which strengthens memory formation in the brain [ 6 ] . Both valence [ 7 , 8 ] and arousal [ 9 , 6 ] , as well as their combination [ 10 ] , have been shown to enhance memory processes. Additionally, affect and memory are closely tied to the personal relevance of a stimulus, as relevance influences both emotional experience [ 11 ] and the likelihood of remembering an event [ 12 ] . Given these well-documented relationships, it is reasonable to hypothesise that perceived affect could serve as a proxy for memory. These and similar findings have been used to motivate various intelligent systems to integrate emotional components into computational memory models, e.g., to drive interactions between users and virtual agents [ 13 , 14 , 15 ] , social robots [ 16 , 17 ] , or between agents in multi-agent systems [ 18 ] . However, despite this prevalent conceptual connection between emotional responses and memorability, Affective Computing research has not yet explored this connection in the context of MER technology. Unfortunately, without targeted empirical exploration, it remains unclear to what extent theory and findings from the behavioural sciences connecting the two phenomena actually translate to many of the settings in which MER technology is developed or expected to operate. Notably, the following aspects span crucial practices of the development and deployment of MER technology but are not sufficiently covered in existing findings connecting the two phenomena: 1. Choice of Annotation Perspectives: Although research robustly links experienced emotions (measured through self-reports and physiological signals) to memory encoding, it remains uncertain whether third-party observed affect annotations, which are widely used in affective computing, can reliably serve as proxies for personal memorability (more detail in Section II-A ). 2. Continuous Conceptualisation : Previous research on the link between affect and memory operationalises those as static states, while in MER systems it is more common and desirable to view those as continuous. The link between continuous annotation of affect and memory has not been studied, to our knowledge (see Section II-B ). 3. Group-based Analysis : Group-based MER systems are crucial for real-world applications like meetings and collaborative tasks, yet existing research on the emotion-memory link primarily focuses on individuals, overlooking the social dynamics that shape both affect and memory in group interactions (see Section II-C ). Given these gaps, the extent to which third-party affect annotations capture memory-relevant information in real-world settings remains an open question. To address this question, in this paper, we present an empirical investigation evaluating the association between annotations of Perceived Group Emotions (Pleasure-Arousal) and Group Memorability. Our study leverages time-continuous annotations of emotions and memorability in dynamic, unstructured group interactions, mirroring real-world conditions relevant to MER applications like online meeting support. We discuss the implications for Affective Computing, situate our findings within broader Affective Science discussions, and highlight key directions for future research to bridge the gap between computational modelling of affect and memory. II Background and Motivation In this section, we briefly expand on some of the core properties of Affective Computing development practices that we believe could limit the insights that existing empirical findings on the emotion-memorability link can provide for the development and applicability of Multimodal Emotion Recognition (MER) technology. To support this discussion, we provide a graphical overview of the components and relationships involved in the development practices we discuss in Figure 1 . Fig. 1: An illustration of perceived vs experienced emotion and the object of this study (shown with blue dashed arrow). The states with observable information are shown in blue and the states we have no access to are shown in orange. The green dashed arrow represents the relationships that have been studied in previous literature. (Based on the component model of emotions according to Scherer (1984) [ 19 ] ) II-A Choice of Annotation Perspective As mentioned in Section I , there is substantial empirical evidence indicating a link between emotional episodes and the memorability of events. However, the findings supporting this link are largely grounded in physiological signals of emotions and self-reports of first-person experience (e.g. [

Read the complete text