TED: Turn Emphasis with Dialogue Feature Attention for Emotion Recognition in Conversation

Junya Ono, Hiromi Wakaki · 2025

Read TED: Turn Emphasis with Dialogue Feature Attention for Emotion Recognition in Conversation by Junya Ono, Hiromi Wakaki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01123.txt.

Introduction

Junya Ono and Hiromi Wakaki improve emotion detection in conversations in a 2025 paper. Knowing who feels what in a dialogue helps chatbots and analysts, but models tend to blur the turns together, marking speakers and order only with crude tokens. Their method, TED, writes turn information directly into the attention mechanism.

Each turn gets a priority from its position and its speaker, and attention scores are adjusted by those dialogue features. Multi-head attention then runs over turn-level vectors. On four standard benchmarks the method performed strongly overall, and it set a new best score on IEMOCAP, a data set with long conversations. The authors argue explicit turn structure beats implicit hints.

Catalog record

CREATORJunya Ono, Hiromi Wakaki
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-01123

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TED: Turn Emphasis with Dialogue Feature Attention for Emotion Recognition in Conversation Junya Ono, Hiromi Wakaki Abstract Emotion recognition in conversation (ERC) has been attracting attention by methods for modeling multi-turn contexts. The multi-turn input to a pretraining model implicitly assumes that the current turn and other turns are distinguished during the training process by inserting special tokens into the input sequence. This paper proposes a priority-based attention method to distinguish each turn explicitly by adding dialogue features into the attention mechanism, called T urn E mphasis with D ialogue (TED). It has a priority for each turn according to turn position and speaker information as dialogue features. It takes multi-head self-attention between turn-based vectors for multi-turn input and adjusts attention scores with the dialogue features. We evaluate TED on four typical benchmarks. The experimental results demonstrate that TED has high overall performance in all datasets and achieves state-of-the-art performance on IEMOCAP with numerous turns. 1 Introduction Emotion recognition in conversation (ERC) has been discussed for over decade Yang et al. 2007 ; Devlin et al. 2019 . In ERC, an emotion label is estimated for the current turn from multiple turns’ worth of utterances and speaker information. Emotion understanding such as ERC has the potential to be used in chatbots, medical situations, and call centers. The methods of ERC are often used on the basis of past and future contexts, external commonsense knowledge, and speaker information Majumder et al. 2019 ; Ghosal et al. 2020 ; Zhu et al. 2021 ; Shen et al. 2021 . Especially, studies that involve modeling multiple turns have grown in number since the appearance of ERC datasets that embody different perspectives including multiple speakers and topics Li et al. 2017 ; Zahiri and Choi 2017 ; Poria et al. 2018 . Figure 1: Concept of our method. This shows the difference in the multi-turn input in related methods. Our method prioritizes each turn to distinguish the turns The mainstream multi-turn methods can exploit past and future turns for more contexts through their use of recurrent neural networks (RNNs), graph neural networks (GNNs), and Transformer Vaswani et al. 2017 . These methods often input the token sequence for only the current turn into a pretrained model, such as BERT Devlin et al. 2019 and RoBERTa Liu et al. 2019 . They cannot use the context other than the current turn for single-turn input. Therefore, the methods of multi-turn input have been recently proposed by inserting special tokens into the input sequence as shown in Figure 1 Li et al. 2020a ; Gu et al. 2020 ; Lee and Choi 2021 ; Kim and Vossen 2021 . These methods can obtain deeper contexts by adding the utterances in multiple turns into the input sequence. On the other hand, they design to distinguish each turn by including information of turn position and speaker as special tokens. However, this multi-turn input implicitly expects to be distinguished between the current turn and other turns to be modeled in the process of machine learning. Figure 2: Example of ERC and dialogue features. TED uses current (C), past (P) and future (F) turns to obtain more contexts. Dialogue features indicate turn priority, same speaker (S) and listener (L). In this case, TED adjusts attention scores for the same speakers (S) in the current turn by using an attention factor \beta^{t} with the turn priority; {t} indicates the turn number; SID indicates the speaker identification. We introduce a concept to distinguish explicitly between each turn and to control a degree of the distinction while making the most of dialogue features. As one of the methods, 1 shows our concept to clarify the distinction by prioritizing each turn and maximizing the priority of the current turn in comparison with related methods Li et al. 2020a ; Kim and Vossen 2021 . This concept uses special tokens of [TURN] and [SEP] as turn breaks and delimiter of the current turn, respectively. Moreover, it adjusts relationship between the current turn and other turns by using priority factor \beta^{t} calculated by dialogue features of turn position and speaker information; {t} indicates position of a turn; SPK indicates Speaker information. Our proposal, a priority-based attention method, called “ T urn E mphasis with D ialogue (TED)”, uses the above dialogue features. First, TED creates a multi-turn input sequence by concatenating all turns with special tokens and also creates a turn-based vector by averaging the token-based vectors at the same positions of the utterance in a turn, called “ T urn- B ased E ncoding (TBE)”. Second, it establishes multi-head self-attention (MHSA) between the turn-based vectors of TBE, called “ T urn- B ased M HSA (TBM)”. Finally, it adds a dialogue layer to TBM to adjust attention scores for three types of all turns, the same speaker turns at the same as a current turn, or the listener turns. Figure 2 shows an example of making adjustments with an attention factor \beta^{t} ; {t} indicates the turn number. The comprehensive results in four typical benchmarks demonstrate that TED has high performance in all datasets and especially archives state-of-the-art performance in IEMOCAP with a lot of turns. 2 Related work ERC has recently attracted attention because it can handle more complex contexts using multiple turns. Numerous powerful approaches have been proposed on the basis of RNNs, GNNs, and Transformers. Many use a pretrained model, such as BERT and RoBERTa, to make sequence representations corresponding to input tokens. 2.1 Multi-turn models Here, we review recent models based on three different neural networks. RNN-based models ICON Hazarika et al. 2018a and CMN Hazarika et al. 2018b have gated recurrent unit (GRU) and memory networks. HiGRU Jiao et al. 2019 has two GRUs for the utterance and conversation. BiF-AGRU Jiao et al. 2020 has a hierarchical memory network with an attention GRU for historical utterances. DialogueRNN Majumder et al. 2019 utilizes the global state as a context and the party state for individual speakers by incorporating bidirectional GRUs for emotional dynamics. COSMIC Ghosal et al. 2020 has a similar structure to DialogueRNN but with added external commonsense knowledge. BiERU Li et al. 2020b devised an efficient and party-ignorant framework by using a bi-recurrent unit. CESTa Wang et al. 2020 handles the global context by using Transformer and individual speakers by using BiLSTM-CRF. DialogueCRN Hu et al. 2021 has contextual reasoning networks that have long short-term memory (LSTM) to understand situations and speaker context. GNN-based models DialogGCN Ghosal et al. 2019 handles the dependency and positional relationship of speakers as a graph structure. RGAT Ishiwatari et al. 2020 has a similar strategy to DialogGCN but with added positional encodings. ConGCN Zhang et al. 2019 builds an entire dataset including utterances and speakers as a large graph. SumAggGIN Sheng et al. 2020 has two stages of summarization and aggregation graphs for capturing emotional fluctuation. DAG-ERC Shen et al. 2021 models the flow between long distance and nearby contexts. TUCORE-GCN Lee and Choi 2021 constructs a graph of relational information in a dialogue with four types of nodes and three types of edges. Figure 3: Turn-based encoding (TBE) model. CUST outputs a multi-turn sequence from utterances in past and future turns with “TURN”

Read the complete text