Speaker Fuzzy Fingerprints: Benchmarking Text-Based Identification in Multiparty Dialogues

Rui Ribeiro, Luísa Coheur, Joao P. Carvalho · 2025

Read Speaker Fuzzy Fingerprints: Benchmarking Text-Based Identification in Multiparty Dialogues by Rui Ribeiro, Luísa Coheur, Joao P. Carvalho (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14963.txt.

Introduction

Rui Ribeiro, Luísa Coheur, and Joao Carvalho test whether a speaker can be named from chat text alone in a 2025 paper. Voice tools fail when only transcripts exist. Their approach builds a fuzzy fingerprint for each speaker from a large pre-trained model.

Speaker-specific tokens and dialogue context feed the fingerprint. Context helped a great deal: accuracy reached about seventy-one percent on a Friends data set and about sixty-eight percent on a Big Bang Theory set. The fuzzy method nearly matched full fine-tuning while using fewer hidden units, which aids interpretation. The team also studied lines that fit any speaker and proposed a way to flag them. Such flags could keep systems from forcing a wrong name.

Catalog record

CREATORRui Ribeiro, Luísa Coheur, Joao P. Carvalho
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14963

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Speaker Fuzzy Fingerprints: Benchmarking Text-Based Identification in Multiparty Dialogues Thanks: This work was supported by national funds through FCT, Fundação para a Ciência e a Tecnologia, under project UIDB/50021/2020 (DOI:10.54499/UIDB/50021/2020) and grant 2022.10640.BD, and by the Recovery and Resilience Plan (RRP) and Next Generation EU European Funds through project C644865762-00000008 Accelerat.AI. Rui Ribeiro, Luísa Coheur, Joao P. Carvalho Abstract Speaker identification using voice recordings leverages unique acoustic features, but this approach fails when only textual data is available. Few approaches have attempted to tackle the problem of identifying speakers solely from text, and the existing ones have primarily relied on traditional methods. In this work, we explore the use of fuzzy fingerprints from large pre-trained models to improve text-based speaker identification. We integrate speaker-specific tokens and context-aware modeling, demonstrating that conversational context significantly boosts accuracy, reaching 70.6% on the Friends dataset and 67.7% on the Big Bang Theory dataset. Additionally, we show that fuzzy fingerprints can approximate full fine-tuning performance with fewer hidden units, offering improved interpretability. Finally, we analyze ambiguous utterances and propose a mechanism to detect speaker-agnostic lines. Our findings highlight key challenges and provide insights for future improvements in text-based speaker identification. I Introduction Speaker identification from voice recordings has been a widely active topic in the Speech Recognition area [ 1 , 2 , 3 , 4 ] , where the goal is to identify the speaker by using the information contained in the speaker’s speech signal. When speaking, an interlocutor comprises features such as the properties of the speaking style that are proper of their idiosyncratic attributes and can be exploited as acoustic features for speaker recognition models. However, when speech is transcribed into text, these acoustic features are lost, removing valuable speaker-specific cues. This limitation makes it significantly harder to distinguish speakers based solely on linguistic patterns. Furthermore, in many real-world scenarios, only textual data is available, such as in movie scripts, chat logs, and historical records, necessitating effective speaker identification methods that rely purely on text. Only a few approaches have attempted to identify the speaker given textual conversational data. For instance, Ma et al. [ 5 ] trained a Convolutional Neural Network (CNN) to identify textual speakers from multiparty dialogues extracted from the Friends TV Show, demonstrating that distinct speaker styles can be captured from text alone and achieving significant improvements over traditional methods. However, with recent advancements in Natural Language Processing, these results are no longer state-of-the-art. Beyond CNN-based approaches, other methodologies have also been explored. Kundu et al. [ 6 ] applied traditional machine learning techniques to detect speaker boundaries and changes in automatically transcribed dialogues, while Su and Zhou [ 7 ] proposed a speaker clustering model that groups utterances without explicit speaker annotations. While these methods highlight the feasibility of textual speaker identification, the task remains inherently difficult due to the frequent use of generic and short utterances and the stylistic overlap between speakers. In this work, we explore new mechanisms to enhance speaker identification by better capturing speaker-specific linguistic patterns and contextual dependencies. To this end, we integrate speaker-specific tokens and context-aware modeling to improve identification performance and reduce ambiguity. Additionally, to ensure better generalization across different domains, we explore a new corpus derived from the Big Bang Theory TV series, complementing existing datasets. Beyond improving classification accuracy, we also investigate the use of the Fuzzy Fingerprint framework to detect ambiguous or generic utterances that lack strong speaker-specific cues. Our contributions to textual speaker identification include: • We explore a new corpus of textual dialogues from the Big Bang Theory that complements the Friends dataset, expanding the range of conversational styles. • We benchmark both datasets 1 1 1 https://github.com/ruinunca/speaker-fuzzy-fingerprints using a recent technique that leverages fuzzy fingerprints from language models (specifically RoBERTa variants), analyzing the impact of conversational context length. • We explore the use of the Fuzzy Fingerprint framework to reduce the number of hidden units while maintaining performance comparable to fully fine-tuned models, as well as to investigate the impact of speaker-agnostic utterances. II Related Work Although many approaches to speaker recognition have traditionally relied on acoustic characteristics extracted from speech signals [ 1 , 2 , 3 , 4 ] , text-centered methods are receiving growing attention for scenarios where audio data are unavailable. One of the earlier efforts to leverage purely textual cues focused on identifying speaker shifts in automatically transcribed dialogs. For instance, Kundu et al. [ 6 ] employed standard machine learning algorithms (such as K-Nearest Neighbors and Naive Bayes) to detect speaker transitions. Ma et al. [ 5 ] devised a CNN tailored for speaker classification in multiparty interactions drawn from the Friends TV series. More recently, Su and Zhou [ 7 ] proposed an unsupervised paradigm that clusters utterances without relying on speaker labels. By integrating a pre-trained language model at the utterance level with a pairwise similarity matrix, the authors demonstrated that natural groupings of utterances can accurately align with speaker identities, further establishing the effectiveness of textual features in speaker recognition tasks. Beyond conventional machine learning and deep learning approaches, alternative representation techniques may provide complementary advantages for text-based speaker identification. One such method is the Fuzzy Fingerprint framework, which has been utilized as an efficient technique for generating compact and distinctive representations of large datasets [ 8 ] . This method has demonstrated its utility in various applications, including authorship attribution, topic classification, and emotion detection [ 9 , 10 , 11 ] . Fuzzy fingerprints are formed by accumulating feature activations (e.g., word occurrences or neural embeddings) from all training samples of a particular class, ordering these features from highest to lowest occurrence, and retaining the most salient subset as the class’s core representation. In the end, a fuzzy fingerprint is a fuzzy set in the discrete universe of the used features [ 12 ] . III Corpora III-A Friends Corpus We use the Friends dataset introduced by [ 13 ] . Each season contains multiple episodes, and each episode is comprised of separate scenes. The scenes in an episode are divided into turns, containing the annotation of the speakers. O RO RA J C P M 0 5 10 15 20 16.29 15 15.27 13.42 13.89 12.3 13.84 Character Relative Frequency (%) Fig. 1 : Distribution of Turns per Speaker for the Friends Corpus (O - Other; RO - Ross; RA - Rachel; J - Joey; C - Chandler; P - Phoebe; M - Monica). Statistic Train Valid Test Total Scenes 2268 332 288 Total Turns 43799 6343 6231 Mean Sentence Length 10.18 9.92 10.17 Std Deviation of Sentence Length 10.59 10.16 10.32 Mean Scene Length 19.31 19.11 21.64 Std Deviation of Scene Length 15.74 14.04 18.06 TABLE I : Statistics of train, validation, and test splits of the Friends dataset. In total, this corpus consists of 3,107 scenes and 61,676 turns. The distribution of utterances per speaker is illustrated in Figure 1 . We only consider the 6 main characters for the labels, while the other characters are considered as the Other label. The percentages for major speakers are fairly consistent; however, the Other speaker category has a larger percentage in the dataset than any individual major speaker. Following [ 5 ]

Read the complete text