Analysis of ABC Frontend Audio Systems for the NIST-SRE24

Sara, Anna, Ladislav, Junyi, Oldřich, Johan, Lin, Jiangyu, Petr, Federico, Lukáš, Themos, Sandro, Dominik, Miroslav, Martin, Tomaš · 2025

Read Analysis of ABC Frontend Audio Systems for the NIST-SRE24 by Sara, Anna, Ladislav, Junyi, Oldřich, Johan, Lin, Jiangyu, Petr, Federico, Lukáš, Themos, Sandro, Dominik, Miroslav, Martin, Tomaš (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15320.txt.

Introduction

Barahona and co-authors analyze the ABC team audio front ends for a 2025 speaker recognition contest. A front end turns a voice clip into a compact vector that captures who is speaking. The contest, run by NIST, set two training rules: a fixed track using only given phone recordings, and an open track that allows public data.

The team explored network designs with varied pooling, a newer network shape, and a large pre trained speech model. In the open track they trained on a data set with 110 thousand speakers in many languages. Those models proved strong and robust. The paper distills workable recipes for building top speaker front ends.

Catalog record

CREATORSara, Anna, Ladislav, Junyi, Oldřich, Johan, Lin, Jiangyu, Petr, Federico, Lukáš, Themos, Sandro, Dominik, Miroslav, Martin, Tomaš
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15320

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Analysis of ABC Frontend Audio Systems for the NIST-SRE24 Sara, Anna, Ladislav, Junyi, Oldřich, Johan, Lin, Jiangyu, Petr, Federico, Lukáš, Themos, Sandro, Dominik, Miroslav, Martin, Tomaš Abstract We present a comprehensive analysis of the embedding extractors (frontends) developed by the ABC team for the audio track of NIST SRE 2024. We follow the two scenarios imposed by NIST: using only a provided set of telephone recordings for training (fixed) or adding publicly available data (open condition). Under these constraints, we develop the best possible speaker embedding extractors for the pre-dominant conversational telephone speech (CTS) domain. We explored architectures based on ResNet with different pooling mechanisms, recently introduced ReDimNet architecture, as well as a system based on the XLS-R model, which represents the family of large pre-trained self-supervised models. In open condition, we train on VoxBlink2 dataset, containing 110 thousand speakers across multiple languages. We observed a good performance and robustness of VoxBlink-trained models, and our experiments show practical recipes for developing state-of-the-art frontends for speaker recognition. Barahona Silnova Mošner Peng Plchot Rohdin Zhang Han Palka Landini Burget Stafylakis Cumani Boboš Hlavaček Kodovsky Pavliček Universidad Autónoma de Madrid,Spain 2 Brno University of TechnologyCzechia 3 Athens University of Economics and Business | Omilia | Archimedes AI/Athena RCGreece 4 Politecnico di Torino, Italy 5 PhonexiaCzechia keywords speaker recognition, NIST-SRE, embedding extractors, VoxBlink † † email: [email protected], {isilnova, imosner, pengjy, iplchot, rohdin}@fit.vut.cz 1 Introduction The major body of speaker verification (SV) research focuses on 16 kHz datasets with audio excerpts extracted from (e.g., YouTube) video clips. The VoxCeleb datasets [ 1 , 2 ] represented a great milestone in pushing the verification performance forward by providing training data comprising thousands of speakers. This initiative has been recently followed by other works, releasing large-scale corpora comprising tens of thousands of speakers [ 3 ] and fostering research in the aforementioned domain. On the other hand, speaker verification in telephony (characterized by various codecs and 8 kHz sampling rate), which is crucial in many real-world applications, seems to be somewhat overlooked in the community. Fortunately, NIST Speaker Recognition Evaluations have consistently pushed the boundaries of speaker recognition technology for decades, focusing especially on challenges related to detecting speakers in noisy conversational telephone speech (CTS) (but also audio from video, AfV, and multi-modal data). The latest edition (SRE24) was no exception. It included a mandatory audio-only track, as well as optional visual-only and audio-visual tracks that require the integration of speech, image, and video data for speaker identification. The main audio track deals with cross-source (CTS vs. AfV) and cross-lingual target and non-target trials. They were drawn from a new multilingual corpus, TELVID-Tunis. As the name suggests, it focuses on non-mainstream languages: Tunisian Arabic, North African French, and accented English. On top of non-standard languages, this latest edition included some novelties such as enrollment duration variability (10, 30, or 60 seconds), shorter test segments (ranging approximately from 5 seconds to 60 seconds), and multi-speaker enrollment data with diarization annotations. The evaluation proposes two training paradigms: fixed and open. The fixed condition constrains participants to use only organizer-provided datasets, including a CTS superset [ 4 ] , NIST SRE 2016 [ 5 ] and 2021 Evaluation [ 6 ] , and Janus Multimedia set [ 7 ] . Such a constraint poses challenges as most of the data comprises CTS only (while the evaluation data contains both CTS and AfV). Moreover, the majority of recordings of the main training data, CTS Superset, are spoken in English (despite containing more than 50 languages), overweighting one of the evaluation languages. In contrast, the open condition allows participants to explore how unlimited additional training data affects system performance. In this paper, our goal is to build strong frontends for speaker verification following the NIST SRE24 rules. We strive to achieve this by the following means: • Variability of frontends : As per empirical evidence detailed in [ 8 ] , diverse frontends tend to be complementary, which provides benefits in fusion. Therefore, we intend to explore ResNet architectures [ 9 ] that have proven strong for speaker embedding extraction in numerous evaluations [ 10 , 11 ] and challenge the recent model, ReDimNet [ 12 ] , that was shown to provide state-of-the-art results on the VoxCeleb trial lists and promised strong generalizability. • Alternative pooling method : Inspired by consistent improvements in results on the previous NIST evaluation data provided by considering the uncertainty of estimates, we use the xi-vector pooling style within embedding extractors [ 13 ] . • Large-scale out-of-domain dataset : As noted before, advances in speaker verification on audio data from videos lead to collecting large-scale datasets. We aim to explore and exploit one of them, VoxBlink2 [ 3 ] , in the non-constrained track. Moreover, we will focus on how such out-of-domain datasets can be beneficially used in the context of various applications. • SSL model : Foundational models trained in a self-supervised way also attracted attention in speaker verification by providing strong results while requiring a shorter time to fine-tune (compared to training embedding extractors from scratch) [ 14 ] . Therefore, we aim to explore them in the challenging data. 2 Proposed method 2.1 Training data and augmentations For the fixed condition, we used the NIST CTS Superset [ 4 ] to train the embedding extractors. To enhance the robustness of the model, we implemented acoustic data enhancement using the Kaldi toolkit [ 15 ] , incorporating noise from the MUSAN database [ 16 ] and room impulse responses from the RIR database [ 17 ] . However, we specifically excluded MUSAN’s Babble noise and Music components to comply with fixed track regulations. Prior to model training, non-speech segments were removed through a Kaldi-style energy-based VAD Voice Activity Detection (VAD) system. The open condition, free from data restrictions, presents an opportunity to leverage large out-of-domain datasets. The largest publicly available speaker verification dataset to date is VoxBlink2 [ 3 ] , which consists of audios from YouTube videos belonging to 111,284 speakers, significantly surpassing the widely used VoxCeleb [ 2 ] . However, since this data does not inherently match the CTS domain, we experimented with downsampling the audio to 8kHz while applying GSM codec to 50% of the data via Sox 1 1 1 https://sourceforge.net/projects/sox/ aiming to simulate the telephone channel. 2.2 Frontend systems In this section, we present the different embedding extractors explored for facing both fixed and open conditions. For reproducibility, we also describe the training setup, implemented using the WeSpeaker toolkit [ 18 , 19 ] . Most of our embedding extractors follow the VoxCeleb recipe, employing all the suggested hyperparameters for the training, which consists of two stages, both aimed at minimizing the AAM-Softmax loss [ 20 ] . The first stage involves training for 150 epochs with a 2-second segment length and employing a scale of 32 for the AAM loss. Initially, no margin is applied, but between epochs 20 and 40, the margin is gradually increased from 0 to 0.2, and this value is maintained for the remainder of the training. The learning rate scheduler uses a 6-epoch warm-up, linearly increasing the rate from 0 to its highest value (0.1) and then exponential decrease to 5e-5

Read the complete text