FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents

Satu, Tomi, Alexandre, Rosa, Lauri, Einar · 2025

Read FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents by Satu, Tomi, Alexandre, Rosa, Lauri, Einar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08981.txt.

Introduction

The authors introduce a speech data set that records tongue and lip motion, in this 2025 paper. Eighteen bilingual speakers of Finnish and Russian read material in their native tongue, in their second language, and in an imitated foreign accent. Sensors tracked how their mouth parts moved.

Two case studies show the range of the data. One tests how second-language and fake accents affect a speaker-check system. The other maps one speaker's mouth patterns across the three modes. The corpus is meant for work on accent, speech science, and speaker technology, where fine motion data are rare.

Catalog record

CREATORSatu, Tomi, Alexandre, Rosa, Lauri, Einar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08981

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents Satu, Tomi, Alexandre, Rosa, Lauri, Einar Abstract We introduce a new FROST-EMA (Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography) corpus. It consists of 18 bilingual speakers, who produced speech in their native language (L1), second language (L2), and imitated L2 (fake foreign accent). The new corpus enables research into language variability from phonetic and technological points of view. Accordingly, we include two preliminary case studies to demonstrate both perspectives. The first case study explores the impact of L2 and imitated L2 on the performance of an automatic speaker verification system, while the second illustrates the articulatory patterns of one speaker in L1, L2, and a fake accent. Hopponen Kinnunen Nikolaev González Hautamäki Tavi Meister University of Eastern FinlandFinland University of Eastern FinlandFinland University of OuluFinland Forensic Laboratory of National Bureau of InvestigationFinland Tallinn University of TechnologyEstonia keywords articulatory phonetics, speaker recognition, accented speech, electromagnetic articulography, Finnish language, Russian language † † email: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected] 1 Introduction Humans produce speech by coordinating the movements of various articulators , such as the tongue and the lips. Humans continuously reshape their vocal tract to modulate the airstream produced by the voice source, the larynx. The resulting acoustic waveform, commonly captured with a microphone, contains latent (i.e. not directly observable) information of articulatory kinematics. A number of alternative techniques to track articulatory movements are available, including x-ray [ 1 ] , ultrasound [ 2 ] , magnetic resonance imaging [ 3 ] , and electromagnetic articulography (EMA). In this study, we focus on EMA, which is a 3D measurement technique that tracks the movements of sensor coils attached to the articulators with an adhesive. Detailed analysis of articulation, which EMA facilitates, is relevant to the study of speech variability. The human vocal tract is highly flexible and can accommodate a wide variation in languages, accents, and speaking styles, to name but a few. One interesting phenomenon is foreign accent . Second language (L2) learners often apply phonemic substitution patterns from their first language (L1), resulting in speech that blends characteristics of both L1 and L2 [ 4 ] . Analyzing the articulatory mechanisms behind L1 and L2 speech production can help identify challenges faced by second language learners. Sometimes fake foreign accents are also produced on purpose in movies, stand-up comedy, and social media. From a linguistic point of view, understanding the production of successfully imitated L2 accents can provide novel insights into accent stereotypes. The new corpus that we introduce specifically addresses variability in the first language (L1), second language (L2) and imitated L2 (fake foreign accent) in bilingual speakers , who are Finnish and Russian speakers with varying degrees of L2 proficiency. The phonetic premises—including the reasons for the specific language choices—are provided in Section 2.1 . Importantly, EMA is not relevant only to phonetic sciences but for the development of novel educational and clinical speech applications. In particular, parallel acoustic and articulatory recordings enable articulatory inversion [ 5 ] , which is the task of estimating articulation movements from the acoustic data, and articulatory synthesis [ 6 ] , which is the task of generating speech based on articulatory parameters. Articulation can also help [ 7 ] to shine a light on the weak points of automatic speaker recognition systems, when faced with purposefully disguised voices [ 8 ] . This is a pressing issue in forensic voice comparison. Another promising use of EMA is its potential in prior kinematic speech knowledge in specific models, which can be utilized in relaxing the acoustic training data requirements. For instance, [ 9 ] utilized EMA data as part in a brain-to-speech synthesis to address data limitations. Another interesting study [ 10 ] used linear probing to study the association of EMA coordinate values with the latent features extracted from various foundational speech models. Their results had remarkably high Pearson correlations. Some foundational models appear to encode articulatory information even if they were not trained to do so. Recent articulatory inversion approaches already use foundational speech models to generate robust features for inversion tasks. The above studies demonstrate the potential role of articulation as a bridge between speech science and technology. Despite its relevance to both fundamental and applied speech research, the currently available EMA resources are surprisingly scarce. Most of them (Table 1 ) are in English and contain only a handful of subjects. To foster research in both speech science and technology, we introduce a new FROST-EMA database ( Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements ). The novelties, compared to prior publicly available EMA corpora, are: (1) the two languages (Finnish and Russian) are not included in existing EMA databases 1 1 1 Apart from our early pilot [ 7 ] , which contained only 4 speakers and was collected with an older EMA device. ; (2) the inclusion of bilingual speakers who all produce L1, L2, and imitated L2 accents; (3) an overall larger number of subjects compared to existing corpora; (4) the use of a state-of-the-art EMA device with a higher sampling rate. The first two very particular constraints, combined with the practical difficulties in recruiting volunteers for invasive and time-consuming EMA recordings, makes the compilation a corpus a tedious task. The study has been reviewed and approved by the Ethics Board of the authors’ institute. While this Interspeech 2025 paper provides a snapshot of an ongoing data collection effort, our plan is to release the FROST-EMA corpus to the community by the first half of 2026 at the latest. 2 The collection of FROST-EMA dataset 2.1 Phonetic basis for the collected data Finnish and Russian, though geographically adjacent, belong to different language families—Finno-Ugric and Indo-European [ 11 ] , respectively—and exhibit distinct phonetic and phonological characteristics (for Finnish, see [ 12 ] and for Russian, see [ 13 ] ). Finnish features a larger vowel inventory with a long-short vowel distinction, whereas Russian has an average vowel inventory without such a contrast. In terms of consonants, Russian possesses a broader set of phonemes, while Finnish distinguishes between single and geminate consonants. Additionally, Finnish employs a fixed stress pattern, whereas Russian has variable stress. These contrasting phonetic properties provide a rich framework for investigating both natural and imitated accents. Table 1: Available EMA datasets \eightpt Corpus Language Speakers f_{s} (Hz) Sensors # Utterances/speaker Conds Availability MOCHA-TIMIT [ 14 ] ENG 1♀, 1♂ 500 8 460 L1 Free mngu0 [ 15 ] BrE 1 200 6 \sim 1300 L1 Upon request USC-TIMIT [ 3 ] AmE 2♀, 2♂ 100 6 460 L1 Upon request EMA-MAE [ 16 ] ENG 10♀10♂L1 ENG, 400 7 45 mins L1 ENG Upon request 10♀10♂L1 CMN L2 CMN accent MSPKA [ 17 ] Italian 1♂, 2♀ 400 7 Varies; 500, 629, 666 Conditions Free DKU-JNU-EMA [ 18 ] 3 CMN dialects 9♂, 7♀ NA 6 \sim 0.66 hours L1 dialect Upon request FROST-EMA [this paper] FIN, RUS 8♀, 10♂ 1250 7 L1 RUS 227, L1 FIN 223 L1,L2,fake-L2 Free CMN refers to Mandarin Chinese. Only active sensors are included in the sensor column. 2.2 Materials In our study,

Read the complete text