Identifying Primary Stress Across Related Languages and Dialects with Transformer-based Speech Encoder Models
Nikola, Ivan, Peter · 2025
Read Identifying Primary Stress Across Related Languages and Dialects with Transformer-based Speech Encoder Models by Nikola, Ivan, Peter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24571.txt.
Introduction
Nikola Ljubesic, Ivan Porupski, and Peter Rupnik automate word stress detection across related tongues in a 2025 paper. Stress, the beat placed on one syllable, carries meaning and aids understanding. Older tools leaned on hand-built sound features and mostly English data.
The team fine-tunes a pretrained speech transformer to label each audio frame. Training uses a new Croatian data set, with tests in Serbian, a Croatian dialect, and Slovenian. The transformer beats a classic classifier on every test set. Scores come near perfect for Croatian and Serbian, with a modest drop for the more distant tongues. A few hundred training words already give strong results. Data and models are released openly.
Catalog record
| CREATOR | Nikola, Ivan, Peter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24571
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Identifying Primary Stress Across Related Languages and Dialects with Transformer-based Speech Encoder Models Nikola, Ivan, Peter Abstract Automating primary stress identification has been an active research field due to the role of stress in encoding meaning and aiding speech comprehension. Previous studies relied mainly on traditional acoustic features and English datasets. In this paper we investigate the approach of fine-tuning a pre-trained transformer model with an audio frame classification head. Our experiments use a new Croatian training dataset, with test sets in Croatian, Serbian, the Chakavian dialect, and Slovenian. By comparing an SVM classifier using traditional acoustic features with the fine-tuned speech transformer, we demonstrate the transformer’s superiority across the board, achieving near-perfect results for Croatian and Serbian, with a 10-point performance drop for more distant Chakavian and Slovenian. Finally, we show that only a few hundred multi-syllabic training words suffice for strong performance. We release our datasets and model under permissive licenses. Ljubešić Porupski Rupnik Jožef Stefan InstituteSlovenia University of LjubljanaSlovenia LjubljanaSlovenia keywords primary stress detection, pre-trained encoder models, South Slavic languages † † email: [email protected], [email protected], [email protected] 1 Introduction Primary stress is a feature of each multi-syllabic word, where one syllable is perceived to stand out from its environment [ 1 ] . It has an important and varying function in different languages, including distinguishing word meaning and function, aiding speech comprehension, as well as communicating various sociolinguistic cues [ 2 , 3 , 4 ] . Automating the identification of the position of primary stress has attracted significant amount of previous work. Traditionally, the task was performed via supervised learning over acoustic features, while recently transformers pre-trained on speech started to be used, but only for probing experiments [ 5 ] and feature extraction [ 6 ] . Most of the intended use cases and datasets are related to computer-assisted language learning [ 1 , 7 ] , speech synthesis [ 8 ] , and some applications in research of children’s speech [ 9 , 10 ] . The vast majority of work has been performed on English, with very infrequent exceptions such as German [ 11 ] and Arabic [ 12 ] . In this work, we investigate the capacity of pre-trained speech transformer models to predict the position of the primary stress in a multi-syllabic words. We achieve that by fine-tuning the transformer network on that task. Our imminent goal is to apply the resulting technology to describe the variation in spoken language by annotating large spoken corpora, but we also plan more specific downstream use cases, including child language and atypical speech processing, as well as language learning. We break from the tradition of English-centric research by building one training and four test sets in various South Slavic languages and dialects. With this setup, we also investigate the transferability of the proposed technology to similar languages and dialects. Our training and one of the test languages is Croatian, particularly interesting for the task due to its high dialectal variability, which results in a varying position of the stress even in official communication. Croatian spreads across the Shtokavian, the Kajkavian and the Chakavian dialectal group. Serbian, our test language, mutually intelligible with standard Croatian, has less variability due to its dominant dialect, Shtokavian, which was used in the standardization of both Croatian and Serbian. Slovenian, another of our test languages, is standardized somewhat closely to the Croatian Kajkavian dialect. Finally, our test dialect is the Chakavian dialect, not present in the Croatian standard, nor as close to the Slovenian standard as Kajkavian [ 3 , 4 , 13 ] . Our primary contributions are the following: (1) we show excellent performance of the pre-trained speech transformers fine-tuned to the task of primary stress identification, (2) we investigate the limitations of the technology when applied to related languages and dialects, (3) we show that supervision of pre-trained models on a few hundred words already yields comparable results to those obtained after fine-tuning the model on ten thousand words, (4) we release a new training dataset and four test sets for related South Slavic languages and dialects, as well as a strong model for Croatian and Serbian 1 1 1 Data and model are available at https://doi.org/10.57967/hf/5658 . . 2 Related work The traditional way of performing primary stress identification has been to use prosodic features, such as nucleus duration, intensity and pitch (F0) [ 1 , 7 , 12 ] , as well as the sonority-based TCSSBC feature contour [ 14 , 15 ] . These features are mostly exploited in a supervised machine learning setup, with infrequent takes on unsupervised approaches [ 16 ] . Most approaches use pre-neural classifiers such as Gaussian Mixture Models [ 1 , 8 , 7 ] , Hidden Markov Models [ 17 ] and Support Vector Machines [ 8 , 15 ] , with some neural exceptions [ 9 , 10 , 12 ] . The performance reported from these experiments on various datasets ranges from 72 to 93% word-level accuracy. Recently, pre-trained speech transformer models have been applied on the task as well, but only either for probing these networks for primary stress signal [ 5 ] , or for feature extraction for various traditional and neural classifiers [ 6 ] . The probing experiments in [ 5 ] report promising results for the availability of the primary stress signal in neural representations, showing that CNN feature extractors already contain a level of relevant signal similar to traditional acoustic features, with higher transformer layers being significantly more predictive of the phenomenon. The classification experiments in [ 6 ] compare traditional acoustic features with syllable-averaged neural representations from pre-trained speech transformers, with neural representations being more informative for the task. These experiments also compare traditional and neural classifiers, showing that the latter are more potent on the task. They, however, miss on the opportunity to fine-tune the transformer model to the task directly, which also introduces information loss while averaging the neural representations, available otherwise for each \SI 20 \milli \second frame, over the span of each syllable. 3 Data 3.1 Sources We construct new training and test datasets by exploiting recently released open datasets in three South Slavic languages and one dialect. The Croatian ParlaStress-HR training and test datasets are constructed from a sample of the ParlaSpeech-HR open dataset of sentence-aligned parliamentary recordings and transcripts of the Croatian parliament [ 18 ] . Transcription sentences are sampled to assure the diversity of speakers and gender balance. We split the dataset into a training portion and a test portion, ensuring no speaker overlap while maintaining gender balance. The Serbian ParlaStress-SR test set is built from the ParlaSpeech-RS dataset, another member of the ParlaSpeech collection of speech and text datasets [ 18 ] , sampling transcript sentences to ensure maximum diversity of speakers, while ensuring gender balance. The Chakavian MićiPrinc-CKM test dataset is a sample of two chapters from the printed and audio book of the translation of Le Petit Prince into the Chakavian dialect. This multimodal book has recently been released as an open dataset with word-level-aligned text and audio [ 19 ] . The Slovenian Artur-SL test set is sampled from the ARTUR dataset [ 20 ] ,