ASR Under Noise : Exploring Robustness for Sundanese and Javanese

Salsabila Zahirah Pranida Muhammad Cendekia Airlangga Rifo Ahmad Genadi, Shady Shehata · 2025

Read ASR Under Noise : Exploring Robustness for Sundanese and Javanese by Salsabila Zahirah Pranida Muhammad Cendekia Airlangga Rifo Ahmad Genadi, Shady Shehata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25878.txt.

Introduction

Salsabila Zahirah Pranida and four co-authors tested speech recognition for two Indonesian languages under noise. Their paper, from twenty twenty-five, deals with Javanese and Sundanese. Whisper based models do well on clean speech for these tongues. How they fare when noise creeps in was not clear. The team tried many training plans, from fake noise mixes to a mask based trick called SpecAugment.

They scored the models across a range of signal to noise levels. Noise aware training lifted strength by a lot, most of all for larger models. A close read of errors showed flaws that split by language. Readers who build speech tools for low resource tongues get a map of what noise costs and what training buys back.

Catalog record

CREATORSalsabila Zahirah Pranida Muhammad Cendekia Airlangga Rifo Ahmad Genadi, Shady Shehata
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25878

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ASR Under Noise : Exploring Robustness for Sundanese and Javanese Salsabila Zahirah Pranida Muhammad Cendekia Airlangga Rifo Ahmad Genadi, Shady Shehata Abstract We investigate the robustness of Whisper-based automatic speech recognition (ASR) models for two major Indonesian regional languages: Javanese and Sundanese. While recent work has demonstrated strong ASR performance under clean conditions, their effectiveness in noisy environments remains unclear. To address this, we experiment with multiple training strategies, including synthetic noise augmentation and SpecAugment, and evaluate performance across a range of signal-to-noise ratios (SNRs). Our results show that noise-aware training substantially improves robustness, particularly for larger Whisper models. A detailed error analysis further reveals language-specific challenges, highlighting avenues for future improvements. Code is available at https://github.com/rifoagenadi/robust_jvsu_asr . 1 Introduction Automatic Speech Recognition (ASR) systems have made remarkable progress in recent years, especially for high-resource languages like English. While modern ASR handles diverse accents ( Rao and Sak 2017 ) and noise ( Seltzer et al. 2013 ) in high-resource languages, it remains unreliable for low-resource ones. Indonesia, with 284M people and over 700 languages, is among the world’s most linguistically diverse countries ( Badan Pusat Statistik 2025 ; Eberhard et al. 2025 ; PetaBahasa 2019 ; BPS 2024 ) . Yet, both remain underrepresented in ASR research and resources. These languages exhibit high dialectal variation and are spoken daily in uncontrolled, noisy settings, which makes them difficult for standard ASR models, which are mostly trained on Indo-European data ( Sani et al. 2012 ) . Figure 1 right illustrates how background noise severely degrades transcription quality, even with advanced models like Whisper. This demonstrates the vulnerability of current ASR systems to real-world acoustic challenges. Amid the growing use of large-scale speech-language models, Whisper has emerged as a strong multilingual ASR system ( Radford et al. 2023 ) . Unlike prior models such as wav2vec 2.0 and XLS-R, Whisper demonstrates superior robustness and generalization, particularly in noisy and low-resource scenarios ( Pratama and Amrullah 2024 ; Shah et al. 2024 ) . These strengths make Whisper an ideal foundation for exploring ASR robustness in Javanese and Sundanese. In this work, we present the first systematic study of ASR robustness to noise in these languages using over 60 hours of training data. Our key takeaways are: (1) evaluating Whisper models across clean and noisy test conditions; (2) exploring training strategies like SpecAugment and noise-aware fine-tuning; (3) analyzing language-specific transcription errors; and (4) releasing our training and evaluation pipeline for reproducibility. This is the first work to benchmark ASR robustness to noise in these languages systematically. 2 Related Works Figure 1: (Left) Training and evaluation pipeline for Whisper-based ASR models. Each fine-tuned model is evaluated on clean and noisy versions of the OpenSLR test set. (Right) Examples of noisy transcriptions in Javanese and Sundanese using Whisper. The top boxes show spoken utterances with noise; the bottom boxes show the corresponding ASR outputs, demonstrating significantly degraded quality under noisy conditions. ASR for Sundanese and Javanese The NusaASR benchmark Cahyawijaya et al. 2023 evaluates ASR models on Javanese and Sundanese primarily in zero-shot settings. While prior work has fine-tuned large models like XLS-R and Whisper Arisaputra et al. 2024 ; Pratama and Amrullah 2024 , these efforts often rely on limited data and lack reproducibility. Moreover, they rarely address robustness under noisy conditions. In contrast, our work provides a more comprehensive evaluation by fine-tuning Whisper across both languages. Noise Robustness Ensuring ASR robustness in noisy environments is a well-recognized challenge Shah et al. 2024 ; Feng et al. 2021 ; Likhomanenko et al. 2020 . Prior work addresses this through data augmentation techniques such as synthetic noise injection and room impulse responses. Among these, SpecAugment Park et al. 2019 has gained popularity as a simple and effective method. Other approaches include noise-aware training Orel and Varol 2023 and denoising front-ends Dissen et al. 2024 . In our work, we independently evaluate SpecAugment and noise-aware fine-tuning, using noise samples from AudioSet Gemmeke et al. 2017 , as two distinct strategies to improve ASR robustness. (a) Javanese (b) Sundanese Figure 2: WER performance of Large-v3 Whisper across different SNR levels for Javanese and Sundanese. Models trained with NoiseTrain consistently outperform others under low-SNR conditions. Higher SNR values indicate cleaner audio. 3 Experimental Setup 3.1 Linguistic Characteristics Javanese Javanese has more than 80 million speakers ( Eberhard et al. 2021 ) and is part of the Austronesian, Malayo Polynesian family ( Cohn and Ravindranath 2014 ) . It is agglutinative with extensive affixation that produces many word forms and is commonly divided into Western, Central, and Eastern varieties, each with distinct phonology and vocabulary ( Wedhawati et al. 2001 ) . A notable feature is its speech levels, such as ngoko (informal) and krama (polite), which encode social hierarchy in interaction ( Isodarus 2020 ) . Sundanese Sundanese, spoken by about 30–40 million people in western Java ( Eberhard et al. 2021 ) , is part of the Austronesian, Malayo Polynesian family and shows agglutinative morphology with rich affixation. Major dialects include Bogor, Priangan, and Cirebon, which differ in vocabulary and pronunciation ( Kurniawan 2013 ) . The language also encodes politeness through registers that guide lexical choice. 3.2 Dataset Data Overview We use the OpenSLR Javanese and Sundanese corpora ( Kjartansson et al. 2018 ) , collected with support from Universitas Gadjah Mada in Yogyakarta and Universitas Pendidikan Indonesia in Bandung. The recordings are read speech from volunteers. These corpora are valuable but do not cover the full range of dialects or spontaneous use. From the full releases (185k utterances / 296 hours for Javanese and 219k utterances / 333 hours for Sundanese), we selected 10 subsets for training and 6 for testing ( Kjartansson et al. 2018 ) . This gives about 60 hours of training data and 10 hours of test data per language, with train and test speakers kept separate (Table 1 ). The size is adequate for baseline ASR, but limited coverage should be considered when interpreting results. While we were unable to identify detailed dialectical or speaker variations from the original paper Kjartansson et al. 2018 , we estimated the proportion of female and male speakers using a fine-tuned version of wav2vec ( Baevski et al. 2020 ) 1 1 1 https://huggingface.co/prithivMLmods/Common-Voice-Gender-Detection . Lang Train Test #Speakers (F%) JV 37,439 6,276 758 (57%) SU 39,560 6,563 529 (57%) Table 1: Number of utterances and unique speakers for each language, with female speaker proportion. Synthetic Noise Data Generation To simulate real-world conditions, we augment clean training data with background noise at various Signal-to-Noise Ratio (SNR) levels, following prior work Orel and Varol 2023 ; Maas et al. 2012 . The noise types reflect common environments like traffic and indoor chatter. Details on the noise selection, SNR values, and mixing procedure are provided in Appendix B . 3.3 Training Pipeline We fine-tune four Whisper variants— Tiny , Medium , Large-v3 , and Large-v3-Turbo —on Javanese and Sundanese ASR using OpenSLR. While these models support the languages, their zero-shot performance is poor due to limited training exposure. We explore three training strategies to improve robustness, as illustrated in Figure 1 . Clean Fine-tuning Models are trained on unmodified OpenSLR data as a baseline. Clean

Read the complete text