IIITH-BUT system for IWSLT 2025 low-resource Bhojpuri to Hindi speech translation
Bhavana Akkiraju, Aishwarya Pothula, Santosh Kesiraju, Anil Kumar Vuppala · 2025
Read IIITH-BUT system for IWSLT 2025 low-resource Bhojpuri to Hindi speech translation by Bhavana Akkiraju, Aishwarya Pothula, Santosh Kesiraju, Anil Kumar Vuppala (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04714.txt.
Introduction
Bhavana Akkiraju and co-authors describe a speech translation system built for a shared task in twenty twenty-five. The system turns spoken Bhojpuri into written Hindi, a hard pair because training data is scarce. Their entry fine-tunes a large multilingual model called SeamlessM4T for this one job.
The team tested many training settings, such as learning speed, batch size, and the number of training steps, to see what helped most. They also stretched the small dataset by speeding up speech and masking parts of the sound, and by training with Marathi speech as well. Careful settings and these simple tricks lifted scores a lot. An error review shows what kinds of mistakes still hurt the output.
Catalog record
| CREATOR | Bhavana Akkiraju, Aishwarya Pothula, Santosh Kesiraju, Anil Kumar Vuppala |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04714
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
IIITH-BUT system for IWSLT 2025 low-resource Bhojpuri to Hindi speech translation Bhavana Akkiraju, Aishwarya Pothula, Santosh Kesiraju, Anil Kumar Vuppala Abstract This paper presents the submission of IIITH-BUT to the IWSLT 2025 shared task on speech translation for the low-resource Bhojpuri-Hindi language pair. We explored the impact of hyperparameter optimisation and data augmentation techniques on the performance of the SeamlessM4T model fine-tuned for this specific task. We systematically investigated a range of hyperparameters including learning rate schedules, number of update steps, warm-up steps, label smoothing, and batch sizes; and report their effect on translation quality. To address data scarcity, we applied speed perturbation and SpecAugment and studied their effect on translation quality. We also examined the use of cross-lingual signal through joint training with Marathi and Bhojpuri speech data. Our experiments reveal that careful selection of hyperparameters and the application of simple yet effective augmentation techniques significantly improve performance in low-resource settings. We also analysed the translation hypotheses to understand various kinds of errors that impacted the translation quality in terms of BLEU. 1 Introduction Speech translation (ST) transforms spoken language into written text in a different language, serving as a critical component in breaking down communication barriers. While significant advancements have been made for high-resource language pairs Jia et al. 2019 ; Bentivogli et al. 2021 , developing effective ST systems for low-resource and dialectal languages remains challenging due to scarce parallel data, inconsistent orthography, and substantial linguistic variation. We address the low-resource scenario of Bhojpuri speech to Hindi text translation within the Indian linguistic context. Despite being spoken by over 50 million people, Bhojpuri suffers from limited quality and diversity in available speech-text corpora Wikipedia contributors 2025 . In contrast, Hindi possesses relatively abundant resources, making it an ideal target language for translation. Our system, developed for the IWSLT 2025 shared task, addresses these resource disparities through a combination of transfer learning from linguistically similar languages, data augmentation techniques, and hyperparameter optimization. Recent end-to-end ST models, such as the Speech-to-Text Transformer Wang et al. 2020b and SeamlessM4T Communication et al. 2023 effectively replace traditional cascade pipelines by jointly modelling ASR and MT, reducing error propagation and latency. However, these approaches typically require substantial labelled data unavailable for most low-resource languages, often leading to overfitting and poor generalisation. Our contributions include: • A systematic investigation of hyperparameter optimization for low-resource ST, identifying that configuration choices such as batch sizes, moderate label smoothing values, and extended warmup periods significantly impact performance on the Bhojpuri-Hindi language pair. • An analysis of data augmentation techniques - SpecAugment Park et al. 2019 and speed perturbation Ko et al. 2015 for low-resource speech translation, demonstrating their effectiveness in expanding our training data by 3x and improving BLEU scores by an average of 2.1 points. • An evaluation of cross-lingual transfer learning through joint fine-tuning with Marathi-Hindi data, empirically showing how linguistic similarities between related Indo-Aryan languages can be leveraged to improve low-resource speech translation performance. The remaining of this paper is organized as follows: Section 2 provides a comprehensive review of related work, Section 3 describes our system including hyperparameter optimization, data augmentation techniques, and joint fine-tuning approach, Section 4 details our experimental setup, Section 5 presents results and analysis, and Section 6 concludes with future directions. 2 Related work Low-resource speech translation (ST) has garnered significant attention through IWSLT shared tasks Agarwal et al. 2023 ; Ahmad et al. 2024 ; Gow-Smith et al. 2023 . The field has evolved from traditional pipeline approaches Post et al. 2013 to end-to-end architectures such as Listen Attend and Spell (LAS) Bérard et al. 2016 , fairseq S2T Wang et al. 2020a ; E. Ortega et al. 2023 , Radhakrishnan et al. 2023 , and transfer learning based methods Kesiraju et al. 2023b ; Kesiraju et al. 2023a . In Mbuya and Anastasopoulos 2023 , explored fine-tuning self-supervised models by incorporating a linear layer for the ST task, which streamlined workflows while maintaining specialized strategies for low-resource scenarios. Shanbhogue et al. 2023 , implemented various data augmentation techniques including audio stretching, back-translation, and paraphrasing. Contemporary approaches to low-resource ST can be categorized into several methodological frameworks. The SETU-DCU submission Zafar et al. 2024 enhanced ST robustness through CTC loss integration and rigorous data cleaning protocols. Post et al. 2013 incorporated pseudo-labelling techniques to expand their training corpus. The JHU IWSLT 2024 system Robinson et al. 2024 demonstrated the efficacy of Whisper-style large models with domain-adaptive pretraining methodologies. Meanwhile, the QUESPA team Ortega et al. 2024 implemented ensemble decoding with cross-lingual knowledge transfer mechanisms. SeamlessM4T Communication et al. 2023 presents a comprehensive approach handling ASR, MT, and ST across more than 100 languages, though its performance on genuinely low-resource languages necessitates substantial adaptation strategies. Our approach focuses on systematically exploring hyperparameter optimization and data augmentation techniques for low-resource speech translation. Unlike previous work that often applies general strategies such as default hyperparameter values, generic data augmentation, standard transfer learning without language-specific considerations and using default model architectures and training strategies without adaptation to low-resource constraints, we conduct a comprehensive investigation specifically tailored to the challenges of the Bhojpuri-Hindi language pair. Our experimentation includes a detailed analysis of learning rates, batch sizes, label smoothing values, and warmup periods, and used two data augmentation techniques (Speed Perturb, SpecAug). Additionally, we examine how cross-lingual transfer from Marathi can supplement these optimizations, leveraging the linguistic proximity between these Indo-Aryan languages. 3 Methodology 3.1 Model architecture Our experiments used SeamlessM4T as the backbone. We experimented with medium (1.2B parameters) and large (2.3 B parameters) variants. The medium consists of a 24-layer conformer speech encoder, 12-layer Transformer text decoder, with 1024-dimensional hidden states, and 16 attention heads. 3.2 Fine-tuning For most of the experiments, we fine-tuned all the parameters on the target language pair, i.e. Bhojpuri–Hindi. We also conducted experiments where fine-tuned on both language pairs Marathi–Hindi and Bhojpuri–Hindi. This model was further fine-tuned for few epochs for the target pair Bhojpuri–Hindi. 3.3 Evaluation We used the standard objective metrics BLEU Papineni et al. 2002 and chrF++ Popović 2017 implemented in sacrebleu Post 2018 to objectively evaluate the translation quality against the reference. 4 Experiments 4.1 Datasets We used only the official IWSLT 2025 shared task dataset for Bhojpuri → Hindi speech translation: Dataset Duration (hrs) No. of utterances Training 20.00 10,171 Dev 2.07 1,056 Test 0.87 750 Table 1: Data split statistics for Bhojpuri-Hindi ST task. For our multilingual experiments, we incorporated an additional Marathi \rightarrow Hindi 1 1 1 IWSLT Marathi-Hindi Dataset parallel corpus from IWSLT, consisting of 16 hours (7,990 utterances) of training data, 3.6 hours (2,103 utterances) of development data, and 0.45 hours (286 utterances) of test data. 4.2 Implementation Our code was based on the original seamless library written in PyTorch. Our experiments used two NVIDIA A100 GPUs (40GB each), employing data parallel training, synchronised batch normalisation, and FP16 mixed precision for optimal computational efficiency. 4.3 Pre-trained model selection Here we report the fine-tuning results for Seamless medium and large v2 variants. We observed that the medium model consistently yielded better translation results than the large one. Hence, we used the medium variant for all subsequent experiments and analysis. Model lr BLEU chrF++ Medium 1e-5 30.5 54.6 Large