Bemba Speech Translation: Exploring a Low-Resource African Language
Muhammad Hazim Al Farouq Kreasof AI Research Labs Jakarta, Indonesia Aman Kassahun Wassie African Institute for Mathematical Sciences (AIMS) Addis Ababa, Ethiopia, Yasmin MoslemADAPT CentreTrinity College DublinDublin, Ireland · 2025
Read Bemba Speech Translation: Exploring a Low-Resource African Language by Muhammad Hazim Al Farouq Kreasof AI Research Labs Jakarta, Indonesia Aman Kassahun Wassie African Institute for Mathematical Sciences (AIMS) Addis Ababa, Ethiopia, Yasmin MoslemADAPT CentreTrinity College DublinDublin, Ireland (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02518.txt.
Introduction
Bemba, spoken widely in Zambia, has almost no digital resources for speech translation. Muhammad Hazim Al Farouq and co-authors describe their Bemba-to-English speech translation system in a 2025 conference paper.
The team chained a speech recognizer to a multilingual translation model, a cascade design suited to scarce data. They boosted the small training set with synthetic data made by back-translation, translating text backward to create new pairs. The paper reports their setup and tests how much the synthetic data helps. The system was entered in the low-resource track of a spoken language translation contest. Work like this shows how far careful data tricks can stretch a handful of recordings.
Catalog record
| CREATOR | Muhammad Hazim Al Farouq Kreasof AI Research Labs Jakarta, Indonesia Aman Kassahun Wassie African Institute for Mathematical Sciences (AIMS) Addis Ababa, Ethiopia, Yasmin MoslemADAPT CentreTrinity College DublinDublin, Ireland |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02518
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bemba Speech Translation: Exploring a Low-Resource African Language Muhammad Hazim Al Farouq Kreasof AI Research Labs Jakarta, Indonesia Aman Kassahun Wassie African Institute for Mathematical Sciences (AIMS) Addis Ababa, Ethiopia, Yasmin MoslemADAPT CentreTrinity College DublinDublin, Ireland Abstract This paper describes our system submission to the International Conference on Spoken Language Translation (IWSLT 2025), low-resource languages track, namely for Bemba-to-English speech translation. We built cascaded speech translation systems based on Whisper and NLLB-200, and employed data augmentation techniques, such as back-translation. We investigate the effect of using synthetic data and discuss our experimental setup. \scalebox{0.5}{\faIcon[regular]{star}} \scalebox{0.5}{\faIcon[regular]{star}} footnotetext: Correspondence: yasmin[at]machinetranslation.io 1 Introduction Low-resource languages face critical limitations due to the scarcity and scattered nature of the available data ( Haddow et al. 2022 ) . Speech translation for low-resource languages involves similar challenges ( Ahmad et al. 2024 ; Moslem 2024 ; Lovenia et al. 2024 ; Abdulmumin et al. 2025 ) , Similarly, speech applications for African languages are very limited due to the lack of linguistic resources. For example, Bemba is an under-resourced language spoken by over 30% of the population in Zambia ( Sikasote and Anastasopoulos 2022 ) . Hence, the IWSLT shared task on speech translation for low-resource languages aims to benchmark and promote speech translation technology for a diverse range of dialects and low-resource languages. We participated in the Bemba-to-English language pair through building cascaded speech translation systems. In other words, we employed Whisper ( Radford et al. 2022 ) for automatic speech recognition (ASR), and NLLB-200 ( Costa-jussà et al. 2022 ) for text-to-text machine translation (MT). For ASR, we fine-tuned Whisper models using two datasets, BembaSpeech and BIG-C. For MT, we fine-tuned the NLLB-200 models using the bilingual segments of the BIG-C dataset, and the “dev” split of the FLORES-200 dataset. In addition, we augmented the Bemba-to-English training data with back-translation of a portion of the Tatoeba dataset from English into Bemba. The back-translated data was filtered based on cross-entropy scores. As Table 4 shows, the systems we submitted to the shared tasks are as follows: • Primary: It uses Whisper-Medium for ASR and NLLB-200 3.3B for MT. • Contrastive 1: It uses Whisper-Small for ASR and NLLB-200 3.3B for MT. • Contrastive 2: It uses Whisper-Small for ASR and NLLB-200 600M for MT. 2 Data The data we used to train our Bemba-to-English speech translation models can be categorized into: (1) authentic data, and (2) synthetic data. The following sections provide more details (cf. Table 1 ). Dataset Language Train Dev Test Audio Big-C Bem-Eng 82,371 2,782 2,763 \resizebox{.5em}{!}{\faCheck}⃝ BembaSpeech Bem 12,421 1,700 1,359 \resizebox{.5em}{!}{\faCheck}⃝ FLORES-200 Bem-Eng 997 0 1,012 \resizebox{.4em}{!}{\faTimes}⃝ Tatoeba Eng 20,121 0 0 \resizebox{.4em}{!}{\faTimes}⃝ Table 1: Data Statistics: The “Language” column specifies which languages are originally available in each dataset. “Train”, “Dev”, and “Test” represent the dataset sizes. The “Audio” column indicates whether each dataset includes audio signals. FLORES-200 BIG-C Training Dataset(s) BLEU chrF++ COMET BLEU chrF++ COMET Big-C 18.13 42.11 53.25 27.83 51.08 53.28 Big-C + Tatoeba 21.67 45.25 55.64 27.82 50.98 53.39 Big-C + FLORES-200 25.21 47.31 57.23 27.96 51.03 53.29 Big-C + FLORES-200 + Tatoeba 25.70 47.75 58.29 28.60 51.38 53.08 Table 2: MT Evaluation: In general, the models trained with both authentic data (Big-C & FLORES-200) and back-translated data (Tatoeba) outperform the models trained with the authentic data only. All the models in this table uses NLLB-200 600M. 2.1 Authentic Data We filtered the authentic data by removing any overlaps between the training data and test data based on the text transcript. For building our models, we used the following data sources. • Big-C is a parallel corpus of speech and transcriptions of image-grounded dialogues between Bemba speakers and their corresponding English translations. It contains 92,117 spoken utterances of both complete and incomplete dialogues, amounting to 187 hours of speech data grounded on 16,229 unique images. The dataset aims to enable the development of speech recognition, speech, and text translation systems for Bemba, as well as facilitate research in language grounding and multimodal model development ( Sikasote and Anastasopoulos 2022 ) . 1 1 1 https://github.com/csikasote/bigc Since this dataset includes audio and transcription in Bemba as well as translation into English, we could use it to build both modules of our cascaded systems, i.e. ASR and MT. Table 7 shows examples of sentence pairs from the Big-C datasets. • BembaSpeech is an ASR corpus for the Bemba language of Zambia. It contains read speech from diverse publicly available Bemba sources; literature books, radio/TV shows transcripts, YouTube video transcripts as well as various open online sources. Its purpose is to enable the training and testing of automatic speech recognition (ASR) systems in Bemba language. The corpus has 14,438 utterances, culminating into 24.5 hours of speech data ( Sikasote et al. 2023 ) . 2 2 2 https://github.com/csikasote/BembaSpeech We used the BembaSpeech dataset in addition to the Big-C dataset to build our ASR models. • FLORES-200 ( Goyal et al. 2022 ) is a bilingual text-only dataset for machine translation. We used the Bemba-to-English “dev” split for training, and the “devtest” split for testing. • Tatoeba ( Tiedemann 2020 ) is a monolingual dataset in English. We used a portion of it for back-translation (cf. Section 2.2 ). 2.2 Synthetic Data We augmented our authentic data (cf. Section 2.1 ) with synthetic data created with back-translation. To this end, we fine-tuned the NLLB-200 600M model in the other direction, i.e. for the English-to-Bemba language pair. Thereafter, we translated the English sentences from Tatoeba into Bemba using the fine-tuned English-to-Bemba NLLB-200 model. For translation, we used CTranslate2 ( Klein et al. 2020 ) , generating the prediction cross-entropy scores for each sentence, and calculating the exponential of the scores for better readability. We filtered data based on the cross-entropy scores, removing low-quality segments. We removed segments with scores less than 0.77 based on manual exploration of samples of the generated back-translations. While the unfiltered back-translated data consists of 85,000 segments, the filtered back-translated data consists of 20,000 segments. Finally, we prepended the source side (Bemba) with the <bt> tag to indicate that the data is synthetic. Moreover, we experimented with removing the <bt> tag and found that this achieves slightly better results when testing with the FLORES-200’s “devtest” split, as the data was already filtered (cf. Table 3 ). FLORES-200 BIG-C Datasets BT Size Filtered <bt> tag BLEU chrF++ COMET BLEU chrF++ COMET BIG-C + Tatoeba 85,155 \resizebox{.4em}{!}{\faTimes}⃝ \resizebox{.5em}{!}{\faCheck}⃝ 20.96 45.06 55.92 28.17 51.26 53.45 20,121 \resizebox{.5em}{!}{\faCheck}⃝ \resizebox{.5em}{!}{\faCheck}⃝ 19.82 44.09 54.79 28.04 51.20 53.51 20,121 \resizebox{.5em}{!}{\faCheck}⃝ \resizebox{.4em}{!}{\faTimes}⃝ 21.67 45.25 55.64 27.82 50.98 53.39 Table 3: Performance of MT models that are based on NLLB-200 600M and trained using both authentic data and augmented back-translated data. There are two pre-processing aspects applied to the augmented data, filtering the data based on cross-entropy scores, and prepending the source sentence with the <bt> tag. Evaluating the models with the devtest split of the FLORES-200 dataset, the highest evaluation scores, in terms BLEU and chrF++, are achieved when the back-translated data is filtered and the <bt> tag is removed. Meanwhile, the AfriCOMET score (COMET) of this model is comparable to the model where the back-translated data is not filtered and the source is prepended