An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese
Tran Ngoc Son, Nguyen Anh Tu, Nguyen Minh TriSamsung SDS R&D Center, Hanoi, Vietnam {tn.son,na.tu,nm.tri}@gsamsung.com; · 2025
Read An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese by Tran Ngoc Son, Nguyen Anh Tu, Nguyen Minh TriSamsung SDS R&D Center, Hanoi, Vietnam {tn.son,na.tu,nm.tri}@gsamsung.com; (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19314.txt.
Introduction
Tran Ngoc Son, Nguyen Anh Tu, and Nguyen Minh Tri present a 2025 arXiv paper on machine translation for low-resource language pairs, testing Vietnamese to Chinese. Neural translation starves without large parallel corpora. The team starts from a multilingual pretrained model, mBART, and a first model trained on the small bilingual set they have.
They then mine single-language texts with a keyword method, keeping sentences close to the target domain. The first model translates those sentences to build extra training data. The scheme beat a plain transformer by eight percent on their measure. The authors present data growth as the main lever for such pairs.
Catalog record
| CREATOR | Tran Ngoc Son, Nguyen Anh Tu, Nguyen Minh TriSamsung SDS R&D Center, Hanoi, Vietnam {tn.son,na.tu,nm.tri}@gsamsung.com; |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 12 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-19314
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese Tran Ngoc Son, Nguyen Anh Tu, Nguyen Minh TriSamsung SDS R&D Center, Hanoi, Vietnam {tn.son,na.tu,nm.tri}@gsamsung.com; Abstract Despite the rise of recent neural networks in machine translation, those networks do not work well if the training data is insufficient. In this paper, we proposed an approach for machine translation in low-resource languages such as Vietnamese-Chinese. Our proposed method leveraged the power of the multilingual pre-trained language model (mBART) and both Vietnamese and Chinese monolingual corpus. Firstly, we built an early bird machine translation model using the bilingual training dataset. Secondly, we used TF-IDF technique to select sentences from the monolingual corpus which are the most related to domains of the parallel dataset. Finally, the first model was used to synthesize the augmented training data from the selected monolingual corpus for the translation model. Our proposed scheme showed that it outperformed 8% compared to the transformer model. the augmented dataset also pushed the model performance. * * footnotetext: These authors contributed equally to this work 1 Introduction Machine translation is a task that aims to translate a sentence from one language to another language. Most of the proposed methods for machine translation models are based on encoder-decoder models Sutskever et al. 2014 Bahdanau et al. 2014 Luong et al. 2015 Vaswani et al. 2017 . The encoder transforms the source language sentence into a vector which is used in the decoder to generate the target language sentence. Recently, with the development of pre-trained language models that make significant improvements in many NLP tasks. Especially, in the machine translation task, multilingual pre-trained models like mBART Liu et al. 2020a M2M Fan et al. 2021 have been pre-training in many languages and giant datasets. In Vietnamese, spite neural machine translation(NMT) achieves impressive results in English-Vietnamese Doan et al. 2021 Ngo et al. 2022a which were trained on the large and high-quality parallel corpus. However, in low-resource pairs language, NMT faces a lack of datasets such as in Vietnamese-Chinese, Vietnamese-France, etc. Building a large and high-quality parallel dataset is quite difficult and takes a lot of cost and time while monolingual corpora are available on the internet. Therefore, in this paper, we propose an efficient approach for machine translation on low-resource languages. Our system participates in the Vietnamese-Chinese Machine Translation challenge in VLSP 2022. The main works can be summarized as follows: 1. We employ the strong multilingual sequence to sequence the pre-trained language model mBART for the machine translation task. 2. We propose data selection and data synthesis techniques from the monolingual corpus that helps improve our machine translation system. The rest of this paper is organized as follows. We first review related work on machine translation methods and recent approaches to low-resource languages. We then present our system, including the model architecture and data processing techniques. Experimental results and analyses are described in the next section. Finally, we conclude the paper. 2 Related Work This section gives a brief review of related work on deep learning-based models for the machine translation task and recent solutions for low-resource languages. Neural machine translation(NMT) is an approach to machine translation that was proposed first time by Kalchbrenner and Blunsom 2013 and Sutskever et al. 2014 which only have two simple components including encoder and decoder based on Gated Recurrent Units (GRU) Chung et al. 2014 or Long Short-Term Memory (LSTM) Hochreiter and Schmidhuber 1997 . This neural network compresses all the information of a source language sentence into a fixed-length vector. This may make it difficult for the neural network to translate long sentences. To deal with this problem Bahdanau et al. 2014 and Luong et al. 2015 proposed attention mechanisms that use alignment information between a source sentence and the corresponding target sentence to enhance the effectiveness of the systems. The transformers architecture is presented first time for machine translation by Vaswani et al. 2017 . They proposed a self-attention mechanism that is a variant of the original attention Bahdanau et al. 2014 and Luong et al. 2015 . By using self-attention, a word is attended to the other words in a sentence. In recent years, with an explosion of deep learning, transformer architectures for language models such as BERT Devlin et al. 2018 , M2M Fan et al. 2021 , mBART Liu et al. 2020a is capable of contextual representation of sentences. Based on the contextual representation ability, the machine translation task has gained significant improvements. Machine translation in low-resource languages is the most difficult problem. To deal with this problem, the most popular method is using the synthesis data from monolingual corpora to enhance the performance of the NMT system. Especially, in Vietnamese, Ngo et al. 2020 , Thi-Vinh et al. 2020 , Cuong and Thu 2020 , Ngo et al. 2022b proposed method using back-translation to generate parallel dataset such as Vietnamese-English, Vietnamese-Chinese, Vietnamese-France, Vietnamese-Japanese. Following these works, we also use back-translation as a data augmentation method but have some adjustments. 3 Approach 3.1 Our proposed method For the low-resource bilingual dataset like Vietnamese-Chinese, we proposed to use data synthesis as an augmentation technique to build more data for the training corpus. Firstly, an machine translation system is finetuned from the pre-trained mBart-50 using existing parallel data. This system is used to translate the target to the source language. The result is a new parallel corpus in which the source side is 200k sentences that are cleaned and selected from nearly 20M sentences for each language (monolingual dataset) by TF-IDF. Then, the synthesized parallel corpus is combined with the real text (bilingual dataset) to train the final system. Figure 1 illustrates our proposed training flow. In this paper, we adopted mBart-50 as the main translation model. Both monolingual and bilingual datasets must be cleaned and preprocessed before feeding to the mBart-50. To build the final translation model, three main steps have to be performed: • Step 1: Training a Vietnamese-English translation model with mBart-50 using the bilingual dataset. • Step 2: Generating an extra bilingual dataset from the clean and selected sentences from the monolingual dataset. • Step 3: Finetuning translation model with the newly generated extra bilingual dataset. We used the same flow for the Chinese-Vietnamese or Vietnamese-Chinese translation models. Figure 1: Flow of data processing and model training 3.2 Text Pre-processing 3.2.1 Data cleaning and Preprocessing The dataset given by 2022 VLSP organizers for the Vietnamese-Chinese Machine Translation task includes Parallel Corpora which has 300k sentence pairs and Monolingual Corpora which has 25M Vietnamese sentences and 19M Chinese sentences. Because of the noisy data, some steps have been taken to clean this dataset. • Removing non-Vietnamese and non-Chinese sentences using a language detection model pycld3 1 1 1 https://pypi.org/project/pycld3 . • Cleaning HTML characters and some special characters. • Removing sentences that are too long (more than 100 words) or too short (less than 2 words). 3.3 Mbart model for Neural Machine Translation For low-resource pair languages like Vietnamese-Chinese or Vietnamese-Korean, public datasets for training NMT are limited both in quantity and quality. As a result, training an NMT model based on a Transformer from scratch led to low accuracy Koehn and Knowles 2017 . Besides,