Low-Resource NMT: A Case Study on the Written and Spoken Languages in Hong Kong Conference: 2021 5th International Conference on Natural Language Processing and Information Retrieval (NLPIR); December 17–20, 2021; Sanya, China 2021 5th International Conference on Natural Language Processing and Info
Hei Yi Mak, Tan Lee · 2025
Read Low-Resource NMT: A Case Study on the Written and Spoken Languages in Hong Kong Conference: 2021 5th International Conference on Natural Language Processing and Information Retrieval (NLPIR); December 17–20, 2021; Sanya, China 2021 5th International Conference on Natural Language Processing and Info by Hei Yi Mak, Tan Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17816.txt.
Introduction
Hei Yi Mak and Tan Lee build translation between written Chinese and written Cantonese in a 2025 paper. Hong Kong reads standard Chinese but speaks Cantonese, whose written form differs in words and grammar. Paired texts for training a translator are scarce.
Alongside twenty-eight thousand pairs gathered from past studies and the web, the team mined seventy-two thousand more from matched Wikipedia pages in the two tongues. The mined pairs lifted scores on every test set. Their transformer system beat a major commercial tool on six of eight test sets. Data craft, the paper shows, can offset scarcity.
Catalog record
| CREATOR | Hei Yi Mak, Tan Lee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17816
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Low-Resource NMT: A Case Study on the Written and Spoken Languages in Hong Kong Conference: 2021 5th International Conference on Natural Language Processing and Information Retrieval (NLPIR); December 17–20, 2021; Sanya, China 2021 5th International Conference on Natural Language Processing and Information Retrieval (NLPIR) (NLPIR 2021), December 17–20, 2021, Sanya, China Price: 15.00 DOI: 10.1145/3508230.3508242 ISBN: 978-1-4503-8735-4/21/12 CCS: Computing methodologies Machine translation Hei Yi Mak, Tan Lee Abstract . The majority of inhabitants in Hong Kong are able to read and write in standard Chinese but use Cantonese as the primary spoken language in daily life. Spoken Cantonese can be transcribed into Chinese characters, which constitute the so-called written Cantonese. Written Cantonese exhibits significant lexical and grammatical differences from standard written Chinese. The rise of written Cantonese is increasingly evident in the cyber world. The growing interaction between Mandarin speakers and Cantonese speakers is leading to a clear demand for automatic translation between Chinese and Cantonese. This paper describes a transformer-based neural machine translation (NMT) system for written-Chinese-to-written-Cantonese translation. Given that parallel text data of Chinese and Cantonese are extremely scarce, a major focus of this study is on the effort of preparing good amount of training data for NMT. In addition to collecting 28K parallel sentences from previous linguistic studies and scattered internet resources, we devise an effective approach to obtaining 72K parallel sentences by automatically extracting pairs of semantically similar sentences from parallel articles on Chinese Wikipedia and Cantonese Wikipedia. We show that leveraging highly similar sentence pairs mined from Wikipedia improves translation performance in all test sets. Our system outperforms Baidu Fanyi’s Chinese-to-Cantonese translation on 6 out of 8 test sets in BLEU scores. Translation examples reveal that our system is able to capture important linguistic transformations between standard Chinese and spoken Cantonese. 1. Introduction Machine translation refers to the practice of translating text or speech from one language to another language using computer software. Traditionally, machine translation systems were designed with rule-based or statistical approaches. Recent research interests have been largely drawn towards the use of artificial neural network, leading to a branch of MT research known as neural machine translation (NMT). State-of-the-art NMT models can be divided into three categories: RNN-based models ( Sutskever et al. 2014 ) , CNN-based models ( Gehring et al. 2017 ) , and transformers ( Vaswani et al. 2017 ) . These models generally follow the encoder-decoder architecture with attention mechanism ( Bahdanau et al. 2014 ) . The encoder-decoder architecture allows end-to-end training with source and target sentences. The attention mechanism enables the decoder to learn to focus on the specific parts of input sentence. This paper presents an NMT system for written-Chinese-to-written-Cantonese translation. Written Chinese corresponds to the written form of the standard spoken Chinese, Mandarin. Cantonese, or referred to as the Yue dialect, is one of the major Chinese dialects. It is spoken by tens of millions of people in Hong Kong, Macau, Guangdong and Guangxi provinces, as well as many overseas Chinese communities in Southeast Asia and North America. Despite being used primarily in spoken form, Cantonese can be transcribed into Chinese char-acters. Such transcription is referred to as written Cantonese. Spoken Cantonese and Mandarin are considered to be mutually unintelligible ( Snow 2004 ) . Their written form, i.e., written Cantonese and written Chinese also exhibit significant lexical and grammatical differences. In Hong Kong, the majority of the population are native Cantonese speakers and communicate in Cantonese in both informal and formal speaking contexts (e.g., daily communication, council meetings, business meeting, news reports, etc.). In typical written contexts, however, Chinese is the preferred language. Nevertheless, writing in Cantonese has become very popular among internet and mobile users. With the prevalence of instant messaging and social media, as well as millions of native speakers around the globe, written Cantonese has found its importance and usage in online social communication. For simplicity of terminology, written Chinese is referred to as “Chinese” and written Cantonese is referred to as “Cantonese” in this paper. Chinese-to-Cantonese MT systems have many potential applications. They allow automatic translation of Chinese documents into Cantonese, which could be readily combined with Cantonese text-to-speech systems to realize applications like storytelling, virtual news anchor, etc. By training in the reverse direction with similar parallel data, a Cantonese-to-Chinese MT system can also be developed. Coupled with Cantonese speech-to-text, the MT system can support the generation of Chinese text caption for Cantonese-speaking recording. Such systems can be very helpful to Mandarin speakers in consuming Cantonese audiovisual materials, e.g., lectures, public speech. NMT models are typically trained on large datasets that contain millions of parallel sentences of the intended language pair. When such parallel data resources are limited, researchers are required to work in the so-called low-resource setting. Parallel data of Chinese and Cantonese are extremely low-resourced. This is mainly because spoken Cantonese is rarely transformed into digital text for archive. Even when a Cantonese corpus is available, substantial efforts are needed to translate the data into Chinese. The scarcity of parallel resources is a common issue in dialect translation ( Honnet et al. 2017 ; Abe et al. 2018 ) . One of the contributions of this paper is to provide a detailed account of how training data are collected in the case of Chinese dialect translation. In particular, we constructed a set of 72K parallel sentences by mining pairs of semantically similar sentences from parallel articles on Chinese Wikipedia and Cantonese Wikipedia using LASER ( Artetxe and Schwenk 2019 ) . We hope to provide useful insights to the MT community on acquiring parallel resources in Chinese and Cantonese and possibly other language pairs. This paper is organized as follows. In the next section, we review the related work on Chinese and Cantonese MT and a similar low-resource task on German dialects. In Section 3, we describe the main linguistic considerations in Chinese-to-Cantonese translation. In Section 4, the NMT model used for Chinese-to-Cantonese MT is described. In Section 5, we report the effort of searching and collecting data resources for MT. In Section 6, the method of mining parallel sentences from Wikipedia is presented. In Section 7, we describe the training process. In Section 8, we evaluate the developed MT system by comparing BLEU scores against a baseline system and Baidu Fanyi 1 1 1 https://fanyi.baidu.com/#cht/yue , and analyze its performance on linguistic transformations with examples. Conclusions are given in Section 9. 2. Related Work Xu and Fung ( Xu and Fung 2012 ) developed an MT system that could generate Chinese transcription from Cantonese speech. The system leveraged the language model statistics of resource-rich language using a syntactic reordering model to improve speech recognition of the low-resource language. Wong and Lee ( Wong and Lee 2018 ) investigated a rule-based approach to translating Chinese sentences to Cantonese sentences at different register levels. This system incorporates annotated lexical mappings and syntactic transformations to perform register-sensitive translation. Honnet et al. ( Honnet et al. 2017 ) designed an MT system from Swiss German to High German with fewer than 60K parallel sentences. Swiss German represents a collection of spoken dialects in Switzerland that are seldom written and generally unintelligible to German speakers outside Switzerland. However, Swiss German