A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala Journal: Natural Language Processing

Surangika Ranathunga, Asanka Ranasinghe, Janaka Shamal, Ayodya Dandeniya, Rashmi Galappaththi, Malithi Samaraweera · 2024

Read A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala Journal: Natural Language Processing by Surangika Ranathunga, Asanka Ranasinghe, Janaka Shamal, Ayodya Dandeniya, Rashmi Galappaththi, Malithi Samaraweera (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02056.txt.

Introduction

Surangika Ranathunga and co-authors built, in a 2024 paper, a shared text corpus in English, Tamil, and Sinhala tagged for named entities. Named entity tools pick out people, places, and groups in text. Tamil and Sinhala lack the training data such tools need, since both are low-resource languages.

Sentences in the corpus line up across all three languages, each tagged the same way. The team set fresh benchmark scores for entity spotting in Sinhala and Tamil using multilingual language models. They also compared how unlike model types handled the task. Last, they showed their entity tool helps a low-resource translation system. The corpus is public, giving builders of Tamil and Sinhala tools a shared base to train and test on.

Catalog record

CREATORSurangika Ranathunga, Asanka Ranasinghe, Janaka Shamal, Ayodya Dandeniya, Rashmi Galappaththi, Malithi Samaraweera
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02056

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala Journal: Natural Language Processing Surangika Ranathunga, Asanka Ranasinghe, Janaka Shamal, Ayodya Dandeniya, Rashmi Galappaththi, Malithi Samaraweera Abstract This paper presents a multi-way parallel English-Tamil-Sinhala corpus annotated with Named Entities (NEs), where Sinhala and Tamil are low-resource languages. Using pre-trained multilingual Language Models (mLMs), we establish new benchmark Named Entity Recognition (NER) results on this dataset for Sinhala and Tamil. We also carry out a detailed investigation on the NER capabilities of different types of mLMs. Finally, we demonstrate the utility of our NER system on a low-resource Neural Machine Translation (NMT) task. Our dataset is publicly released: https://github.com/suralk/multiNER . 1 Introduction Named Entity Recognition (NER) is the process of identifying Named Entities (NEs) in natural language text. An NE can be a word or a phrase, and the detected entities are categorized into predetermined categories such as person, location and organization. As an example, consider the sentence: John works in Facebook at Los Angeles . This sentence contains three NEs: John (person), Facebook (organization) and Los Angeles (location). NER either acts as an intermediate step for, or helps to improve many high-level Natural Language Processing (NLP) tasks such as question answering [ 1 ] , Neural Machine Translation (NMT) [ 2 ] , Information Retrieval [ 3 ] and automatic text summarization [ 4 ] . There have been recent developments with respect to NE tag sets [ 5 ] , tagging schemes [ 6 ] , as well as NER algorithms [ 5 ] . Most of the NER algorithms that produced promising results are supervised, meaning that they are trained with NE annotated datasets. However, many low-resource languages have little to no annotated data [ 7 ] . Sinhala and Tamil are examples of low-resource languages [ 7 , 8 ] with limited NE annotated data [ 9 ] . Manamini et al.’s [ 10 ] is the only publicly available Sinhala NE dataset that has been manually annotated. FIRE corpus 1 1 1 http://fire.irsi.res.in/fire/2023/home is the only publicly available manually annotated NE dataset for Tamil. WikiANN [ 11 ] and LORELEI [ 12 ] are multilingual (but not multi-way pararell) NE annotated datasets that contain both Sinhala and Tamil. However, WikiANN has been automtically annotated using entity linking and LORELEI is hidden behind a paywall. In this paper, we present a multi-way parallel English-Sinhala-Tamil NE annotated corpus that consists of 3835 sentences per language. This corpus is annotated using the CONLL03 tag set [ 13 ] , which has four tags: persons (PER), locations (LOC), organizations (ORG) and miscellaneous (MISC). The corpus was annotated using Beginning-Inside-Outside (BIO) format. We provide a comprehensive analysis of this dataset and the data creation process. This dataset is publicly released 2 2 2 https://github.com/suralk/multiNER . The benefit of having such multi-way parallel datasets is that they can serve as good test beds to evaluate language-specific NER capabilities of pre-trained multilingual Language Models (mLMs) such as mBERT [ 14 ] and XLM-R [ 15 ] , which form the basis of modern-day NLP systems. In other words, if the mLM is fine-tuned with data from individual languages, this results in language-specific NER models, which can be probed to identify how their performance varies across languages. On the other hand, fine-tuning the mLM with all the language data of the multi-way parallel corpus results in a single NER model that caters for all the languages included in the corpus. How the performance of these models varies depending on the linguistic properties of individual languages is useful in building optimal language-specific NER systems. Despite the NER models built on mLMs outperforming the more traditional Deep Learning (DL) models such as Recurrent Neural Networks (specifically BiLSTM-CRF (Bi-Directional Long Short Term Memory with a CRF layer) [ 5 , 16 ] ), these newer techniques have not been employed for Sinhala or Tamil NER. In order to evaluate the Sinhala and Tamil NER capabilities, we experimented with different types of pre-trained LMs (pLMs): language-specific pLMs, language family-specific pLMs and mLMs. Our results show that, when the language is already included in the pLM, NER systems built on all these type of pLMs significantly outperform those that use the BiLSTM-CRF model. We also show that a multilingual NER model built by fine-tuning XLM-R with our multi-way parallel corpus outperforms (or is on-par with) the NER models trained for individual languages. Finally, we demonstrate the utility of the built NER models by using their output in building an NMT system. The NMT system, which was trained using the NEs identified using our NER system as an additional input significantly outperformed the baseline NMT model. 2 Related Work In this section, we provide a brief overview of NE tag sets, annotation schemes, NE annotated datasets and NER techniques based on pLMs. We also discuss NER research available for Sinhala and Tamil. 2.1 NE Tag Sets One prominent tag set that has been widely used in NER is the CONLL03 tag set [ 13 ] . This tag set has only 4 tags - Person, Location, Organization and Miscellaneous. The Co-reference and Entity Type tag set [ 17 ] has 12 NE types (7 numerical types, 10 nominal entity types and temporal types). WNUT2017 [ 18 ] corpus has 6 entity types. ACE 2005/2008 [ 19 ] has identified 7 NE types, as people, organizations, locations, facilities, Geo political entities, weapons, vehicles and events. ACE was defined for the tourism domain, thus can be considered as a domain-specific tag set. While having more NE types helps in extracting more information, training Machine Learning (ML) models for the task becomes very data intensive, which in turn makes manual data annotation costly. As shown by Azeez and Ranathunga [ 20 ] , if a sufficient annotated corpus is not created, having a fine-grained tag set is not useful, as many NE tags will not have sufficient data points to train the ML model. 2.2 Annotation Schemes Flat NE annotation schemes have been most commonly used in previous research. Alshammari et al. [ 21 ] present a comprehensive list of such annotation schemes. Some popular NE annotation schemes are IO (tags each token either as an inside tag (I) or an outside tag (O), where NEs are marked as I), IOB (which is also known as BIO - tags each token either as beginning (B) of a known NE, inside (I) it, or outside (O) of any known NE) and IOE (similar to IOB, but tags the end of an NE (E) instead of its beginning). The main drawback of these flat annotation schemes is that they are unable to capture nested NEs. As an alternative, markup NE annotation schemes have been proposed [ 22 , 23 , 24 ] . While more comprehensive annotation schemes allow the extraction of more fine-grained information, this requires more input from humans who annotate the data and ML models need more data to learn. Therefore, still the BIO tagging scheme, which was used in the CONLL 2003 shared task [ 13 ] is being commonly used. 2.3 NER techniques Recently, pLMs such as BERT [ 14 ] , RoBERTa [ 25 ] and their

Read the complete text