IndoNLP 2025: Shared Task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages

Deshan Sumanathilaka, Isuri Anuradha, Ruvan Weerasinghe, Nicholas Micallef, Julian Hough · 2025

Read IndoNLP 2025: Shared Task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages by Deshan Sumanathilaka, Isuri Anuradha, Ruvan Weerasinghe, Nicholas Micallef, Julian Hough (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05816.txt.

Introduction

Deshan Sumanathilaka and co-authors report a shared task run at IndoNLP 2025 on typing Indo-Aryan languages in their own scripts. Many speakers write these languages in Latin letters day to day, and current keyboards turn that improvised spelling back into native script poorly.

The task asked teams to build a real-time tool that converts the Romanized text as the user types. Eleven teams signed up, and four reached the final round with systems for Sinhala, Hindi, and Malayalam. The paper reviews the task design, the entries, and the results. The authors say such tools both fix a daily typing pain and widen digital access for low-resource languages with large speaker bases.

Catalog record

CREATORDeshan Sumanathilaka, Isuri Anuradha, Ruvan Weerasinghe, Nicholas Micallef, Julian Hough
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05816

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

IndoNLP 2025: Shared Task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages Deshan Sumanathilaka, Isuri Anuradha, Ruvan Weerasinghe, Nicholas Micallef, Julian Hough Abstract The paper overviews the shared task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages. It focuses on the reverse transliteration of low-resourced languages in the Indo-Aryan family to their native scripts. Typing Romanized Indo-Aryan languages using ad-hoc transliterals and achieving accurate native scripts are complex and often inaccurate processes with the current keyboard systems. This task aims to introduce and evaluate a real-time reverse transliterator that converts Romanized Indo-Aryan languages to their native scripts, improving the typing experience for users. Out of 11 registered teams, four teams participated in the final evaluation phase with transliteration models for Sinhala, Hindi and Malayalam. These proposed solutions not only solve the issue of ad-hoc transliteration but also empower low-resource language usability in the digital arena. 1 Introduction Languages transcend mere systems of communication. They are profound reflections of the cultures they represent. Embedded within each language are the accumulated wisdom, traditions, beliefs, and historical narratives cherished by its speakers. South Asia, a region renowned for its cultural richness and linguistic diversity, is home to a vast array of languages that serve not only as tools for communication but also as vibrant expressions of its multifaceted heritage. Among these, the Indo-Aryan languages hold a significant place, forming part of the larger Indo-Iranian branch of the Indo-European language family. As of the early 21st century, the Indo-Aryan languages (sometimes called Indic languages) have more than 800 million speakers, primarily concentrated east of the Indus River in Bangladesh, North India, Eastern Pakistan, Sri Lanka, Maldives, and Nepal KJ et al. 2024 . Additionally, large immigrant and expatriate Indo-Aryan-speaking communities can be found in Northwestern Europe, Western Asia, North America, the Caribbean, Southeast Africa, Polynesia, and Australia. There are over 200 known Indo-Aryan languages, further emphasizing the immense diversity within this linguistic group Talukdar and Sarma 2023 . Historically, the Indo-Aryan languages have evolved from Old Indo-Aryan (Sanskrit) through Middle Indo-Aryan (Prakrits) to their modern forms, such as Hindi, Bengali, Gujarati, Punjabi, and others. This historical evolution underscores their linguistic richness and role as carriers of cultural and historical narratives Masica 1993 . In contrast, the Dravidian languages constitute a distinct language family, primarily spoken in South India and parts of Eastern and Central India, as well as in northeastern Sri Lanka, Pakistan, Nepal, Bangladesh, and among diaspora communities worldwide. The Indo-Aryan and Dravidian languages illustrate South Asia’s profound linguistic and cultural complexity, weaving a vibrant tapestry of communication and heritage that resonates far beyond the region De Silva 2019 . With the advancement of digital technologies, communication on social media has become increasingly prominent. Social media users often employ both native scripts and Romanized versions of their native languages for accessible communication and compatibility with digital platforms. However, the transliteration of Indo-Aryan languages into native scripts remains under-explored, mainly due to a scarcity of language resources needed for their development. While numerous rule-based systems have been implemented to transliterate Indo-Aryan languages, significant challenges persist in addressing ad-hoc transliteration effectively. Social media users frequently rely on abbreviated and informal typing styles to communicate in their native languages. This variability in linguistic representation complicates the verification of hate speech and misinformation, as the diverse formats pose substantial challenges for detection and analysis Sumanathilaka et al. 2023 . The key contributions of this shared task are : • Introducing novel models for Romanized scripts to Native Indo-Aryan Languages Transliteration. • Introducing a standard evaluation dataset for five languages: Sinhala, Hindi, Bengali, Gujarati and Malayalam. Moving forward, the paper will discuss related works, shared task overview, results and discussion along with the conclusions. 2 Related Works Early back-transliteration methods for Indo-Aryan languages heavily relied on rule-based approaches, employing predefined character mappings and linguistic rules. For example, Vidanaralage et al. 2018 introduced a rule-based method for Sinhala transliteration using transliteration and phoneme rule bases. Similarly, Tennage et al. 2018 utilized character mapping tables for converting Sinhala into English. These methods demonstrated effectiveness in controlled environments but often faltered with informal Romanized text, such as Singlish, which includes significant variability and ambiguity. Researchers have combined rule-based methods with statistical models to overcome the limitations of purely rule-based systems. Liwera and Ranathunga 2020 applied a trigram model trained on Romanized Sinhala YouTube comments, integrated with rule-based techniques, to address transliteration challenges. Hybrid models have shown promise but struggle with contextual dependencies and ambiguities in informal text. Classical machine learning models were explored to transliterate Hindi and Marathi to English efficiently Rathod et al. 2013 . Recent advancements in deep learning have revolutionized transliteration tasks. Neural network-based sequence-to-sequence models have gained traction, particularly those utilizing recurrent architectures. Kunchukuttan et al. 2015 introduced Brahmi-Net, a transliteration system leveraging statistical approaches for 18 Indo-Aryan languages. Nanayakkara et al. 2022 employed a Bidirectional LSTM and LSTM for Sinhala transliteration, incorporating character-level context. Transformer architectures have emerged as state-of-the-art for transliteration due to their ability to handle long-range dependencies with self-attention mechanisms. Zohrabi et al. 2023 employed Transformer-based models for Azerbaijani transliteration while Madhani et al. 2023 introduced IndicXlit, a multilingual system leveraging Transformer architecture. Pretrained multilingual models like mT5 and M2M100 have also been fine-tuned for transliteration tasks, treating transliteration as a translation problem. A deep learning approach proposed by Deselaers et al. 2009 used a deep belief network for Arabic-English transliteration, and a few other Arabizi to Arabic transliteration are explored Masmoudi et al. 2019 ; Al-Badrashiny et al. 2014 . Hybrid approaches combining rule-based, statistical, and deep learning methods have demonstrated increased accuracy and robustness. Athukorala and Sumanathilaka 2022 proposed a hybrid approach for Singlish transliteration, integrating rule-based methods with fuzzy logic and achieving a word-level accuracy of 0.64. This work has been further extended to Ngram with Rule based approach by Sumanathilaka et al. 2023 and later Dharmasiri and Sumanathilaka 2024 by using a GRU for efficient transliteration. TAMZHI by Mudiyanselage and Sumanathilaka 2024 developed a back-transliteration system for Romanized Tamil, achieving 93% character-level accuracy and 70% word-level accuracy. Pix2Pix Generative Adversarial Network for transliterating ancient Indian scripts, demonstrating the adaptability of GANs for image-to-text transliteration were further explored Sharma et al. 2023 . The availability of datasets like Dakshina Roark et al. 2020 and Aksharantar Madhani et al. 2023 has significantly enhanced transliteration model training. Sumanathilaka et al. 2024 has built a large corpus of data which consists of Sinhala and its Romanized Sinhala patterns along with a dataset to handle Word sense disambiguation in Romanized Sinhala scripts. These large-scale resources provide robust transliteration pairs, enabling models to learn from diverse and complex Romanized text patterns. However, the lack of standardized rules for transliteration leads to ad-hoc spelling variations, including the omission or alteration of vowels, inconsistent phonetic mappings, and user-specific adaptations. These inconsistencies create significant challenges in accurately converting Romanized text into native scripts. Existing keyboard systems and transliteration tools frequently fall short of addressing these complexities, often producing inaccurate or unintelligible outputs Mammadzada 2023 . This results in frustration for users, reduces the accessibility of native scripts, and hinders seamless communication in their preferred languages. Therefore, the demand for a robust and reliable solution to this problem is

Read the complete text