Unsupervised Bilingual Lexicon Induction for Low Resource Languages

Charitha Rathnayake, P.R.S. Thilakarathna, Uthpala Nethmini, Rishemjith Kaur, Surangika Ranathunga · 2024

Read Unsupervised Bilingual Lexicon Induction for Low Resource Languages by Charitha Rathnayake, P.R.S. Thilakarathna, Uthpala Nethmini, Rishemjith Kaur, Surangika Ranathunga (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16894.txt.

Description

Word-by-word dictionaries between languages power much translation software, yet many small languages have none. Charitha Rathnayake and co-authors work on building such dictionaries with no human-made seed at all. Their 2024 paper studies unsupervised bilingual lexicon induction, which starts from word patterns inside each language alone and aligns the two maps.

Many add-ons to the core method exist, but each was tested on its own. The team ran the add-ons together inside the VecMap framework on English paired with Sinhala, Tamil, and Punjabi, hunting for the mix that gains the most. The trials show which combinations stack and which cancel out. As a bonus, the team releases fresh English-Sinhala and English-Punjabi dictionaries for others to use.

Catalog record

CREATORCharitha Rathnayake, P.R.S. Thilakarathna, Uthpala Nethmini, Rishemjith Kaur, Surangika Ranathunga
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16894

Browse the full library · About the Tower of Babel