Tulun : Transparent and Adaptable Low-resource Machine Translation

Raphaël Merx, Hanna Suominen, Lois Hong, Nick Thieberger, Trevor Cohn, Ekaterina Vylomova · 2025

Read Tulun : Transparent and Adaptable Low-resource Machine Translation by Raphaël Merx, Hanna Suominen, Lois Hong, Nick Thieberger, Trevor Cohn, Ekaterina Vylomova (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18683.txt.

Introduction

Raphaël Merx and five co-authors present Tulun, a translation platform built for languages with little digital text and for specialist topics where ordinary machine translation fails. Rather than retraining a model, which small organizations and non-technical users cannot do, Tulun pairs a neural translation engine with a large language model. That model then post-edits the output using glossaries and translation memories the users supply.

The open-source web platform lets users create and edit those terminology resources, keeping human expertise inside the process. On medical and disaster relief tasks in Tetun and Bislama, Tulun beat plain machine translation by roughly 17 to 22 ChrF++ points. Across six low-resource languages in the FLORES benchmark, it also beat both standalone translation models and language-model-only approaches, gaining on average 2.8 points over a large multilingual model.

Catalog record

CREATORRaphaël Merx, Hanna Suominen, Lois Hong, Nick Thieberger, Trevor Cohn, Ekaterina Vylomova
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18683

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tulun : Transparent and Adaptable Low-resource Machine Translation Raphaël Merx, Hanna Suominen, Lois Hong, Nick Thieberger, Trevor Cohn, Ekaterina Vylomova Abstract Machine translation (MT) systems that support low-resource languages often struggle on specialized domains. While researchers have proposed various techniques for domain adaptation, these approaches typically require model fine-tuning, making them impractical for non-technical users and small organizations. To address this gap, we propose Tulun , 1 1 1 Tulun means ”assistance” in Tetun, highlighting a philosophy of augmenting rather than replacing human expertise. a versatile solution for terminology-aware translation, combining neural MT with large language model (LLM)-based post-editing guided by existing glossaries and translation memories. Our open-source web-based platform enables users to easily create, edit, and leverage terminology resources, fostering a collaborative human-machine translation process that respects and incorporates domain expertise while increasing MT accuracy. Evaluations show effectiveness in both real-world and benchmark scenarios: on medical and disaster relief translation tasks for Tetun and Bislama, our system achieves improvements of 16.90-22.41 ChrF++ points over baseline MT systems. Across six low-resource languages on the FLORES dataset, Tulun outperforms both standalone MT and LLM approaches, achieving an average improvement of 2.8 ChrF points over NLLB-54B. Tulun is publicly accessible at bislama-trans.rapha.dev . 1 Introduction Machine translation (MT) systems have transformed how organizations manage their translation needs Stefaniak (2022) ; Utunen et al. (2023) , yet domain accuracy and consistency remain a significant challenge, particularly for low-resource languages Haddow et al. (2022) ; Khiu et al. (2024) ; Marashian et al. (2025) . For instance, a health organization we work with in Timor-Leste struggled to leverage MT to accurately translate medical education materials from English to Tetun, despite having a glossary and a corpus of past translations that could inform MT output. Tetun, a low-resource language that is the lingua franca in Timor-Leste, lacks available corpora in the health domain Merx et al. (2024) , making in-domain resources particularly valuable to improve MT accuracy. This case exemplifies a broader challenge: model adaptation and deployment requires technical expertise, and commercial MT providers rarely offer low-resource language support, let alone terminology customization, leaving no practical option for a small organization to rely on MT for low-resource in-domain translation. Figure 1: System overview with example translation from English to Tetun (en-tdt). The system components and data are configurable by end-users. Translation memories and terminology management are well-established tools in professional translation software, improving translation accuracy while reducing cognitive load Dillon and Fraser (2006) ; Drugan et al. (2023) . Research has demonstrated that lexicons can bring substantial accuracy gains, particularly for low-resource MT Jones et al. (2023) . However, existing approaches to incorporate terminology constraints into neural MT systems typically require model fine-tuning Reid and Artetxe (2022) ; Niehues (2021) , making them inaccessible to small organizations Bane et al. (2023) . Recent advances in large language models (LLMs) offer a promising alternative: while LLMs may underperform specialized MT systems for low-resource languages Robinson et al. (2023) , their ability to adapt to new contexts at inference time Brown et al. (2020) makes them particularly suitable for terminology-aware post-editing Raunak et al. (2023) . To address these challenges, we propose Tulun , a versatile solution that combines neural MT with LLM-based post-editing, guided by existing glossaries and translation memories (Figure 1 ). Tulun continuously adapts as new entries are added to the translation memory and glossary. Packaged as an open-source 2 2 2 Code: github.com/raphaelmerx/tulun/ , MIT license web platform, it relies on a modular architecture that allows users to configure their choice of MT system, LLM, and retrieval options, as well as create, edit, and rely on terminology resources. To see a demo video of Tulun , visit https://youtu.be/fQFwOxzR4MI . Our system has the following characteristics: • Accurate : On real-world medical and disaster relief translation tasks for Tetun and Bislama (national language of Vanuatu), our system shows impressive improvements of 16.90-22.41 ChrF++ points over baseline MT systems ( section 4.1 ). A broader evaluation across six low-resource languages on the FLORES dataset shows Tulun outperforms both standalone MT and LLM approaches ( section 4.2 ). • User-friendly : Our usability study, based on the System Usability Scale (SUS), averages an excellent score of 81.25, with users rating the system’s overall usefulness at 5/5 for their translation tasks ( section 4.1.2 ). • Adaptable : Target language, MT model, and prompt are all configurable from the user interface (UI). Glossary and translation memories can be bulk-imported and managed through the UI ( section 3.1 ). • Transparent : Users can verify how their glossary entries and past translations inform the current translation. • Lightweight : Easy to deploy ( section 3.2 ), does not require model training. Fundamentally, Tulun represents a shift in MT philosophy, moving away from the paradigm of users as passive consumers of opaque systems Liebling et al. (2022) , toward one where users’ expertise and preferences actively shape the translation process Liu et al. (2025) . By making glossary and translation memory matches explicit to users, and by allowing configuration of the underlying data and systems, Tulun aims to foster a transparent, collaborative process that respects and leverages users’ domain knowledge. This approach benefits low-resource in-domain translation, where local expertise is often the most valuable resource for producing accurate, culturally appropriate translations Nekoto et al. (2020) . 2 Related Work Glossary and translation memory integration in MT The integration of custom terminology and translation memories into MT systems can deliver more consistent, domain-adapted translations Scansani and Dugast (2021) . Recent research has demonstrated that such lexical customization brings substantial accuracy gains, particularly for low-resource MT Jones et al. (2023) . Approaches to incorporate terminology constraints into neural MT models include replacing source words with their target translation in the source Reid and Artetxe (2022) , and prepending dictionary entries to the source text Niehues (2021) . However, these approaches typically require either custom models, or model fine-tuning, which can be resource-intensive for smaller organizations Bane et al. (2023) , and prone to catastrophic forgetting Saunders (2022) . Tulun addresses this gap by providing a deployable solution that requires no model training while delivering terminology-consistent translations. LLMs and Automated Post-Editing (APE) The ability for LLMs to adapt to new tasks at inference time Brown et al. (2020) makes them of interest to both MT Moslem et al. (2023a) and related tasks, such as synthetic data generation and automated post-editing Moslem et al. (2023b) . While their MT accuracy can lag behind that of specialized MT models when translating into low-resource languages Robinson et al. (2023) or in specialized domains Uguet et al. (2024) , Raunak et al. (2023) find that combining specialized MT with an LLM for APE results in more accurate translations than each module used in isolation (measured using COMET on high-resource language pairs). Confirming the potential of LLMs at the post-editing stage, Ki and Carpuat (2024) give external feedback to an LLM to improve MT outputs, and Lu et al. (2025) find that LLMs can identify and correct translation mistakes across high and low-resource language pairs. These findings suggest that LLMs can be valuable for terminology-aware post-editing, where their adaptation capabilities are combined with the robustness of specialized

Read the complete text