Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030?

Georg Rehm, Annika Grützner-Zahn, Fabio Barth · 2025

Read Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030? by Georg Rehm, Annika Grützner-Zahn, Fabio Barth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12886.txt.

Introduction

Europe's smaller languages risk being left behind as language technology races ahead, since big models need huge training text that many languages lack. Rehm, Grützner-Zahn, and Barth ask whether multilingual models, trained on many languages at once, can serve as an off-ramp for those languages. They survey the current support for European languages and the research behind the idea.

Early evidence shows such models can perform well even for languages with little data. The authors end with the open questions that must be settled before the approach can be applied in a systematic way, toward the goal of digital language equality by 2030.

Catalog record

CREATORGeorg Rehm, Annika Grützner-Zahn, Fabio Barth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12886

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030? Georg Rehm, Annika Grützner-Zahn, Fabio Barth Abstract Large language models (LLMs) demonstrate unprecedented capabilities and define the state of the art for almost all natural language processing (NLP) tasks and also for essentially all Language Technology (LT) applications. LLMs can only be trained for languages for which a sufficient amount of pre-training data is available, effectively excluding many languages that are typically characterised as under-resourced . However, there is both circumstantial and empirical evidence that multilingual LLMs, which have been trained using data sets that cover multiple languages (including under-resourced ones), do exhibit strong capabilities for some of these under-resourced languages. Eventually, this approach may have the potential to be a technological off-ramp for those under-resourced languages for which “native” LLMs – and LLM-based technologies – cannot be developed due to a lack of training data. This paper, which concentrates on European languages, examines this idea, analyses the current situation in terms of technology support and summarises related work. The article concludes by focusing on the key open questions that need to be answered for the approach to be put into practice in a systematic way. 1 Introduction Especially in today’s data-driven, machine learning and language model-based era of Language Technologies (LTs), it is intuitively evident that some languages have better, more advanced, more sophisticated technology support than others. This intuitive notion concerns not only the scope and deployment of technologies in applications or devices but also the coverage and robustness of these technologies. The natural language processing literature typically distinguishes between high-resource and low-resource , i. e., under-resourced , languages. Concentrating on the languages of Europe and the European Union (EU), we are faced with a multilingual society with 24 official EU Member State languages and more than 60 additional languages (regional and minority languages, co-official languages etc.) plus languages of immigrants, trade partners and also tourists. For decades, most European languages have been considered under-resourced, which is peculiar since all EU Member State languages are politically on equal footing according to the EU Treaty. However, technically they are clearly not – this fact has been officially recognised by the European Union ( European Parliament 2018 ) in a resolution that calls for the development of technologies and resources for all European languages, supported through a large-scale and long-term funding programme. Concurrently with the publication of the EP resolution ( European Parliament 2018 ) and the subsequent EU project European Language Equality (ELE, see Section 2 ), the full potential and groundbreaking capabilities of large language models (LLMs) have become clear to the natural language processing (NLP) and Artificial Intelligence (AI) research community and also to the public at large with a number of products such as, most notably, ChatGPT, released in November 2022. Through the development of multilingual LLMs, which have been trained on data sets covering not only one but multiple languages, there is growing evidence that such multilingual LLMs exhibit strong capabilities for some or many of the under-resourced languages they have been trained on even though only very little training data for these languages was actually included in the pre-training data set. If this does in fact hold for many or all under-resourced languages, this technical approach could become a technical off-ramp for these under-resourced languages for which our field is unable to develop “native” stand-alone technologies simply because there is too little pre-training data available. This off-ramp would not only avoid digital language extinction and digital language death, it would also bring us closer to digital language equality in Europe, maybe even in time by 2030, as initially demanded by the ELE project. This position paper attempts to examine the question if multilingual LLMs can be considered an off-ramp for under-resourced languages with a special emphasis on European languages. First, Section 2 introduces the concept of digital language equality including the current state of technology support of Europe’s languages. Section 3 concentrates on the available data including a look at the current situation and recent developments how to improve it. Section 4 introduces the idea of making use of multilingual LLMs, trained on both high as well as low-resource languages, as an off-ramp for low-resource languages. Section 5 concludes the article. 2 Digital Language Inequality in Europe In Europe’s multilingual setup, all 24 official EU languages are granted equal status by the EU Charter and the Treaty on EU. The EU is also home to over 60 regional and minority languages which have been protected and promoted under the European Charter for Regional or Minority Languages (ECRML) treaty since 1992, in addition to various sign languages and the languages of immigrants as well as trade partners. Artificial Intelligence, Natural Language Processing, Natural Language Understanding, Language Technologies, and Speech Technologies have the potential to enable multilingualism – and the multilingual European information society – technologically but, as the META-NET White Paper Series Europe’s Languages in the Digital Age ( Rehm and Uszkoreit 2012 ) found already in 2012, our languages suffer from an extreme imbalance in terms of technological support: English is very well supported through technologies, tools, data sets and corpora, but languages such as, among others, Maltese, Estonian and Icelandic have hardly any support at all. In fact, the 2012 study assessed “at least 21 European languages to be in danger of digital extinction”. If, as mentioned above, all European languages are supposed to be on an equal footing in general, technologically, they clearly are not ( Kornai 2013 ) . After the META-NET findings and several follow-up projects, studies and recommendations ( Rehm and Uszkoreit 2013 ; STOA 2017 , e. g.,) , the joint CULT/ITRE report Language Equality in the Digital Age ( European Parliament 2018 ) was eventually passed by the European Parliament with an overwhelming majority in September 2018. It concerns the improvement of the institutional framework for LT policies at the EU level, EU research and education policies to improve the future of LTs in Europe, and the extension of the benefits of LTs for both private companies and public bodies. This EP resolution also recognises that there is an imbalance in terms of technology support of Europe’s languages, that there has been a substantial amount of progress in research and technology development and that a large-scale, long-term funding programme should be established to ensure full technology support for all of Europe’s languages. The goal is to enable multilingualism technologically since “the EU and its institutions have a duty to enhance, promote and uphold linguistic diversity in Europe” ( European Parliament 2018 ) . While the resolution was an important milestone, there has been no concrete follow-up action along the lines laid out in the resolution, i. e., to set up “a large-scale, long-term coordinated funding programme for research, development and innovation in the field of language technologies, at European, national and regional levels, tailored specifically to Europe’s needs and demands” ( European Parliament 2018 ) . In the meantime, however, many highly influential breakthroughs in the area of language-centric AI have been achieved, mostly by large enterprises in the US and Asia, especially approaches and technologies concerning

Read the complete text