Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data
Shaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann · 2025
Read Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data by Shaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00469.txt.
Introduction
Ji and five co-authors taught word models in twenty twenty-five to serve five hundred tongues, trained on paired texts across more than two thousand tongue pairs. Adding paired texts is a live choice in wide pre-training. They built a large paired set and four models from one base family.
Training ate up to hundreds of billions of word chunks in the runs. Tests spanned seven tasks and twelve bench sets. Paired texts helped most tongues, with the clearest gains for scarce ones. The set, models, and code were all shared in the open.
Catalog record
| CREATOR | Shaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 151 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00469