Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data

Shaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann · 2025

Read Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data by Shaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00469.txt.

Introduction

Ji and five co-authors taught word models in twenty twenty-five to serve five hundred tongues, trained on paired texts across more than two thousand tongue pairs. Adding paired texts is a live choice in wide pre-training. They built a large paired set and four models from one base family.

Training ate up to hundreds of billions of word chunks in the runs. Tests spanned seven tasks and twelve bench sets. Paired texts helped most tongues, with the clearest gains for scarce ones. The set, models, and code were all shared in the open.

Catalog record

CREATORShaoxiong Ji, Zihao Li, Jaakko Paavola, Indraneil Paul, Hengyu Luo, Jörg Tiedemann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE151 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00469

Browse the full library · About the Tower of Babel