small Models, BIG Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages

Daniil Gurgurov, Ivan Vykopal, Josef van Genabith, Simon Ostermann · 2025

Read small Models, BIG Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages by Daniil Gurgurov, Ivan Vykopal, Josef van Genabith, Simon Ostermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10140.txt.

Description

Daniil Gurgurov and three co-authors study how to adapt small language models for tongues with little training text, in a 2025 paper. Big models still do poorly on many such tongues. Small models trained on many tongues may fit better, since their size suits small data sets.

The team tested three light add-on methods that change few weights. They trained on plain text and on facts from a concept graph, using less than a modest store of text. Small data gave clear gains on word tests and on tasks like topic sorting and name finding. Light add-ons matched full retraining while using far fewer changed weights.

Catalog record

CREATORDaniil Gurgurov, Ivan Vykopal, Josef van Genabith, Simon Ostermann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10140

Browse the full library · About the Tower of Babel