Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgish
Cedric Lothritz, Jordi Cabot, Laura Bernardy · 2025
Read Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgish by Cedric Lothritz, Jordi Cabot, Laura Bernardy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01667.txt.
Description
Cedric Lothritz, Jordi Cabot, and Laura Bernardy wrote this 2025 paper testing language models in Luxembourgish, a small language the industry mostly ignores. Evaluation data for such languages barely exists, so the team borrowed a ready-made yardstick: official language proficiency exams, the kind used to certify human learners.
Run through the exams, the big frontier models, Claude and DeepSeek-R1 among them, scored high, while smaller models flopped. More useful still, exam performance predicted how the same models did on other Luxembourgish language tasks. The authors argue proficiency tests are a cheap, credible benchmark for low-resource languages, no custom data set required. The case study doubles as a warning about which languages get left behind.
Catalog record
| CREATOR | Cedric Lothritz, Jordi Cabot, Laura Bernardy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-01667