Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgish

Cedric Lothritz, Jordi Cabot, Laura Bernardy · 2025

Read Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgish by Cedric Lothritz, Jordi Cabot, Laura Bernardy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01667.txt.

Description

Cedric Lothritz, Jordi Cabot, and Laura Bernardy wrote this 2025 paper testing language models in Luxembourgish, a small language the industry mostly ignores. Evaluation data for such languages barely exists, so the team borrowed a ready-made yardstick: official language proficiency exams, the kind used to certify human learners.

Run through the exams, the big frontier models, Claude and DeepSeek-R1 among them, scored high, while smaller models flopped. More useful still, exam performance predicted how the same models did on other Luxembourgish language tasks. The authors argue proficiency tests are a cheap, credible benchmark for low-resource languages, no custom data set required. The case study doubles as a warning about which languages get left behind.

Catalog record

CREATORCedric Lothritz, Jordi Cabot, Laura Bernardy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01667

Browse the full library · About the Tower of Babel