The Roles of English in Evaluating Multilingual Language Models

Wessel Poelman, Miryam de Lhoneux · 2024

Read The Roles of English in Evaluating Multilingual Language Models by Wessel Poelman, Miryam de Lhoneux (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08392.txt.

Description

Wessel Poelman and Miryam de Lhoneux wrote a position paper in 2024 on the place of English in tests of language models that work in many tongues. English is often used to prompt such models, in part because training text in other tongues is scarce. The authors argue that this habit mixes up two distinct jobs. One job is getting a task done, and the other is truly grasping a language.

Through examples from test sets and test setups, the paper shows how the two jobs pull in opposed ways. Using English as a prompt layer may lift task scores while hiding weak grasp of the target tongue. The authors urge the field to move away from this loose practice and focus on real language grasp.

Catalog record

CREATORWessel Poelman, Miryam de Lhoneux
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-08392

Browse the full library · About the Tower of Babel