The Roles of English in Evaluating Multilingual Language Models
Wessel Poelman, Miryam de Lhoneux · 2024
Read The Roles of English in Evaluating Multilingual Language Models by Wessel Poelman, Miryam de Lhoneux (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08392.txt.
Description
Wessel Poelman and Miryam de Lhoneux wrote a position paper in 2024 on the place of English in tests of language models that work in many tongues. English is often used to prompt such models, in part because training text in other tongues is scarce. The authors argue that this habit mixes up two distinct jobs. One job is getting a task done, and the other is truly grasping a language.
Through examples from test sets and test setups, the paper shows how the two jobs pull in opposed ways. Using English as a prompt layer may lift task scores while hiding weak grasp of the target tongue. The authors urge the field to move away from this loose practice and focus on real language grasp.
Catalog record
| CREATOR | Wessel Poelman, Miryam de Lhoneux |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08392