Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation
Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected] · 2026
Read Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation by Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24904.txt.
Description
Klaudia-Doris Thellmann leads a check on how much translation slip in multilingual tests distorts scores for large language models. Machine-translated benchmarks are widely used, yet errors in them get little scrutiny. The team compares machine-judged error spans and a span-aware baseline against expert human marks. They then ask how far target-side errors alone explain score drops once English-side flaws are held fixed. Span agreement proves far from trivial on real benchmarks, and translation errors track steady, multi-point drops in translated accuracy.
Catalog record
| CREATOR | Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80809 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24904