Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation

Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected] · 2026

Read Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation by Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24904.txt.

Description

Klaudia-Doris Thellmann leads a check on how much translation slip in multilingual tests distorts scores for large language models. Machine-translated benchmarks are widely used, yet errors in them get little scrutiny. The team compares machine-judged error spans and a span-aware baseline against expert human marks. They then ask how far target-side errors alone explain score drops once English-side flaws are held fixed. Span agreement proves far from trivial on real benchmarks, and translation errors track steady, multi-point drops in translated accuracy.

Catalog record

CREATORKlaudia-Doris Thellmann, Bernhard Stadler, Michael Färber, Jens Lehmann [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80809
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24904

Browse the full library · About the Tower of Babel