Lost in Translation: Do LVLM Judges Generalize Across Languages?
Md Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang · 2026
Read Lost in Translation: Do LVLM Judges Generalize Across Languages? by Md Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19405.txt.
Description
Laskar and colleagues introduce MM-JudgeBench, a first large-scale test for judge models across languages and modalities. Auto judges like reward models matter for aligning large vision-language models, yet they are almost always tested in English only. The test holds over 60,000 pairwise preference cases spanning 25 diverse languages. It pairs a general vision-language subset with a chart-centric reasoning subset.
Catalog record
| CREATOR | Md Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59204 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.19405