Lost in Translation: Do LVLM Judges Generalize Across Languages?

Md Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang · 2026

Read Lost in Translation: Do LVLM Judges Generalize Across Languages? by Md Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19405.txt.

Description

Laskar and colleagues introduce MM-JudgeBench, a first large-scale test for judge models across languages and modalities. Auto judges like reward models matter for aligning large vision-language models, yet they are almost always tested in English only. The test holds over 60,000 pairwise preference cases spanning 25 diverse languages. It pairs a general vision-language subset with a chart-centric reasoning subset.

Catalog record

CREATORMd Tahmid Rahman Laskar, Mohammed Saidul Islam,, Mir Tafseer Nayeem, Md Amran Bhuiyan, Mizanur Rahman,, Shafiq Joty, Enamul Hoque , Jimmy Xiangji Huang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59204
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19405

Browse the full library · About the Tower of Babel