Litmus (Re)Agent : A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models

Avni Mittal, Shanu Kumar, Sandipan Dandapat, Monojit Choudhury · 2026

Read Litmus (Re)Agent : A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models by Avni Mittal, Shanu Kumar, Sandipan Dandapat, Monojit Choudhury (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08970.txt.

Description

Mittal and three colleagues study predictive multilingual evaluation: guessing how a model will do on a task in a language with no benchmark scores. Coverage is sparse and evidence uneven across languages, tasks, and model families. Their benchmark holds 1,500 questions across six tasks and five evidence scenarios, keeping accessible evidence separate from ground truth. Their Litmus (Re)Agent system breaks queries into hypotheses, retrieves evidence, and merges predictions by feature-aware aggregation. It beats five rivals overall, with the biggest gains where direct evidence is weak or absent.

Catalog record

CREATORAvni Mittal, Shanu Kumar, Sandipan Dandapat, Monojit Choudhury
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77631
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.08970

Browse the full library · About the Tower of Babel