MASEval: Extending Multi-Agent Evaluation from Models to Systems

Cornelius Emde, Alexander Rubinstein, Anmol Goel, Ahmed Heakl, Sangdoo Yun, Seong Joon Oh, Martin Gubri, §Equal contribution. Parameter Lab, University of Oxford, University of Tübingen · 2026

Read MASEval: Extending Multi-Agent Evaluation from Models to Systems by Cornelius Emde, Alexander Rubinstein, Anmol Goel, Ahmed Heakl, Sangdoo Yun, Seong Joon Oh, Martin Gubri, §Equal contribution. Parameter Lab, University of Oxford, University of Tübingen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08835.txt.

Description

Cornelius Emde and colleagues judge whole agentic systems, not just their models. Current benchmarks fix the agent setup, hiding how topology, orchestration, and error handling shape results. Their MASEval library compares full systems across three benchmarks, three models, and three frameworks, finding that picking a framework swings results as much as picking a model.

Catalog record

CREATORCornelius Emde, Alexander Rubinstein, Anmol Goel, Ahmed Heakl, Sangdoo Yun, Seong Joon Oh, Martin Gubri, §Equal contribution. Parameter Lab, University of Oxford, University of Tübingen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39703
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08835

Browse the full library · About the Tower of Babel