Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation

Wayner Barrios, SouYoung Jin · 2026

Read Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation by Wayner Barrios, SouYoung Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13099.txt.

Description

Wayner Barrios and SouYoung Jin introduce CRYSTAL, a benchmark that grades multimodal reasoning by its steps, not just final answers. Its 6,372 cases carry verifiable intermediate steps for scoring. Two metrics check step precision and recall, with a penalty for steps out of order. Four models drafted the reference chains, which were clustered and passed through human gates. Tests of 20 models exposed cherry-picking and jumbled chains that answer scores hide. A causal process reward then lifted step scores by 32% in training.

Catalog record

CREATORWayner Barrios, SouYoung Jin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118553
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13099

Browse the full library · About the Tower of Babel