Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation
Wayner Barrios, SouYoung Jin · 2026
Read Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation by Wayner Barrios, SouYoung Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13099.txt.
Description
Wayner Barrios and SouYoung Jin introduce CRYSTAL, a benchmark that grades multimodal reasoning by its steps, not just final answers. Its 6,372 cases carry verifiable intermediate steps for scoring. Two metrics check step precision and recall, with a penalty for steps out of order. Four models drafted the reference chains, which were clustered and passed through human gates. Tests of 20 models exposed cherry-picking and jumbled chains that answer scores hide. A causal process reward then lifted step scores by 32% in training.
Catalog record
| CREATOR | Wayner Barrios, SouYoung Jin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 118553 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13099