MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems

Xinwu Ye, Chengfan Li, Siming Chen, Wei Wei, Xiangru Tang · 2025

Read MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems by Xinwu Ye, Chengfan Li, Siming Chen, Wei Wei, Xiangru Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01891.txt.

Introduction

Xinwu Ye and four co-authors build a test set for science skill in Chinese language models in a 2025 paper. Top models now read images and text at once, yet their science logic skill was untested in that mixed mode. Their set holds math and physics problems in Chinese, each in text-only and text-plus-image form. Human notes add a grade of hardness, a full worked answer, and a topic tag for each problem.

The best model tried scored just under 64 percent on the set. Image problems tripped models most, since text and image clues must be read as one. Hard problems split the field far more than easy ones. Code for the set is free on a public code site, and the data sit on a shared model hub. The team frames the set as a hard bar for next models.

Catalog record

CREATORXinwu Ye, Chengfan Li, Siming Chen, Wei Wei, Xiangru Tang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE141 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01891

Browse the full library · About the Tower of Babel