OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmology

Chengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai · 2025

Read OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmology by Chengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-01243.txt.

Description

Chengfeng Zhou and co-authors build OphthBench, a 2025 benchmark for testing large language models on eye medicine as practiced in China. Models are being eyed for clinic use, but their real competence and limits in this specialty were unmeasured. The benchmark follows a typical eye clinic's flow and splits it into five scenes: education, triage, diagnosis, treatment, and prognosis.

Across those scenes sit nine tasks and five hundred ninety-one questions of varied type. The team then ran thirty-nine popular models through the full set and dissected the results. A clear gap showed between what the models can do and what clinical work demands. The authors publish the framework so developers can see exactly where models fail, in which scene and which task type. They frame the benchmark as a map for closing that gap before any ward deployment.

Catalog record

CREATORChengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-01243

Browse the full library · About the Tower of Babel