OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmology
Chengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai · 2025
Read OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmology by Chengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-01243.txt.
Description
Chengfeng Zhou and co-authors build OphthBench, a 2025 benchmark for testing large language models on eye medicine as practiced in China. Models are being eyed for clinic use, but their real competence and limits in this specialty were unmeasured. The benchmark follows a typical eye clinic's flow and splits it into five scenes: education, triage, diagnosis, treatment, and prognosis.
Across those scenes sit nine tasks and five hundred ninety-one questions of varied type. The team then ran thirty-nine popular models through the full set and dissected the results. A clear gap showed between what the models can do and what clinical work demands. The authors publish the framework so developers can see exactly where models fail, in which scene and which task type. They frame the benchmark as a map for closing that gap before any ward deployment.
Catalog record
| CREATOR | Chengfeng Zhou, Ji Wang, Juanjuan Qin, Yining Wang, Ling Sun, Weiwei Dai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-01243