MedErrBench: A Fine-Grained Multilingual Benchmark for Medical Error Detection and Correction with Clinical Expert Annotations

Congbo Ma, Yichun Zhang, Yousef Al-Jazzazi, Ahamed Foisal, Laasya Sharma, Yousra Sadqi, Khaled Saleh, Jihad Mallat, Farah E. Shamout · 2026

Read MedErrBench: A Fine-Grained Multilingual Benchmark for Medical Error Detection and Correction with Clinical Expert Annotations by Congbo Ma, Yichun Zhang, Yousef Al-Jazzazi, Ahamed Foisal, Laasya Sharma, Yousra Sadqi, Khaled Saleh, Jihad Mallat, Farah E. Shamout (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05692.txt.

Description

Errors in clinical text, such as a wrong diagnosis or treatment, can seriously harm patients, and language models now write much of that text. Congbo Ma and 8 co-authors introduce MedErrBench, a benchmark for detecting, locating, and correcting medical errors, built with experienced clinicians. It rests on a taxonomy of ten common error types and covers English, Arabic, and Chinese.

Cases are natural clinical texts, annotated and reviewed by domain experts. The team tested general purpose, language specific, and medical language models on all three tasks. Results show notable gaps, worst in the non English settings, which the authors link to weak clinical grounding and language awareness. The benchmark and its evaluation protocols are public. The authors aim to push multilingual clinical language technology toward safer, fairer AI in health care worldwide. The data set is released through supplementary material and a public repository.

Catalog record

CREATORCongbo Ma, Yichun Zhang, Yousef Al-Jazzazi, Ahamed Foisal, Laasya Sharma, Yousra Sadqi, Khaled Saleh, Jihad Mallat, Farah E. Shamout
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05692

Browse the full library · About the Tower of Babel