Expert-level validation of AI-generated medical text with scalable language models
Asad Aali † , Vasiliki Bikia, Maya Varma, Nicole Chiou, Sophie Ostmeier, Arnav Singhvi, Magdalini Paschali, Ashwin Kumar, Andrew Johnston, Karimar Amador-Martinez, Eduardo Juan Perez Guerrero, Paola Naovi Cruz Rivera, Sergios Gatidis, Christian Bluethgen, Eduardo Pontes Reis, Eddy D. Zandee van Rill · 2025
Read Expert-level validation of AI-generated medical text with scalable language models by Asad Aali † , Vasiliki Bikia, Maya Varma, Nicole Chiou, Sophie Ostmeier, Arnav Singhvi, Magdalini Paschali, Ashwin Kumar, Andrew Johnston, Karimar Amador-Martinez, Eduardo Juan Perez Guerrero, Paola Naovi Cruz Rivera, Sergios Gatidis, Christian Bluethgen, Eduardo Pontes Reis, Eddy D. Zandee van Rill (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03152.txt.
Description
Asad Aali and a very large team of co-authors wrote this 2025 paper on checking AI-written medical text without paying doctors to read every line. Clinicians increasingly draft with language models, yet errors hide subtly and expert review does not scale. Even frontier models judging other models miss clinically important mistakes.
Their method, MedVAL, trains evaluator models on synthetic data alone, no physician labels or gold references required, to test whether a generated text stays faithful to its input. A companion benchmark, MedVAL-Bench, holds 840 physician-annotated outputs across six real medical tasks for honest grading. Ten leading models were put through it. The authors argue distilled, self-supervised evaluators can approach expert-level validation and make clinical AI auditable at scale.
Catalog record
| CREATOR | Asad Aali † , Vasiliki Bikia, Maya Varma, Nicole Chiou, Sophie Ostmeier, Arnav Singhvi, Magdalini Paschali, Ashwin Kumar, Andrew Johnston, Karimar Amador-Martinez, Eduardo Juan Perez Guerrero, Paola Naovi Cruz Rivera, Sergios Gatidis, Christian Bluethgen, Eduardo Pontes Reis, Eddy D. Zandee van Rill |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03152