Using randomization to compare AI and expert-generated formative assessment questions in medical education
Akshat Kumar, Katie Stinson, Litao Wang, Elizabeth Nugent, Yongsu Lee, Mark Hormann, LaTanya Jones Love, Toufeeq Ahmed Syed · 2026
Read Using randomization to compare AI and expert-generated formative assessment questions in medical education by Akshat Kumar, Katie Stinson, Litao Wang, Elizabeth Nugent, Yongsu Lee, Mark Hormann, LaTanya Jones Love, Toufeeq Ahmed Syed (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13159578.txt.
Description
Akshat Kumar and seven co-authors compare AI-made and expert-made test questions for medical students in a 2026 paper. AI tools can now write complex multiple-choice questions. Judging their quality against expert work is hard with old methods. The team used random assignment for a fair test.
Students got one AI or expert question each day for four weeks. Scores did not differ between the two question sets. Student views of the two sets also matched closely. But 53 percent of AI questions were rated easy or very easy. Only 31 percent of expert questions got that rating. The authors say new checks are needed beyond student feel.
Catalog record
| CREATOR | Akshat Kumar, Katie Stinson, Litao Wang, Elizabeth Nugent, Yongsu Lee, Mark Hormann, LaTanya Jones Love, Toufeeq Ahmed Syed |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |