Process-Oriented, Behaviorally Anchored Assessment of Clinical Reasoning in Large Language Models and the Effect of Extended Thinking: Protocol for a Prospective, Multigroup, Comparative Study
Adnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu · 2026
Read Process-Oriented, Behaviorally Anchored Assessment of Clinical Reasoning in Large Language Models and the Effect of Extended Thinking: Protocol for a Prospective, Multigroup, Comparative Study by Adnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13643257.txt.
Description
Adnan Agha and five colleagues present a 2026 study protocol for judging the reasoning process of medical language models, not just their final answers. Most tests score multiple-choice accuracy and say little about how a model thinks, even as extended-thinking modes spread.
Their tool, REACT-AI, rates externalized reasoning across thirteen subdomains with behaviorally anchored scales. Six flagship models will answer urgent care cases in standard and extended modes, scored blind by human experts and by a guarded model-judge pipeline. Later, the validated pipeline will grade 3,600 outputs. Student responses were already collected under seal by mid 2026.
Catalog record
| CREATOR | Adnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |