Process-Oriented, Behaviorally Anchored Assessment of Clinical Reasoning in Large Language Models and the Effect of Extended Thinking: Protocol for a Prospective, Multigroup, Comparative Study

Adnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu · 2026

Read Process-Oriented, Behaviorally Anchored Assessment of Clinical Reasoning in Large Language Models and the Effect of Extended Thinking: Protocol for a Prospective, Multigroup, Comparative Study by Adnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13643257.txt.

Description

Adnan Agha and five colleagues present a 2026 study protocol for judging the reasoning process of medical language models, not just their final answers. Most tests score multiple-choice accuracy and say little about how a model thinks, even as extended-thinking modes spread.

Their tool, REACT-AI, rates externalized reasoning across thirteen subdomains with behaviorally anchored scales. Six flagship models will answer urgent care cases in standard and extended modes, scored blind by human experts and by a guarded model-judge pipeline. Later, the validated pipeline will grade 3,600 outputs. Student responses were already collected under seal by mid 2026.

Catalog record

CREATORAdnan Agha, Muhammad Jalil, Eram Anwar, Omran Bakoush, Javad Sarvestan, Charlotte Ahmadu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
LICENSEhttps://creativecommons.org/licenses/by/4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/article/MED/PMC13643257

Browse the full library · About the Tower of Babel