Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare

Max Lamparth, Declan Grabb, Amy Franks, Scott Gershan, Kaitlyn N. Kunstman, Aaron Lulla, Monika Drummond Roots, Manu Sharma, Aryan Shrivastava, Nina Vasan, Colleen Waickman · 2025

Read Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare by Max Lamparth, Declan Grabb, Amy Franks, Scott Gershan, Kaitlyn N. Kunstman, Aaron Lulla, Monika Drummond Roots, Manu Sharma, Aryan Shrivastava, Nina Vasan, Colleen Waickman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16051.txt.

Introduction

Max Lamparth and ten co-authors build a fairness test set from real mental health work in a 2025 paper. Exam-style benchmarks miss the mess of daily practice. Psychiatry also carries sharp bias risks for language models. Their dataset comes from clinicians, with no model help.

Five decision domains anchor the set of tasks. Treatment, diagnosis, notes, monitoring, and triage are covered. Demographic details are swapped out for variables by design. Each case can be re-skinned across gender and background. Ambiguous cases carry expert preference data with uncertainty. Twenty-two models were run through the set as a check. Demographics swayed some model choices unfairly.

Catalog record

CREATORMax Lamparth, Declan Grabb, Amy Franks, Scott Gershan, Kaitlyn N. Kunstman, Aaron Lulla, Monika Drummond Roots, Manu Sharma, Aryan Shrivastava, Nina Vasan, Colleen Waickman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE99 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16051

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare Max Lamparth, Declan Grabb, Amy Franks, Scott Gershan, Kaitlyn N. Kunstman, Aaron Lulla, Monika Drummond Roots, Manu Sharma, Aryan Shrivastava, Nina Vasan, Colleen Waickman Abstract Current medical language model (LM) benchmarks often over-simplify the complexities of day-to-day clinical practice tasks and instead rely on evaluating LMs on multiple-choice board exam questions. In psychiatry especially, these challenges are worsened by fairness and bias issues, since models can be swayed by patient demographics even when those factors should not influence clinical decisions. Thus, we present an expert-created and annotated dataset spanning five critical domains of decision-making in mental healthcare: treatment, diagnosis, documentation, monitoring, and triage. This U.S.-centric dataset — created without any LM assistance — is designed to capture the nuanced clinical reasoning and daily ambiguities mental health practitioners encounter, reflecting the inherent complexities of care delivery that are missing from existing datasets. Almost all base questions with five answer options each have had the decision-irrelevant demographic patient information removed and replaced with variables, e.g., for age or ethnicity, and are available for male, female, or non-binary-coded patients. This design enables systematic evaluations of model performance and bias by studying how demographic factors affect decision-making. For question categories dealing with ambiguity and multiple valid answer options, we create a preference dataset with uncertainties from the expert annotations. We outline a series of intended use cases and demonstrate the usability of our dataset by evaluating sixteen off-the-shelf and six (mental) health fine-tuned LMs on category-specific task accuracy, on the fairness impact of patient demographic information on decision-making, and how consistently free-form responses deviate from human-annotated samples. 1 Introduction Benchmarks in medical AI are pivotal for gauging progress and guiding model development. Evaluations typically rely on medical student or specialty board-style exams ( Jin et al., 2021 ; Pal et al., 2022 , e.g.) . However, even for humans, numerous studies indicate that success in these standardized tests only weakly correlates with clinicians’ real-world performance Saguil et al. (2015) ; Murphy (2023) ; Murphy (2024) , a disconnect that can be especially problematic in psychiatry, where diagnosis and management hinge on subjective judgments and interpersonal nuances. Recent findings underscore this need for more grounded, task-specific benchmarks in mental health Raji et al. (2025) . Although traditional exams emphasize factual knowledge, effective psychiatric practice demands a broader range of skills, from titrating medication to deciding on emergent hospitalization (see Appendix C for an extensive discussion on the limitations of medical exam-style questions). While newer benchmarks such as MedS-bench ( Wu et al., 2025 ) emphasize high-level clinical tasks, psychiatry-specific evaluations remain limited, particularly those co-created by clinicians and human experts who navigate the daily ambiguities inherent to mental healthcare. To address this gap, we introduce MENTAT ( MENtal health Tasks AssessmenT )—a dataset and evaluation framework focused squarely on the pragmatic, real-world tasks in psychiatry designed and annotated by mental health clinicians. Our expert-curated approach departs from standardized exam-style questions in several ways: (1) it emphasizes genuine clinical tasks such as triage, diagnosis, treatment, monitoring, and documentation; (2) it captures the inherent ambiguities in mental healthcare via multiple plausible answer options and preference annotations with uncertainties rather than enforcing a single “correct” fact-based response for two categories (triage and documentation); and (3) it leverages a diverse team of practicing psychiatrists to mitigate biases and ensure the relevance of each question to everyday clinical practice. In this paper, we present MENTAT, describe its design and creation process, and demonstrate its utility comparing sixteen off-the-shelf and six fine-tuned language models (LMs) in multiple-choice and free-form settings, with a specific focus on patient demographic sensitivity in decision-making performance to evaluate biases induced by patient demographic information. We also examine how MENTAT can serve as a “ground-truth" reference for gauging model consistency in open-ended clinical responses. In contrast to most medical benchmarks that assess fact recall, our dataset targets decision-making performance, a critical yet challenging aspect of real-world psychiatry. In summary, our key contributions are: • We introduce MENTAT, an expert-curated dataset that emphasizes real-world psychiatric ambiguities over exam-like fact recall across five mental healthcare practice domains: diagnosis, treatment, monitoring, triage, and documentation. • We provide a hierarchical annotation pipeline, open licensing, and detailed coverage that allow for straightforward adjustments and support multiple evaluation paradigms to empower future work. • We outline several use cases of MENTAT and demonstrate its applicability by evaluating decision-making accuracy across MENTAT’s five categories, how performance is impacted by patient demographic information (bias), and how using MENTAT as a ground-truth reference can be valuable when evaluating free-form LM outputs. • We find significant differences in decision-making quality and lack of fairness resulting from sensitivity to patient demographic information across tested models. The datasets with the annotation analysis pipeline are publicly available on GitHub 1 1 1 github.com/maxlampe/mentat (MIT license). 2 Related Work Numerous benchmarks and datasets have been introduced to train or evaluate AI systems for medical applications ranging from genetics, radiology, cardiology, and EMR applications Shang et al. (2025) ; Chaves et al. (2023) ; Oh et al. (2023) to medical exam-like content such as MedQA Jin et al. (2021) , MMMU Yue et al. (2024) , NEJM Image Challenges The New England Journal of Medicine (2024) , and Path-VQA He et al. (2021b) , alongside exam-based tasks like MedMCQA ( Pal et al., 2022 ) and MMLU ( Hendrycks et al., 2021 ) . Broader efforts include MedS-bench ( Wu et al., 2025 ) , a large dataset constructed through web scraping and LM-generating a synthetic data set of clinical tasks, and Google’s Gemini initiative Saab et al. (2024) or state-of-the-art graduate-level and human expert benchmarks ( Rein et al., 2024 ; Center for AI Safety et al., 2026 ) . In mental health, researchers have compiled datasets of counseling sessions ( Adhikary et al., 2024 ) , explored AI-driven diagnostic reasoning ( Tu et al., 2025 ) , and automated clinical documentation ( Falcetta et al., 2023 ; Axios, 2024 ) . They have also investigated therapy referrals ( Sin, 2024 ; Habicht et al., 2024 ) , peer support ( Sharma et al., 2023 ) , patient attitudes ( Pataranutaporn et al., 2023 ) , and augmented care via automated psychotherapy, diagnosis, and biometric stress analysis ( Higgins et al., 2023 ; Thieme et al., 2023 ; Li et al., 2023 ; Balan et al., 2024 ; Kasula, 2023 ; Ates et al., 2024 ) , with broader safety considerations ( Ganguli et al., 2022 ; Wang et al., 2023 ; Zhang et al., 2024 ; Liu et al., 2025 ) , concrete safety concerns in mental health emergencies ( Grabb et al., 2024 ) , and demographic biases ( Zack et al., 2024 ; Gabriel et al., 2024 ; Moore et al., 2025 ) remaining active concerns. Unlike the existing exam-style benchmarks (which face known limitations with multiple-choice formats ( Griot et al., 2025 ) ) and multi-specialty medical datasets, our work focuses specifically on capturing the everyday ambiguities of mental healthcare tasks that often lack a single “correct” answer supported by extensive human expert input without intentionally contaminating the data with LM

Read the complete text