Assessing the Quality of Mental Health Support in LLM Responses through Multi-Attribute Human Evaluation

Abeer Badawi, Md Tahmid Rahman Laskar, Elahe Rahimi, Sheri Grach, Lindsay Bertrand, Lames Danok, Frank Rudzicz, Jimmy Huang, Elham Dolatabadi · 2026

Read Assessing the Quality of Mental Health Support in LLM Responses through Multi-Attribute Human Evaluation by Abeer Badawi, Md Tahmid Rahman Laskar, Elahe Rahimi, Sheri Grach, Lindsay Bertrand, Lames Danok, Frank Rudzicz, Jimmy Huang, Elham Dolatabadi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18630.txt.

Introduction

Abeer Badawi and eight colleagues grade how well language models help with health of the mind. More people seek help than there are trained staff to give it. That gap makes smart chat tools a tempting path to help at scale. A chat tool can be warm and quick, yet warmth is not the same as good care. Its words must be safe, apt, and in line with what trained staff would prize.

The team grade chats on more than one trait at once, with human judges. Traits can pull in unlike ways: a warm reply may still be unsafe. A score per trait shows where a tool is strong and where it is frail. Such grades could help clinics pick tools with care. They could also set a bar that tools must clear before they meet the public.

Catalog record

CREATORAbeer Badawi, Md Tahmid Rahman Laskar, Elahe Rahimi, Sheri Grach, Lindsay Bertrand, Lames Danok, Frank Rudzicz, Jimmy Huang, Elham Dolatabadi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18630

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Assessing the Quality of Mental Health Support in LLM Responses through Multi-Attribute Human Evaluation Abeer Badawi, Md Tahmid Rahman Laskar, Elahe Rahimi, Sheri Grach, Lindsay Bertrand, Lames Danok, Frank Rudzicz, Jimmy Huang, Elham Dolatabadi Abstract The escalating global mental health crisis, marked by persistent treatment gaps, availability, and a shortage of qualified therapists, positions Large Language Models (LLMs) as a promising avenue for scalable support. While LLMs offer potential for accessible emotional assistance, their reliability, therapeutic relevance, and alignment with human standards remain challenging to address. This paper introduces a human-grounded evaluation methodology designed to assess LLM-generated responses in therapeutic dialogue. Our approach involved curating a dataset of 500 mental health conversations from datasets with real-world scenario questions and evaluating the responses generated by nine diverse LLMs, including closed-source and open-source models. More specifically, these responses were evaluated by two psychiatric-trained experts, who independently rated each on a 5-point Likert scale across a comprehensive 6-attribute rubric. This rubric systematically captures Cognitive Support (Guidance, Informativeness, Safety) and Affective Resonance (Empathy, Helpfulness, Interpretation), providing a multi-dimensional perspective on therapeutic quality. Our analysis reveals that LLMs provide strong cognitive reliability by producing safe, coherent, and clinically appropriate information, but they demonstrate unstable affective alignment, particularly in empathy and helping. Although closed-source models (e.g., GPT-4o) offer balanced therapeutic responses, open-source models show greater variability and emotional flatness. We reveal a persistent cognitive–affective gap and highlight the need for failure-aware, clinically grounded evaluation frameworks that prioritize relational sensitivity alongside informational accuracy in mental-health-oriented LLMs. We advocate for balanced evaluation protocols with human-in-the-loop that center on therapeutic sensitivity and provide a framework to guide the responsible design and clinical oversight of mental-health-oriented conversational AI. Introduction Generative AI, particularly Large Language Models (LLMs), has emerged as a transformative technology with applications spanning numerous domains. In recent years, LLMs have been extensively explored for mental health applications, ranging from detection ( 23 ) and screening ( 35 ) tasks to their use as conversational agents for support and intervention ( 24 ; 18 ) . These studies have shown that LLMs can outperform traditional baselines and, in some contexts, demonstrate empathy superior to that of human peers ( 39 ) . Despite this potential, significant limitations and safety concerns preclude the deployment of LLMs in real-world clinical scenarios ( 17 ) . Recent research has identified critical issues, including demographic and racial biases in responses ( 8 ) , a lack of clinical judgment ( 14 ) , and dangerous failures such as the underestimation of suicide risks in high-risk scenarios ( 12 ; 20 ) . For example, OpenAI released that about 0.15% of weekly active users have conversations that include explicit indicators of suicidal planning or intent, noting that ChatGPT has on the order of 800 million weekly active users ( 32 ) . A central challenge in addressing these flaws is the lack of robust, clinically-grounded evaluation methods. Many current studies rely on data from social media platforms like Reddit, with a notable absence of data from authentic therapeutic interactions ( 21 ; 15 ) . Furthermore, evaluation often employs automatic metrics like BLEU ( 33 ) and ROUGE ( 26 ) , which are ill-suited for measuring nuanced therapeutic qualities. This highlights a critical need for new evaluation frameworks that utilize expert human-in-the-loop assessment to ensure clinical reliability and safety. We address these limitations through four objectives that we build on a previously established framework for mental health LLM assessment ( 6 ) . First, we select 500 counseling conversations from MentalChat16K ( 36 ) , EmoCare ( 37 ) , and CounselChat ( 27 ) , with real user messages, consisting of genuine help-seeking conversations in which some excerpts have been lightly rephrased or de-identified using AI to preserve privacy while maintaining the original intent. Second, we systematically compare nine diverse LLMs (e.g., GPT-4o, Gemini 2.0-Flash, Claude 3.5 Haiku, Llama 3.1, Qwen, DeepSeek) under identical psychiatrist-style prompting to generate 4,500 responses, enabling controlled model-to-model comparison. Third, two psychiatric-trained evaluators rate every response on six therapeutic attributes (Guidance, Informativeness, Safety, Interpretation, Empathy, Helpfulness) using a 5-point Likert scale, which we summarize into Cognitive Support (factual/structural quality) and Affective Resonance (emotional alignment) ( 25 ) . Finally, we quantify reliability and risk by analyzing both average quality (mean \pm std) and failure modes (frequency of \leq 3 and \geq 4 out of 5 scores), and we measure which models clinicians actually prefer in practice. Our study reveals promising results, yet also shortcomings that warrant caution when using large language models for mental health support. Across all nine models and all 4,500 responses, overall human-rated quality reaches 4.29 \pm 0.47 (1–5 scale), and top models score above 4.8/5 on Safety and Interpretation. But affective reliability is far less stable: some models receive low ( \leq 3/5) scores for Empathy or Helpfulness. The evaluation also shows a clear performance hierarchy: GPT-4o is the strongest overall, followed by Gemini 2.0-Flash and GPT-4o-mini, while Claude-3.5-Haiku lags other closed models. Llama-3.1 is the best open-weight model; Qwen-3 ranks last, with weak cognitive and affective support. Derived from this expert human evaluation, we reveal a persistent gap between the cognitive and affective attributes. While the evaluated LLMs demonstrate strong cognitive reliability by performing exceptionally well on Safety and Interpretation, they show significant inconsistency in affective skills like Empathy and Helpfulness. We describe this gap as the cognitive–affective gap: LLMs can generate advice that is structured and safe, yet emotionally misattuned in moments that require empathy and containment. To summarize, this work makes the following contributions: (1) A Human-Grounded Evaluation Framework: We conduct a robust evaluation framework for LLM-generated mental health responses, based on 500 counseling conversations from three datasets and assessed by two psychiatric-trained domain experts across a comprehensive 6-attribute rubric, encompassing both Cognitive Support and Affective Resonance following ( 6 ) framework. (2) Nuanced Analysis of LLM Therapeutic Capabilities and Failures: We provide a fine-grained analysis of the 9 diverse LLMs, identifying their strengths across various therapeutic attributes while quantifying the frequency and nature of low-quality responses. Our work exposes inconsistencies in LLM reliability and systematically demonstrates how certain models produce unsafe or emotionally misaligned content, revealing a critical ”cognitive-affective gap” in current LLM performance for mental health. (3) Advocacy for Failure-Aware Evaluation in mental health AI: Our study makes a compelling case for shifting evaluation paradigms in mental health AI by highlighting therapeutic risks and inconsistencies. We advocate for methodologies that prioritize safety and robustness over aggregate performance benchmarks, providing actionable insights for the ethical design, development, and deployment of LLM-based mental health support systems. Related Work Recent research has explored LLMs for mental health detection and screening tasks. Six studies in the current literature focus specifically on detection and screening applications, particularly targeting depression, suicide risk, and general mental health conditions ( 16 ; 22 ; 41 ; 40 ; 30 ) . These studies consistently report that LLMs outperform traditional neural network baselines, especially when enhanced with techniques such as emotion-enhanced prompting or fine-tuning approaches. 41 demonstrated that ChatGPT achieves superior performance among examined LLMs, though it still underperforms compared to advanced supervised methods specifically designed for mental health tasks. Similarly, 30 found that ChatGPT improved F-score

Read the complete text