Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling

Keqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li · 2025

Read Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling by Keqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03607.txt.

Introduction

Keqi Chen and five co-authors present Psy-Insight, a data set for mental health counseling in two thousand twenty-five. Language models could aid mental health support through in-context learning. Data for training them is thin, most of all in Chinese. Psy-Insight is billed as the first data set of its kind.

It holds face to face, multi turn counseling talks in two languages. Tags cover therapy type, emotion, strategy, and topic. Each turn has its reasons spelled out, and each session has guidance. Tests show trained models copy style and grasp strategy too. The set can train models to act as kind counselors with clear logic.

Catalog record

CREATORKeqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03607

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling Keqi Chen, Zekai Sun, Yuhua Wen, Huijun Lian, Yingming Gao, Ya Li Abstract The in-context learning capabilities of large language models (LLMs) show great potential in mental health support. However, the lack of counseling datasets, particularly in Chinese corpora, restricts their application in this field. To address this, we constructed Psy-Insight, the first mental health-oriented explainable multi-task bilingual dataset. We collected face-to-face multi-turn counseling dialogues, which are annotated with multi-task labels and conversation process explanations. Our annotations include psychotherapy, emotion, strategy, and topic labels, as well as turn-level reasoning and session-level guidance. Psy-Insight is not only suitable for tasks such as label recognition but also meets the need for training LLMs to act as empathetic counselors through logical reasoning. Experiments show that training LLMs on Psy-Insight enables the models to not only mimic the conversation style but also understand the underlying strategies and reasoning of counseling. Our code, expert evaluation results, and the Psy-Insight dataset have been open-sourced 1 1 1 https://ckqqqq.github.io/Demo/Psy-Insight/ . 1 Introduction According to a report from the World Health Organization WHO (2022) , 71% of individuals in low-income countries suffering from mental disorders are unable to receive timely treatment. Patients with mental disorders require affordable and easily accessible mental health support. Mental support chatbots offer a possible solution Liu et al. (2023) . General-purpose large language models (LLMs) such as ChatGPT OpenAI (2023) and Bard Google (2023) have shown their potential in various conversations, which include enhanced in-context learning Min et al. (2022) and chain-of-thought Wei et al. (2022) capabilities for multiple tasks. This allows LLMs to effectively handle the complex situations encountered in real-life multi-turn counseling. For mental-supported LLMs, there is a need for suitable dialogue datasets for finetuning. However, the potential of large models has not been fully explored on previous counseling datasets. Firdaus et al. (2020a) ; Firdaus et al. (2020b) . These datasets are characterized by short labels for single-task annotations, such as emotion or entity labels, which are designed for subtasks within the traditional pipeline chatbots, such as emotion classification Manisha et al. (2023) named entity recognition Nadeau and Sekine (2007) , and knowledge graph completion Peng et al. (2022) . However, the strength of LLMs lies not only in their ability to recognize and classify but also in their potential to reason step-by-step. They can capture analysis and explanation from therapists beyond what is described by short labels (eg., “Beth fears that sharing could worsen things” vs “Fear”). Therefore, when designing datasets for mental support LLMs, these capabilities should be reflected in the datasets. The focus should not only be on short labels for simple tasks but also on reasoning annotations that describe the thought process for complex task scenarios. Figure 1: The left section presents Psy-Insight’s counseling dialogues and annotations, while the right section illustrates the corresponding multi-tasks for these annotations. The Psy-Insight dataset features 951 sessions of multi-turn counseling dialogues annotated with step-by-step reasoning and multi-task labels. Within a session, the therapist and client engage in 56̃0 turns of dialogue on a single topic. We have annotated counseling dialogues at various granularity levels. The example of Chinese data is shown in Table 10 Motivated by the goals, we constructed the Psy-Insight , the first Bilingual corpus of Explainable Multi-turn Counseling . Our dataset includes 520 sessions of English multi-turn counseling and 431 sessions in Chinese. These multi-turn dialogues can help LLMs imitate human face-to-face counseling. Moreover, the explainable annotations in Psy-Insight can help LLMs understand the analysis and logic behind counseling. The examples from the Psy-Insight dataset are presented in Table 9 , and Table 10 . Dialogue and annotations in Psy-Insight are structured hierarchically, from broad to specific: Case - Session - Turn. Previous studies Sanh et al. (2021) showed that explicit multitask learning can improve the generation ability of LLMs. So, the Psy-Insight collected labels of counseling across a range of psychological task scenarios. The counseling data in Psy-Insight, collected from blogs and books, is annotated with multitask labels and step-by-step processes at both turn-level and session level. As shown in Figure 1 , Psy-Insight features 5 concise labels for multi-task learning ( Sanh et al., 2021 , MTL) and 6 descriptive annotations for LLMs analysis and reasoning. Concise Labels include emotion labels (e.g., Guilt), psychotherapy method labels (e.g., Cognitive Behavioral Therapy), strategies (e.g., Questions), and topics (e.g., Academic Pressure). Additionally, Psy-insight also includes descriptive annotations for analysis and reasoning, such as background, guide, summary, theme, and so on. As shown in Figure 1 , these annotations make the Psy-Insight dataset suitable for at least five psychological tasks in NLP, such as emotion/psychotherapy/strategy classification, retrieved argument generation ( Lewis et al., 2020 , RAG) , and dialogue generation. We have conducted finetuning and RAG experiments on Psy-Insight. The automatic and expert evaluations have shown that Psy-Insight can enhance the performance of mental support LLMs. Our contributions are summarized as follows: • We construct a bilingual, explainable multi-turn counseling corpus, Psy-Insight, for easily training mental support LLMs. • Psy-Insight includes step-by-step reasoning and multi-task labels at both session and turn levels. These annotations meet LLMs’ requirements for chain-of-thought and multi-task learning. We trained LLMs on the Psy-Insight dataset, and our results showed that it enhances their performance in mental support. • We invited mental health experts to compare our dataset with baseline datasets. Expert reviews show that our dataset is of high quality. The results of the expert evaluation can also be used for further research. Figure 2: Comparison of the construction processes of previous datasets and Psy-Insight dataset. As shown on the left side, previous datasets primarily focused on annotating short labels, which are suitable for the subtask in pipeline. The Psy-Insight dataset shown on the right side emphasizes the interpretability of the dialogue process, with step-by-step reasoning and session-level guide and explanation. We also collect multi-tasks labels based on each dialog sessions, offering data to enhance LLM’s generalization ability. 2 Related Work We focus on multi-turn dialogue data for mental support, which is related to research on emotional chatbots and psychological datasets. 2.1 Mental Support Chatbots Some researchers Vaidyam et al. (2019) ; Liu and Sundar (2018) ; Kretzschmar et al. (2019) ; Smith and Masthoff (2018) ; Zhou et al. (2018) first evaluated the ability of chatbots to offer mental support. Majumder et al. (2020) tried to offer emotional support by mimicking clients’ emotions. Advances in language models have led researchers such as Cheng et al. (2022) ; Peng et al. (2023) ; Tu et al. (2022) ; Peng et al. (2022) to integrate strategic components like knowledge graphs, emotion classifiers, and strategy encoders to guide chatbots in mental support response through pipeline modular. Following ChatGPT’s emergence, the capabilities of LLMs have attracted researchers to apply them to psychological tasks. Studies by Elyoseph et al. (2023) ; Li et al. (2023) show that LLMs excel in emotional chat, with finetuning in high-quality datasets. Researchers Chen et al. (2023b) ; Qiu et al. (2023) finetuned LLMs on emotional datasets. Their private LLMs outperformed traditional pipeline chatbots in mental support. 2.2 Dialogue Dataset for Mental Support Early research Chadha and Kaushik (2021) ; Yao et al. (2022a)

Read the complete text