CBT-Audio: Evaluating Audio Language Models for Patient-Side Distress Intensity Estimation in CBT Session Recordings
Qixuan Hu 1 , Shuchang Ye 1 , Xumou Zhang 1 , Anastasia Serafimovska 2 , Anastasia Suraev 2 , Amit Saha 1 , Ping-hsiu Lin 4 , Sydney Su, Usman Naseem 3 , Adam G. Dunn 5 , Jinman Kim 1 1 School of Computer Science, Faculty of Engineering, University of Sydney, Australia 2 School of Psychology, Faculty of Science, University of Sydney, Australia 3 School of Computing, Faculty of Science and Engineering, Macquarie University, Australia 4 CHeBA (Centre for Healthy Brain Ageing), School of Clinical Medicine, Discipline of Psychiatry & Mental Health, The University of New South Wales, Australia 5 Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Australia [email protected] · 2026
Read CBT-Audio: Evaluating Audio Language Models for Patient-Side Distress Intensity Estimation in CBT Session Recordings by Qixuan Hu 1 , Shuchang Ye 1 , Xumou Zhang 1 , Anastasia Serafimovska 2 , Anastasia Suraev 2 , Amit Saha 1 , Ping-hsiu Lin 4 , Sydney Su, Usman Naseem 3 , Adam G. Dunn 5 , Jinman Kim 1 1 School of Computer Science, Faculty of Engineering, University of Sydney, Australia 2 School of Psychology, Faculty of Science, University of Sydney, Australia 3 School of Computing, Faculty of Science and Engineering, Macquarie University, Australia 4 CHeBA (Centre for Healthy Brain Ageing), School of Clinical Medicine, Discipline of Psychiatry & Mental Health, The University of New South Wales, Australia 5 Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Australia [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17370.txt.
Introduction
Qixuan Hu and colleagues introduce CBT-Audio, a dataset for judging patient distress from spoken cognitive-behavioural-therapy sessions with audio language models. It holds 1,802 patient turns from 96 public recordings, with expert-validated distress labels on a subset. Tests of ten open models under audio-only, transcript-only and combined inputs show audio adds value beyond text, most clearly when words and vocal delivery diverge.
Catalog record
| CREATOR | Qixuan Hu 1 , Shuchang Ye 1 , Xumou Zhang 1 , Anastasia Serafimovska 2 , Anastasia Suraev 2 , Amit Saha 1 , Ping-hsiu Lin 4 , Sydney Su, Usman Naseem 3 , Adam G. Dunn 5 , Jinman Kim 1 1 School of Computer Science, Faculty of Engineering, University of Sydney, Australia 2 School of Psychology, Faculty of Science, University of Sydney, Australia 3 School of Computing, Faculty of Science and Engineering, Macquarie University, Australia 4 CHeBA (Centre for Healthy Brain Ageing), School of Clinical Medicine, Discipline of Psychiatry & Mental Health, The University of New South Wales, Australia 5 Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Australia [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69678 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.17370
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CBT-Audio: Evaluating Audio Language Models for Patient-Side Distress Intensity Estimation in CBT Session Recordings Qixuan Hu 1 , Shuchang Ye 1 , Xumou Zhang 1 , Anastasia Serafimovska 2 , Anastasia Suraev 2 , Amit Saha 1 , Ping-hsiu Lin 4 , Sydney Su, Usman Naseem 3 , Adam G. Dunn 5 , Jinman Kim 1 1 School of Computer Science, Faculty of Engineering, University of Sydney, Australia 2 School of Psychology, Faculty of Science, University of Sydney, Australia 3 School of Computing, Faculty of Science and Engineering, Macquarie University, Australia 4 CHeBA (Centre for Healthy Brain Ageing), School of Clinical Medicine, Discipline of Psychiatry & Mental Health, The University of New South Wales, Australia 5 Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Australia [email protected] Abstract Cognitive behavioural therapy is widely used to help patients understand and manage psychological distress. It is often delivered through spoken conversation, where therapists attend not only to what patients say, but also to how they say it, because these cues can help therapists decide how to respond and adapt treatment. Progress in building AI systems for CBT remains largely limited to text, partly because most available datasets are text based and shareable spoken CBT data are scarce under ethical and privacy constraints. This creates a blind spot because text based models and evaluations cannot capture the mismatch between the transcript and the patient’s voice, even though therapists often rely on this mismatch to understand patient distress. We introduce CBT-Audio, a dataset for evaluating patient distress estimation from spoken CBT sessions with audio language models. CBT-Audio contains 1,802 patient turns from 96 publicly available CBT recordings, with turn-level distress labels validated on an experts-annotated subset. We evaluate 10 open source audio language models under three input conditions, where models receive only patient audio, only the transcript, or both audio and transcript. Our results show that audio can provide useful information beyond text, especially when combined with transcripts. Adding audio to transcript input improves distress estimation over using the transcript alone in 8 of 10 model families, with significant gains in 4, and case studies show the clearest benefit when verbal content and vocal delivery diverge. CBT-Audio makes spoken patient behaviour measurable for AI evaluation in CBT-related tasks and supports future work on audio language models for mental health interaction. 1 Introduction Cognitive Behavioral Therapy (CBT) is among the most widely practiced and empirically supported psychotherapies. It has shown efficacy for depression, anxiety, and a range of related disorders [ 23 , 10 ] . CBT is delivered through spoken conversation between therapist and patient. During sessions, the therapist tracks how the patient’s emotional state shifts and adjusts the intervention accordingly [ 46 ] . With the global shortage of trained mental health professionals [ 4 , 38 ] and recent work exploring Large Language Models (LLMs) for CBT assistance, response generation, and evaluation [ 50 , 22 , 39 ] , AI systems are increasingly being studied as tools to support CBT practice, supervision, and quality evaluation. Recent work has started applying LLMs to CBT-related tasks. CBT-Bench [ 50 ] introduced a structured benchmark spanning CBT based knowledge classification task, and therapeutic response generation, finding that LLMs handle knowledge questions well but struggle with complex therapeutic reasoning and response generation. On the data side, several efforts have constructed synthetic therapy dialogue datasets to address the scarcity of real counseling data. Cactus [ 27 ] generated multi-turn CBT conversations using client personas and structured counseling strategies. DiaCBT [ 51 ] extended this to multi-session dialogues guided by cognitive conceptualisation diagrams. Thousand Voices of Trauma [ 6 ] produced synthetic conversations modeling emotional progression across trauma therapy stages. RealCBT [ 44 ] compared synthetic CBT sessions with CBT case studies or role play and found that synthetic dialogues, while fluent and coherent, tend to lack the emotional variability and reactivity patterns observed in therapy case studies done by real human experts, particularly on the patient side. Across all of these efforts, the input to the language models are limited to the text transcripts of therapy conversations. Treating CBT sessions as text removes an entire modality of clinical information. Therapists are trained to attend to the mismatch between what a patient says and how they say it [ 32 , 16 , 20 ] . In a live session, the same words can carry different clinical meanings depending on how they are spoken. A patient saying “I’m fine” in a flat, withdrawn voice signals something different from the same words spoken calmly and steadily. A long pause before answering a difficult question can indicate avoidance, but this signal is often lost or simplified when sessions are modeled as text transcript. Rapid speech and rising pitch may reveal distress that the words themselves do not express. Vocal cues such as tone, pace, hesitation, tremor, and shifts in speech rate convey information about patient distress beyond what words alone provide [ 11 , 16 , 37 , 26 ] . When models are evaluated only on transcripts, they cannot access these signals. Without an evaluation that includes spoken CBT session recordings, there is no way to measure whether models can use the vocal cues that therapists rely on. We introduce CBT-Audio, a dataset for testing whether audio language models (ALMs) can estimate patient distress intensity from spoken CBT session recordings. Figure 1 gives an overview of the dataset construction and evaluation pipeline. Our goal is not to build a clinical diagnostic system. Instead, we study a controlled evaluation question: when the same patient utterance is represented as audio, transcript, or both, do current open-source ALMs use the audio to improve ordinal distress estimation? This directly tests whether models benefit from access to the speech modality that transcript-only evaluations remove. CBT-Audio contains 1,802 patient-side utterances from 96 publicly available CBT educational recordings. Each utterance is labeled on a 1–5 distress intensity scale adapted from the Positive and Negative Affect Schedule (PANAS), a widely used measure for rating the intensity of positive and negative emotions, including distress [ 45 , 13 ] . PANAS has also been used in prior CBT outcome and CBT-AI evaluation work [ 27 , 51 , 17 ] . The labels are validated using ratings from five independent expert annotators. Our contributions are as follows. 1. We introduce CBT-Audio, a dataset for evaluating patient distress estimation from spoken CBT sessions with audio language models. CBT-Audio provides source URLs, clip timestamps, transcripts, turn level distress labels and evaluation configuration, enabling reproducible evaluation. 2. We show that current open source ALMs can use audio cues from CBT speech to estimate patient distress. Audio only input does not consistently outperform transcript only input, but adding audio to the transcript improves distress estimation in 8 of 10 model families, with significant gains in four. This shows that audio can carry cues indicating patient distress that complement the transcript, even when audio is not the stronger single modality. 3. We provide a systematic evaluation of 10 open source ALMs under three input conditions, where models receive only patient audio, only the transcript, or both audio and transcript. Because each model is evaluated on the same patient