When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews
2026
Read When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24651.txt.
Introduction
Watawana and co-authors expose a bias in automatic depression detection from interviews. Models trained on full doctor-patient transcripts can exploit the interviewer's fixed prompts, rather than the patient's words. Across three datasets, ANDROIDS, DAIC-WOZ, and E-DAIC, this shortcut yields high scores without using participant language. Restricting models to participant utterances spreads the evidence and reflects genuine linguistic cues. The bias holds across datasets and architectures. The authors urge analyses that locate evidence by time and speaker.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22759 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.24651
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews Abstract Automatic depression detection from doctor–patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interpretability remains limited: strong performance is often reported without revealing what drives predictions. We analyze three datasets—ANDROIDS, DAIC-WOZ, and E-DAIC—and identify a systematic bias from interviewer prompts in semi-structured interviews. Models trained on interviewer turns exploit fixed prompts and positions to distinguish depressed from control subjects, often achieving high classification scores without using participant language. Restricting models to participant utterances distributes decision evidence more broadly and reflects genuine linguistic cues. While semi-structured protocols ensure consistency, including interviewer prompts inflates performance by leveraging script artifacts. Our results highlight a cross-dataset, architecture-agnostic bias and emphasize the need for analyses that localize decision evidence by time and speaker to ensure models learn from participants’ language. Keywords: Depression Corpora, Clinical Interviews, Graph Convolutional Network (GCN) When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews Hasindri Watawana 1,2 , Sergio Burdisso 1 , Diego A. Moreno-Galván 3 Fernando Sánchez-Vega 3 , A. Pastor López-Monroy 3 , Petr Motlicek 1,4 , Esaú Villatoro-Tello 1 1 Idiap Research Institute, Switzerland {hwatawana,sburdisso,pmotlicek,evillatoro}@idiap.ch 2 EPFL, Switzerland 3 Centro de Investigación en Matemáticas (CIMAT), Mexico {diego.moreno,fernando.sanchez,pastor.lopez}@cimat.mx 4 Brno University of Technology, Czech Republic Abstract content 1. Introduction Language has long been established as a powerful indicator of personality, socio-emotional state, and mental health 15 ; 18 . This insight has spurred a rich body of work at the intersection of AI, natural language processing, and clinical psychology, showing that structured interviews and written responses can reveal important aspects of cognitive and behavioral functioning, particularly for automatic depression detection 13 ; 21 ; 20 . While a growing body of recent work trains automatic depression detection models on both participant responses and interviewer prompts 22 ; 1 ; 14 ; 17 , it remains unclear how much each contributes to model performance. This study asks: to what extent can a model classify a participant as depressed using only the interviewer’s questions? Surprisingly, across three datasets (ANDROIDS, DAIC-WOZ, E-DAIC) and two model families, interviewer-only ( I ) models often match or outperform participant-only ( P ) models. This does not imply that interviewer questions are clinically more informative; instead, qualitative analyses show that I -models exploit systematic shortcuts tied to the interview script, focusing on recurring prompts and question positions—a phenomenon we term prompt-induced bias . Our main contributions are threefold: (1) We quantify prompt-induced bias across multiple datasets, showing that I -models can outperform P -models. (2) We show that this effect is model-agnostic by reproducing it with both graph convolutional networks and transformers. (3) We provide qualitative analyses that reveal how I -models concentrate on narrow interview segments while P -models distribute evidence more broadly. These findings highlight the importance of carefully handling interviewer prompts and verifying that models truly leverage participant language rather than spurious cues. 2. Datasets Within clinical practice, the initial assessment of mental illness is a semi-structured interview in which clinicians pose a standardized yet open-ended sequence of questions to elicit symptoms, history, and functioning. This protocol balances consistency with flexibility, enabling reliable assessment while allowing patients to elaborate in their own words. The depression corpora we study are designed to simulate such standardized screening protocols for identifying people at risk: an interviewer (human or virtual) delivers a controlled set of prompts to ensure replicability and coverage of diagnostic cues, while participants respond freely, producing language that can be analyzed for clinical indicators. 2.1. DAIC-WOZ The Distress Analysis Interview Corpus – Wizard of Oz (DAIC-WOZ) dataset 10 contains semi-structured clinical interviews in North American English conducted by “Ellie”, an animated virtual interviewer controlled by a human in another room (“Wizard-of-Oz” setup where a user interacts with a mock interface controlled, to some degree, by a person). DAIC-WOZ is multimodal—audio, video, and manual transcripts—and includes PHQ-8 depression ratings for each participant 12 . Ellie’s design emphasizes replicability and consistency: she draws from a finite repertoire of 191 prompts spanning general questions (e.g., lifestyle and sleep), neutral backchannels, positive/negative empathic responses, surprise tokens, continuation prompts (e.g., “could you tell me more?”), and miscellaneous control items. This controlled prompting is intended to elicit behaviors associated with depression and related symptoms while reducing variability attributable to the interviewer. DAIC-WOZ comprises 189 subjects split into 107 train, 35 development, and 47 test interviews. 2.2. E-DAIC E-DAIC 3 is an extension of DAIC-WOZ that preserves the same interview format while scaling the collection. The key difference is that the virtual interviewer used in E-DAIC is fully automatic in contrast to the human-controlled Ellie in DAIC-WOZ. While DAIC-WOZ distributes complete two-speaker transcripts (interviewer and participant), E-DAIC provides only the participant side. Therefore, for our experiments, we prepare automatic transcripts for E-DAIC using WhisperX 10 pipeline. The process is explained in Section 3 . We corrected several mislabeled subjects after verifying inconsistencies between PHQ scores and the provided binary labels, an issue noted previously by others 2 . The official split sizes are 163 train, 56 development, and 56 test interviews. 2.3. ANDROIDS ANDROIDS 7 is an Italian speech corpus for depression detection collected “in the wild” using laptop microphones. Each participant was recorded in two tasks: a Reading Task (RT)—everyone reads the same short, simple story to reduce literacy/education effects—and an Interview Task (IT) with spontaneous speech, where the interviewer is instructed to ask only minimal prompts, yielding semi-structured but low-intervention interactions. In this work we focus only on the data from the IT task. It comprises 116 native-Italian participants (64 depressed, 52 controls), with controls matched to the depressed cohort by gender, age, and education to minimize demographic confounds. Interviews are manually segmented into turns; diagnostic labels (“depressed” vs. “control”) come from clinicians following DSM-5 criteria. 1 1 1 Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition 3 . The release includes audio and acoustic features, but no ground-truth transcripts. Therefore, we generated WhisperX transcripts for ANDROIDS (see Section 3 ). 3. Methodology Model Source ANDROIDS DAIC-WOZ E-DAIC † P I Avg D C Avg D C Avg D C 8 ✓ – – – 0.84 0.80 0.89 0.80 0.67 0.94 11 ✓ ✓ 0.93 – – – – – – – – 6 ✓ ✓ 0.92 0.93 0.9 – – – – – – 14 ✓ ✓ – – – 0.81 – – – – – P -Longformer ✓ 0.79 0.82 0.76 0.71 0.61 0.81 0.67 0.56 0.79 I -Longformer ✓ 0.98 0.98 0.98 0.73 0.64 0.83 0.65 0.50 0.80 P -GCN ✓ 0.93 0.95 0.92 0.85 0.81 0.88 0.70 0.54 0.86 I -GCN ✓ 0.97 0.97 0.98 0.88 0.85 0.91 0.74 0.57 0.90 Table 1: Development-set F 1 scores on ANDROIDS, DAIC-WOZ, and E-DAIC. For each dataset, we report macro-average ( Avg ), Depressed ( D ), and Control ( C ) F 1 . ANDROIDS scores are reported as 5-fold averages; DAIC-WOZ and E-DAIC use the official development split. Sources: participant ( P ) and interviewer ( I ) text. Upper block: text-only baselines from prior work. Bold = best in group; underlined = best overall text-only result. \boldsymbol{{\dagger}} : E-DAIC prior-work results use