”Where does it hurt?” - Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialogues

Tom Röhr, Soumyadeep Roy, Fares Al Mohamad, Jens-Michalis Papaioannou, Wolfgang Nejdl, Felix Gers, Alexander Löser · 2025

Read ”Where does it hurt?” - Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialogues by Tom Röhr, Soumyadeep Roy, Fares Al Mohamad, Jens-Michalis Papaioannou, Wolfgang Nejdl, Felix Gers, Alexander Löser (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19077.txt.

Introduction

Tom Röhr and six co-authors map the path of doctors' intentions through medical conversations, in this 2025 paper. A good physician steers the dialogue with targeted questions, moving from symptoms toward a plan. No one had charted those intention shifts before. The team built a fine grained taxonomy of doctor intents, based on the standard SOAP note structure, with help from clinicians.

Over five thousand turns from a public dialogue benchmark were labeled by medical experts recruited online. Models, both generative and encoder based, learned the broad structure well but stumbled at the seams between categories. The data also revealed common trajectories, recurring routes conversations take. Filtering dialogue by intent then boosted automatic summarization sharply. The data set and guidelines are public for builders of diagnosis support tools.

Catalog record

CREATORTom Röhr, Soumyadeep Roy, Fares Al Mohamad, Jens-Michalis Papaioannou, Wolfgang Nejdl, Felix Gers, Alexander Löser
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19077

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

”Where does it hurt?” - Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialogues Tom Röhr, Soumyadeep Roy, Fares Al Mohamad, Jens-Michalis Papaioannou, Wolfgang Nejdl, Felix Gers, Alexander Löser Abstract In a doctor-patient dialogue, the primary objective of physicians is to diagnose patients and propose a treatment plan. Medical doctors guide these conversations through targeted questioning to efficiently gather the information required to provide the best possible outcomes for patients. To the best of our knowledge, this is the first work that studies physician intent trajectories in doctor-patient dialogues. We use the ‘Ambient Clinical Intelligence Benchmark’ (Aci-bench) dataset for our study. We collaborate with medical professionals to develop a fine-grained taxonomy of physician intents based on the SOAP framework ( S ubjective, O bjective, A ssessment, and P lan). We then conduct a large-scale annotation effort to label over 5000 doctor-patient turns with the help of a large number of medical experts recruited using Prolific, a popular crowd-sourcing platform. This large labeled dataset is an important resource contribution that we use for benchmarking the state-of-the-art generative and encoder models for medical intent classification tasks. Our findings show that our models understand the general structure of medical dialogues with high accuracy, but often fail to identify transitions between SOAP categories. We also report for the first time common trajectories in medical dialogue structures that provide valuable insights for designing ‘differential diagnosis’ systems. Finally, we extensively study the impact of intent filtering for medical dialogue summarization and observe a significant boost in performance. We make the codes and data, including annotation guidelines, publicly available at https://github.com/DATEXIS/medical-intent-classification . † † paperid: 8547 1 Introduction Doctor-patient dialogues are complex interactions where physicians must efficiently gather information, reason through differential diagnoses, and formulate treatment plans. While NLP research has made significant strides in tasks like medical entity recognition ( Wu et al. 2017 ) , summarization ( Michalopoulos et al. 2022 ) , and dialogue act classification ( Blache et al. 2020 ) , most of the work in differential diagnosis modeling focuses primarily on retrospective clinical notes ( Fast et al. 2024 ; Figueroa et al. 2024 ) . However, these notes often present a flattened and post hoc representation of patient information, neglecting the dynamic trajectories during real-time clinical conversations. These dynamic trajectories are non-linear and an evolving process of clinical reasoning during patient encounters. Clinicians often revise their assessments and decisions as new information emerges throughout the consultation. This dynamic process involves continuous interpretation and re-interpretation of patient data, which is challenging to capture in static notes. In contrast to static notes, dialogues capture the evolving intents of physicians as they navigate the complexities of a patient encounter. Works such as AMIE ( Tu et al. 2025 ) demonstrate that state-of-the-art language models can effectively simulate clinical interviews by synthesizing patient interactions. Nonetheless, how physicians transition between these steps remains largely unexplored. To the best of our knowledge, we present the first comprehensive study of physician intent trajectories within medical dialogues using Aci-bench ( Yim et al. 2023 ) , one of the richest datasets of doctor-patient interactions. In close collaboration with medical professionals, we introduce a fine-grained taxonomy of physician intents as shown in Figure 1 , based on the established SOAP framework ( Weed 1971 ) as our first research contribution . This fine-grained taxonomy includes multiple intents per SOAP category, thus providing a highly detailed representation of how clinicians navigate patient engagements. Figure 1: Proposed fine-grained physician intent taxonomy in relation to the SOAP framework, developed in consultation with medical experts. There are 8 subjective , 3 objective , 2 assessment , and 6 plan intents. We include an additional category called others , which inherits the Chitchat intent. As our second research contribution , we annotate the Aci-bench dataset with the proposed intent taxonomy and make it publicly available. Through a large-scale crowd-sourcing effort with around 90 medical experts recruited through the Prolific platform from across the globe, we annotate more than 5,000 dialogue turns, creating a unique resource for analyzing physician trajectories in clinical conversations. The general structure of trajectories during a differential diagnosis ( Pearn 2022 ) is shown in Figure 2 . Figure 2: Physician intent trajectory during a clinical conversation. After multiple turns of subjective symptom-taking, the doctor transitions to objective examinations to conclude a clinical assessment. Finally, the conversation concludes with multiple turns for treatment planning. Please note that physicians may only use subgraphs of this general structure, depending also on the patient’s comorbidity, clinical history, and other factors. We strongly believe this annotated dataset will facilitate and encourage more research in this critical, under-explored research area. To gain a deeper understanding of intent trajectories in doctor-patient dialogues and their potential impact on the current state-of-the-art models, we perform an extensive benchmarking and characterization study, which forms our third research contribution . We evaluate generative and encoder-based models on the task of medical intent classification and next intent prediction. Our analysis uncovers key challenges in capturing intent transitions across SOAP categories. Additionally, we identify common physician intent trajectories in doctor-patient dialogues. These trajectories offer valuable insights for the design of dialogue systems to support differential diagnosis. As our final research contribution , we investigate the potential impact on current SOTA models that do not explicitly consider our proposed fine-grained physician intent taxonomy, over a critical, downstream task of dialogue-to-medical-note summarization. We observe that filtering dialogues for physician intents improves summarization quality. We release our dataset 2 2 2 https://huggingface.co/DATEXIS , annotation guidelines, and code 3 3 3 https://github.com/DATEXIS/medical-intent-classification to the community to support further research at the intersection of clinical NLP and dialogue-driven clinical decision. 2 Related Work Recent work released medical dialogue corpora to accelerate the development of medical dialogue systems (MDS). We divide these works into two distinct groups. Non-annotated medical dialogues. These datasets do not contain specific dialogue annotations and have either a small number of examples ( Papadopoulos Korfiatis et al. 2022 ) , only include short dialogues ( Ben Abacha et al. 2023 ) , or are not freely accessible ( Krishna et al. 2021 ; Zhang et al. 2021 ; Finley et al. 2018 ; Enarvi et al. 2020 ) . Larger datasets like ( Zeng et al. 2020 ) are non-English and lack real-world conversations. Annotated medical dialogues. Works such as ReMeDi ( Yan et al. 2021 ) , MIE ( Zhang et al. 2020 ) , Code-Mixed ( Dowlagar and Mamidi 2023 ) , IMCS-21 ( Chen et al. 2022 ) , and MediTOD ( Saley et al. 2024 ) curate resources that align with annotations to solve MDS tasks. Such tasks include annotations for medical entity recognition, natural language generation, or dialogue act classification. Our work focuses on a more detailed annotation of physician intents in dialogues guided by the SOAP taxonomy. We use SOAP because it is a widely adopted standard for documenting clinical notes. Therefore, our approach bridges the gap between the representation of patient interactions in dialogues and their documentation in medical notes. Modeling differential diagnosis in medical dialogue systems. Several studies, including AMIE ( Tu et al. 2025 ) , MEDDxAgent ( Rose et al. 2025 ) ,

Read the complete text