Toward Human-Centered Interactive Clinical Question Answering System
Dina Albassam · 2025
Read Toward Human-Centered Interactive Clinical Question Answering System by Dina Albassam (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18928.txt.
Introduction
Dina Albassam builds a question answering system for clinical notes in a 2025 paper. Doctors can query patient records by typing or by speaking aloud. Answers are extracted from the note and highlighted in place on screen.
That highlighting lets a physician verify any answer instantly against the source. Several automatic scores judged answer quality across the test sets used. Exact wording matched only about half to three-fifths of the time. Meaning-level similarity scored far higher, above eighty-seven percent overall. Simulated clinician personas also rated the system's usability in practice. Readers see transparency treated as a feature, not an afterthought.
Catalog record
| CREATOR | Dina Albassam |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18928
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toward Human-Centered Interactive Clinical Question Answering System Dina Albassam Abstract Unstructured clinical notes contain essential patient information but are challenging for physicians to search and interpret efficiently. Although large language models (LLMs) have shown promise in question answering (QA), most existing systems lack transparency, usability, and alignment with clinical workflows. This work introduces an interactive QA system that enables physicians to query clinical notes via text or voice and receive extractive answers highlighted directly in the note for traceability. The system was built using OpenAI models with zero-shot prompting and evaluated across multiple metrics, including exact string match, word overlap, SentenceTransformer similarity, and BERTScore. Results show that while exact match scores ranged from 47% to 62%, semantic similarity scores exceeded 87%, indicating strong contextual alignment even when wording varied. To assess usability, the system was also evaluated using simulated clinical personas. Seven diverse physician and nurse personas interacted with the system across scenario-based tasks and provided structured feedback. The evaluations highlighted strengths in intuitive design and answer accessibility, alongside opportunities for enhancing explanation clarity. I Introduction Electronic health records (EHRs) have become foundational to modern healthcare, serving as digital repositories of patient information. By 2015, over 96% of non-federal acute care hospitals in the United States had adopted certified EHR systems to manage clinical data. These systems offer valuable opportunities for both individual patient care and broader data-driven insights. While structured components such as lab results and demographics are useful, the majority of clinically rich information resides in unstructured formats—namely, clinical notes written by physicians, including progress summaries, discharge instructions, and diagnostic impressions. Unstructured clinical text captures nuanced reasoning, contextual decisions, and temporal sequences that are essential to understanding patient status. This makes it an ideal target for large language models (LLMs), which excel at processing free-text inputs [ 1 ] . As a result, medical question answering (QA) systems powered by LLMs have emerged as promising tools to help clinicians extract relevant insights from complex patient records. For example, a system that can answer, “Has the patient ever taken Atropine for hypertension?”, by reviewing multiple clinical notes, could significantly reduce the cognitive burden of manual chart review and support informed clinical decisions. Despite this promise, current medical QA systems often fall short in practice. Clinicians report difficulties using these tools effectively—especially when interacting with unstructured notes—due to issues related to usability, interpretability, and trust. As Kell et al. [ 2 ] highlighted in a systematic review, only 7 out of 79 biomedical QA systems had undergone any user evaluation, and none incorporated essential features like uncertainty estimation or source attribution. Most systems remain confined to technical research prototypes, lacking integration into clinical workflows and failing to meet the practical needs of healthcare professionals. Existing QA research has explored a wide range of data types and approaches. Structured QA systems such as quEHRy [ 3 ] and DrugEHRQA [ 4 ] convert clinician queries into database-friendly formats, enabling high precision but limited flexibility. In the domain of unstructured data, Elgedawy et al. [ 5 ] and Yu et al. [ 1 ] introduced retrieval-augmented generation (RAG) methods to answer questions based on clinical notes. However, these systems offer little in terms of interactive design or physician-centered evaluation. Similarly, Jin et al. [ 3 ] used templated QA techniques for specific extraction tasks, offering limited adaptability. Outside clinical notes, systems such as MedQA [ 6 ] , AskHERMES [ 8 ] , and SemBioNLQA [ 9 ] focus on medical literature, and while they have demonstrated value, they do not address the challenge of navigating free-text clinical narratives. Patient-oriented tools like PaniniQA [ 10 ] and README [ 11 ] aim to simplify discharge summaries for lay comprehension but are not designed for physician use. Altogether, these efforts underscore a persistent gap: few QA systems provide end-to-end, interactive support for physicians querying free-text clinical notes. Furthermore, existing evaluation strategies do not adequately reflect real-world clinical utility. To address these challenges, this study explores the following research questions: • How can interactive clinical QA systems be designed to support physicians’ information-seeking behaviors, particularly in enhancing the trustworthiness and transparency of model-generated answers? • What are physicians’ perceptions of usability, interpretability, and trust when interacting with QA tools in clinical workflows? • How do simulated agent-based (AI persona) evaluations compare to traditional human user studies in assessing QA systems’ effectiveness and utility? In response to these questions, this study contributes the following: • A comparative evaluation of OpenAI models using zero-shot prompting on the emrQA-msquad dataset to select the most suitable model for clinical QA. • The design and implementation of an interactive QA system that allows physicians to ask questions via text or voice and highlights exact answer spans in clinical notes to promote transparency and trust. • An initial user evaluation using AI personas, which simulate real clinical scenarios to assess the system’s usability and effectiveness in a scalable, automated manner. This work aims to bridge the gap between technical QA advancements and real-world clinical applicability, laying the foundation for trustworthy, physician-centered QA systems for unstructured medical notes. II Related Work II-A Clinical QA Systems Medical QA systems have increasingly leveraged large language models (LLMs) to reason over both structured and unstructured electronic health record (EHR) data. Structured QA systems such as quEHRy [ 12 ] and DrugEHRQA [ 4 ] convert clinician queries into formal database queries using concept normalization and SQL translation, achieving high precision. DrugEHRQA also introduces a benchmark dataset combining structured and unstructured sources to support multimodal QA development. In contrast, unstructured QA systems use retrieval-augmented generation (RAG) techniques to process narrative clinical data. Elgedawy et al. [ 5 ] and Yu et al. [ 1 ] proposed RAG pipelines—one as a chatbot interface and another using a federated architecture (FedRAG)—to provide contextual answers from clinical notes. Jin et al. [ 3 ] implemented a templated QA approach to extract information about injection drug use, demonstrating the utility of QA for specific clinical tasks. Other systems focus on broader medical knowledge. MedQA [ 6 ] and AskHERMES [ 8 ] answer definitional queries using MEDLINE, showing improved clinician satisfaction over general-purpose tools like Google. SemBioNLQA [ 9 ] supports multiple question types (yes/no, factoid, list, summary) and performs competitively in BioASQ using semantic retrieval and UMLS-based techniques. Some QA tools are patient-centric. PaniniQA [ 10 ] enhances patient understanding of discharge summaries through quizzes, and README [ 11 ] generates simplified, contextual lay definitions. Both systems are evaluated with patients and demonstrate the impact of QA on comprehension. Despite these innovations, most systems remain technical prototypes. A review by Kell et al. [ 2 ] found only 7 of 79 biomedical QA systems underwent any user evaluation, and none supported features like uncertainty estimation or source attribution. This underscores a critical gap in usability, interpretability, and trustworthiness—leaving physician-centered QA systems underdeveloped. II-B Information-Seeking Needs of Physicians Effective QA system design must consider how clinicians search and interpret information during care. Vong and Then [ b15 ] assessed the CliniCluster system using the PICO framework and found that clinicians often ask vague or evolving questions and benefit from tools supporting query refinement and semantic exploration. AskHERMES [ 8