Human-Centered AI in Multidisciplinary Medical Discussions: Evaluating the Feasibility of a Chat-Based Approach to Case Assessment.
Shinnosuke Sawano, Satoshi Kodera · 2025
Read Human-Centered AI in Multidisciplinary Medical Discussions: Evaluating the Feasibility of a Chat-Based Approach to Case Assessment. by Shinnosuke Sawano, Satoshi Kodera (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16464.txt.
Introduction
Shinnosuke Sawano and Satoshi Kodera tested a chat based platform in 2025 where doctors from several fields judge hard cases together with AI help. Their focus was heart patients who live with several long term illnesses at once.
The team built five test cases from past reports and the doctors' own experience, then studied the group chats. AI summaries cut the time needed to sum up a case by about eighty percent on average. Made up facts in the summaries were rare, and harmful ones rarer still. Group judgments also mapped medical knowledge in richer detail than a single doctor's view. The authors say the approach looks workable for real case meetings.
Catalog record
| CREATOR | Shinnosuke Sawano, Satoshi Kodera |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-16464
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Human-Centered AI in Multidisciplinary Medical Discussions: Evaluating the Feasibility of a Chat-Based Approach to Case Assessment. Shinnosuke Sawano, Satoshi Kodera Abstract In this study, we investigate the feasibility of using a human-centered artificial intelligence (AI) chat platform where medical specialists collaboratively assess complex cases. As the target population for this platform, we focus on patients with cardiovascular diseases who are in a state of multimorbidity, that is, suffering from multiple chronic conditions. We evaluate simulated cases with multiple diseases using a chat application by collaborating with physicians to assess feasibility, efficiency gains through AI utilization, and the quantification of discussion content. We constructed simulated cases based on past case reports, medical errors reports and complex cases of cardiovascular diseases experienced by the physicians. The analysis of discussions across five simulated cases demonstrated a significant reduction in the time required for summarization using AI, with an average reduction of 79.98%. Additionally, we examined hallucination rates in AI-generated summaries used in multidisciplinary medical discussions. The overall hallucination rate ranged from 1.01% to 5.73%, with an average of 3.62%, whereas the harmful hallucination rate varied from 0.00% to 2.09%, with an average of 0.49%. Furthermore, morphological analysis demonstrated that multidisciplinary assessments enabled a more complex and detailed representation of medical knowledge compared with single physician assessments. We examined structural differences between multidisciplinary and single physician assessments using centrality metrics derived from the knowledge graph. In this study, we demonstrated that AI-assisted summarization significantly reduced the time required for medical discussions while maintaining structured knowledge representation. These findings can support the feasibility of AI-assisted chat-based discussions as a human-centered approach to multidisciplinary medical decision-making. 1 Introduction Human-centered medical artificial intelligence (AI) research that explores AI integration in medical and envisions new clinical workflows is critically needed to alleviate the burden on medical professionals and enable seamless coexistence between humans and AI in medical practice (1-3). In recent years, large language models (LLMs) such as Chat Generative Pretrained Transformer (ChatGPT) have emerged as promising solutions and demonstrated their potential to enhance access to medical information for both medical professionals and patients by facilitating efficient knowledge retrieval, patient education, and clinical decision support (4-6). In studies evaluating the performance of ChatGPT on the United States Medical Licensing Examination (USMLE), researchers demonstrated that ChatGPT performed as well as medical students (7-9). However, the implementation of AI in medical practice has not progressed significantly. This is because medical environments are optimized individually, which makes AI integration potentially disruptive to existing workflows. Additionally, challenges persist, such as hallucination and high output variability. As a result, there are significant challenges in using generative AI such as ChatGPT independently for medical assessments (10). To address these challenges, in this study, we investigate the feasibility of using a human-centered chat platform where medical specialists assess complex cases collaboratively by leveraging AI to enhance efficiency and facilitate structured discussions. As the target population for this platform, we focus on patients with cardiovascular diseases who are in a state of multimorbidity, that is, suffering from multiple chronic conditions (11, 12). In previous studies, researchers reported that 64.9% of individuals aged 65–84 and 81.5% of those aged 85 and older experience multimorbidity (13). This poses significant challenges in medical care and service provision because it affects not only individual diseases but also the interactions between them, which effects patients’ health, quality of life, and medical resource utilization (14). Patients with multimorbidity often receive treatment from multiple medical specialists, which leads to a lack of care coordination and increasing concerns about adverse events caused by overlapping or interacting pharmacotherapies (15). Given this background, there is a growing need to establish a new healthcare system that provides comprehensive assessments for patients with multimorbidity. In this study, we evaluate simulated cases with multiple diseases using a chat application with physicians and assess its feasibility, the efficiency gains achieved through AI utilization, and the quantification of discussion content. 2 Method 2.1 Study design We reviewed chat discussions among physicians in simulated cases. We constructed the simulated cases based on past case reports, incident reports of medical errors, and complex cases of cardiovascular diseases experienced by physicians who did not participate in the discussions. Two physicians reviewed and double-checked these cases to ensure medical validity. We selected the physicians included in the discussions based on their expertise and clinical experience in providing comprehensive evaluations for patients with multimorbidity, including cardiovascular diseases. The eligibility criteria included being board-certified specialists in cardiology, nephrology, diabetology, or general medicine, or primary care physicians, and a minimum of five years of clinical experience post-medical licensure. Additionally, the physicians were required to fully understand the objectives of this study and the significance of the AI-assisted chat platform, and provide informed consent for participation. 2.2 Workflow of Generative AI-Assisted Case Discussion Discussions were conducted on the generative AI chat platform based on the medical information of the simulated cases. We used ChatGPT-4o as the generative AI model (16). We assigned specialists, cardiologists, nephrologists, diabetologists, and primary care physicians to each case. We informed participating physicians in advance that the cases were fictional. The assigned specialists recorded the physicians medical evaluations on the platform based on the medical information provided for the simulated cases. Once all participating physicians completed their evaluations, we summarized the content using generative AI with custom prompts. Physicians at our institution reviewed and revised the validity of the AI-generated summaries as necessary while simultaneously verifying for hallucinations. 2.3 Evaluation of the Impact of Generative AI To evaluate efficiency, we measured the time required to generate the final reports, focusing on the impact of AI-generated summaries on the overall report creation process. To maintain consistency in processing, we used a standardized format for the reports. Specifically, we compared the time required from the AI-generated summary to the finalized report with the time required to create the report without AI assistance. We calculated the time based on the final report’s word count, assuming a writing speed of 60 characters per minute. 2.4 Review of Hallucinations in AI-Generated Text by Physicians To evaluate hallucinations in AI-generated summaries, we conducted morphological analysis on pre-finalized text reviewed by physicians (17, 18). In other words, the raw output generated by AI was directly reviewed by physicians. For morphological analysis, we used MeCab with the Neologd dictionary (https://github.com/neologd/mecab-ipadic-neologd/blob/master/README.ja.md), which is widely used for Japanese text processing (17). The AI-generated summaries were analyzed morphologically, and the physicians identified and counted all instances of expressions that failed to convey the intended meaning accurately, clearly erroneous descriptions, and other hallucinations through manual verification. Furthermore, among all hallucinations, we classified those considered medically unacceptable as "harmful hallucinations." We further categorized harmful hallucinations based on whether they occurred at the word level or sentence level. For quantification, we calculated the error rate by determining the proportion of all hallucinations or harmful hallucinations relative to the total number of morphological tokens extracted per case. 2.5 Knowledge Graph-Based Comparison of Multidisciplinary Teams and Single Physician Assessments For quantitative evaluation in English, we systematically organized the medical knowledge generated from the discussions into a structured knowledge graph, and integrated both AI-generated summaries based on multidisciplinary discussions reviewed by physicians and summaries independently created by a single physician. This approach