Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System

Fang Zeng, Zhiliang Lyu, Quanzheng Li, Xiang Li · 2024

Read Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System by Fang Zeng, Zhiliang Lyu, Quanzheng Li, Xiang Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06828.txt.

Introduction

Fang Zeng, Zhiliang Lyu, Quanzheng Li, and Xiang Li present RadCouncil in a 2024 paper on drafting the summary section of radiology reports. Doctors write findings first, then distill them into a short verdict, a slow step ripe for aid. Their frame splits the work among three language model agents with set roles.

One agent fetches like reports from a vector store. A second drafts the verdict from the findings plus those models, and a third grades the draft and sends notes back. On chest X-ray cases the council beat a lone agent on accuracy, style match, and clarity.

Catalog record

CREATORFang Zeng, Zhiliang Lyu, Quanzheng Li, Xiang Li
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06828

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System Fang Zeng, Zhiliang Lyu, Quanzheng Li, Xiang Li Abstract This study introduces "RadCouncil," a multi-agent Large Language Model (LLM) framework designed to enhance the generation of impressions in radiology reports from the finding section. RadCouncil comprises three specialized agents: 1) a "Retrieval" Agent that identifies and retrieves similar reports from a vector database, 2) a "Radiologist" Agent that generates impressions based on the finding section of the given report plus the exemplar reports retrieved by the Retrieval Agent, and 3) a "Reviewer" Agent that evaluates the generated impressions and provides feedback. The performance of RadCouncil was evaluated using both quantitative metrics (BLEU, ROUGE, BERTScore) and qualitative criteria assessed by GPT-4, using chest X-ray as a case study. Experiment results show improvements in RadCouncil over the single-agent approach across multiple dimensions, including diagnostic accuracy, stylistic concordance, and clarity. This study highlights the potential of utilizing multiple interacting LLM agents, each with a dedicated task, to enhance performance in specialized medical tasks and the development of more robust and adaptable healthcare AI solutions. 1 Introduction In radiology workflow, radiologists traditionally interpret imaging studies and manually draft detailed reports, including an "impression" section that summarizes clinically significant findings and possible diagnosis, which is a vital part of the report for referring physicians and patient care. This process is time-consuming and subject to variability impacted by the radiologist’s knowledge and experience [ 1 ] . Automated impression generation has the potential to improve report consistency, reduce radiologist workload, and enhance the overall quality of radiology reports [ 2 ] . Such a feature is especially needed with the recent growth in the demands for medical imaging, which are straining radiologists, leading to possible burnout and impacting their ability to provide timely and precise reports [ 3 ] . Large Language Models (LLMs) have shown exceptional capabilities in understanding and generating text that is coherent and contextually relevant, making them promising tools for auto-generating impressions from findings in radiology reports. A few studies have investigated LLMs’ ability for the impression summarization task [ 2 , 4 , 5 , 6 , 7 , 8 ] , demonstrating the potential of LLMs to revolutionize radiology workflow by automating the report generation process. Various techniques, such as prompt engineering, model fine-tuning, and retrieval-augmented generation (RAG) [ 6 ] , have been used to improve their performance and mitigate the limitations of LLMs, including hallucinations and insufficient domain knowledge. However, current approaches often lack the ability to integrate these techniques together within a unified framework and the scalability to incorporate new techniques and/or requirements in impression generation. Furthermore, applying LLM, even with domain adaptation techniques, for text generation without reflecting and reviewing the output of the model risks introducing hallucinations that could compromise the reliability of the results and lead to serious errors in clinical decision-making [ 9 , 10 ] . Recently, multi-agent systems have emerged as a promising approach for various natural language processing tasks [ 11 ] . By leveraging the strengths of multiple specialized agents, each focusing on specific tasks, a multi-agent system can enhance the overall quality and reliability of the model performance. In addition, techniques that are used to enhance the performance of a single LLM can be readily applied to each agent and the whole system. Thanks to the advanced natural language and code/structured data processing capability of LLMs, multi-agent systems can understand model inputs in generalizable forms, write and execute codes, and perform reasoning to orchestrate the collaborative process, effectively "wrapping" any models in a more intelligent and robust form. With efficient memory management among agents, the multi-agent system can mitigate the limitations of the context window size, thus improving the handling of extensive and complex tasks for more accurate and relevant outputs. Furthermore, through the collaboration among LLM agents, the multi-agent system can combine diverse capabilities from models with different domain expertise, potentially leading to emergent properties [ 12 ] . Some preliminary attempts have been made to test multi-agent frameworks in healthcare settings, such as clinical trial outcome prediction [ 13 ] , diagnosis from case reports [ 14 ] , medical education [ 15 ] , and medical question answering [ 16 , 17 ] . In this study, we develop a multi-agent framework named “RadCouncil” to generate the impression section in radiology reports based on the finding section, using reports from chest X-ray imaging as the study case. RadCouncil consists of three specialized agents: (1) a Retrieval agent that searches for similar reports from an external database, (2) a Radiologist agent that generates the impression section from the finding section with reference to the retrieved reports, and (3) a Review agent that evaluates the generated impression and provides feedback to the Radiologist agent with possible revisions. The main contribution of this work is twofold: RadCouncil presents an innovative multi-agent approach that mimics the workflow in radiology clinical practice, incorporating key steps of report writing, reference to documents, and review by self/peers. Secondly, the demonstrated effectiveness of RadCouncil in analyzing and generating specialized medical text shows the potential of multi-agent systems across a broad spectrum of medical applications, extending beyond radiology to other healthcare domains where complex task coordination and interplay among different roles are crucial. 2 Methodology Figure 1 : Diagram of the proposed RadCouncil framework, illustrating the interactions between the three agents, the report database, and the user. 2.1 RadCouncil: Overview and Workflow In this work, we introduce a multi-agent system for generating radiology report impressions, leveraging both the generative power of LLMs and established medical knowledge from existing reports. The system employs three specialized agents: a Report Retriever that uses vector similarity matching to find relevant historical reports, a Radiologist agent that generates impressions with reference to retrieved examples, and a Reviewer agent that ensures consistency between findings and generated impressions. This architecture not only helps maintain accuracy but also reduces hallucinations by grounding the generation process in real medical examples and cross-checking by multiple LLMs. An illustrative example of the RadCouncil’s workflow is shown in Table 1. Table 1 : Sample workflow of the RadCouncil system User PROCEDURE NAME: XR CHEST PORTABLE. FINDINGS: Lines/tubes: ET tube tip is projected at the level of the carina with tip projected near the origin of the right mainstem bronchus. Evidence of embolization coils project over the right upper hemithorax. Lungs: Patchy ill-defined opacities in the right upper lung are again noted. No gross pulmonary edema. Pleura: There is no pleural effusion or pneumothorax. Heart and mediastinum: The heart and the mediastinum are unchanged. Bones: The thoracic skeleton is unchanged. Retriever Here are some examples: EXAMPLE 1: PROCEDURE NAME: XR CHEST PORTABLE. FINDINGS: Lines/tubes: Endotracheal tube terminates below the carina in the proximal right mainstem bronchus. Lungs: Redemonstration of patchy ill-defined opacities in the right mid to upper lung. Focus of patchy opacity in the left base is unchanged. No pulmonary edema. Pleura: There is no pleural effusion or pneumothorax. Heart and mediastinum: The heart and the mediastinum are unchanged. Bones: The thoracic skeleton is unchanged. IMPRESSION: Endotracheal tube terminates in the proximal right mainstem bronchus. Unchanged patchy opacities in the right mid to upper

Read the complete text