MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow

Ziyue Wang, Junde Wu, Linghan Cai, Chang Han Low, Xihong Yang, Qiaxuan Li, Yueming Jin · 2025

Read MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow by Ziyue Wang, Junde Wu, Linghan Cai, Chang Han Low, Xihong Yang, Qiaxuan Li, Yueming Jin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18968.txt.

Introduction

Ziyue Wang, Yueming Jin and their co-authors present MedAgent-Pro, an agent workflow for evidence-based medical diagnosis from text and images in 2025. Vision-language models often jump straight to answers drawn from experience, without the stepwise, measured reasoning clinics require.

Their workflow splits diagnosis into a disease-level plan and patient-level steps. A retrieval agent pulls medical guidelines to shape the plan, and specialist tools supply quantitative checks for each patient. Each step is verified before the final call, to enforce sound logic. Tests across body regions, scan types and diseases beat mainstream and expert models.

Catalog record

CREATORZiyue Wang, Junde Wu, Linghan Cai, Chang Han Low, Xihong Yang, Qiaxuan Li, Yueming Jin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-18968

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow Ziyue Wang, Junde Wu, Linghan Cai, Chang Han Low, Xihong Yang, Qiaxuan Li, Yueming Jin Abstract In modern medicine, clinical diagnosis relies on the comprehensive analysis of primarily textual and visual data, drawing on medical expertise to ensure systematic and rigorous reasoning. Recent advances in large Vision–Language Models (VLMs) and agent-based methods hold great potential for medical diagnosis, thanks to the ability to effectively integrate multi-modal patient data. However, they often provide direct answers and draw empirical-driven conclusions without quantitative analysis, which reduces their reliability and clinical usability. We propose MedAgent-Pro, a new agentic reasoning paradigm that follows the diagnosis principle in modern medicine, to decouple the process into sequential components for step-by-step, evidence-based reasoning. Our MedAgent-Pro workflow presents a hierarchical diagnostic structure to mirror this principle, consisting of disease-level standardized plan generation and patient-level personalized step-by-step reasoning. To support disease-level planning, an RAG-based agent is designed to retrieve medical guidelines to ensure alignment with clinical standards. For patient-level reasoning, we propose to integrate professional tools such as visual models to enable quantitative assessments. Meanwhile, we propose to verify the reliability of each step to achieve evidence-based diagnosis, enforcing rigorous logical reasoning and a well-founded conclusion. Extensive experiments across a wide range of anatomical regions, imaging modalities, and diseases demonstrate the superiority of MedAgent-Pro to mainstream VLMs, agentic systems and state-of-the-art expert models. Ablation studies and human evaluation by clinical experts further validate its robustness and clinical relevance. Code is available at https://github.com/jinlab-imvr/MedAgent-Pro. Figure 1: Comparison of diagnostic outcomes for two diseases across mainstream VLMs, medical agentic systems, and our proposed MedAgent-Pro workflow. 1 Introduction Clinical diagnosis, a core task in medical practice, entails synthesizing various clinical information to reach a conclusion [ 1 , 2 , 3 ] , where clinicians make decisions mainly based on visual cues from medical imaging and textual information from patient records. For example, it is common for clinicians to examine radiology images to identify tumor scales, or analyze pathological images to detect potential cancer. Early AI-assisted methods focus on adopting pure image analysis models to support diagnosis, such as tumor segmentation in oncology and pneumonia severity assessment in chest radiology. Recently, Vision-Language Models (VLMs) have demonstrated that integrating multi-modal information can significantly benefit diagnosis, and medical visual question answering (VQA) is a key task in this context, where models answer textual questions based on medical images [ 4 , 5 , 6 , 7 ] . However, ordinary VQA formulated in the existing methods still fails to reflect the real-world diagnostic processes. Rather than performing such one-hop QA, clinical diagnosis involves a standardized, step-by-step process [ 1 , 8 , 9 , 10 , 11 ] . The diagnostic process for each patient typically involves two stages: i) Determining the target disease and formulating a standardized workflow based on medical guidelines with multiple clinical indicators to support the decision; ii) Step-by-step analysis of personalized data, combining qualitative and quantitative assessments to evaluate these indicators. Throughout the process, each analytical step builds on previous assumptions and is supported by relevant literature or analysis provided by specialized tools For example, in glaucoma diagnosis, the cup-to-disc ratio is a key indicator, which relies on the accurate localization of the optic cup and disc regions in the preceding step. In contrast, conventional VQA models often generate diagnostic conclusions hastily, relying on empirical internal knowledge without fine-grained analysis. Due to the strict requirement of medical diagnosis, current methods fall short of meeting the clinical standard: VLMs [ 12 , 13 , 14 , 4 , 15 ] have shown strong performance across a range of tasks [ 16 , 17 , 18 ] , however, they lack sufficient medical knowledge and remain inadequate for in-depth medical analysis. While models like GPT-o1 [ 12 ] and DeepSeek-R1 [ 14 ] incorporate reasoning capabilities to support more structured medical analysis, their limited fine-grained visual perception ability impairs quantitative analysis and reduces their effectiveness in clinical applications. Meanwhile, agentic systems [ 19 , 20 , 21 , 22 , 23 ] have extended the capabilities of VLMs by introducing more professional models [ 24 , 25 , 26 , 27 , 28 ] . However, current medical agentic systems [ 29 , 30 , 31 , 32 , 33 ] simply glued all tools together instead of a clinically oriented workflow, functioning more as an integrated toolbox than an end-to-end automated solution. As a result, when asked to provide a diagnosis, these systems simply invoke the internal VLM without selecting appropriate tools to support their decision-making. In summary, existing methods treat medical diagnosis as an empirical one-hop question-answering task, relying solely on VLMs’ internal knowledge to make qualitative judgments. However, modern medical practice emphasizes evidence-based diagnosis, which requires structured reasoning and clinical evidence [ 8 , 9 , 10 , 11 ] . To tackle these issues, we propose MedAgent-Pro, a reasoning agentic workflow tackling versatile multi-modal medical diagnosis tasks. We aim to design a workflow that aligns with modern medical criteria, provides decision support with medical guidelines and quantitative analysis as shown in Fig. 1 . Our MedAgent-Pro embraces a hierarchical structure to simulate the modern clinical procedure, conduct step-by-step reasoning at the disease and patient levels. Disease-level planning generates standardized diagnostic plans, while Patient-level reasoning follows these plans to analyze personalized information. To ensure that diagnostic plans align with clinical guidelines, MedAgent-Pro incorporates an RAG agent that retrieves relevant medical knowledge. In line with modern clinical workflows, which routinely employ specialized tools for diagnostic support, MedAgent-Pro integrates expert tools such as visual models to enable accurate quantitative evaluation of clinical indicators. Furthermore, to maintain the rigor of multi-step clinical reasoning, we propose an evidence-based analysis in which the system evaluates the reliability of each step’s output before proceeding to the next step, ensuring that every diagnostic inference is grounded in a sound and trustworthy foundation. Our key contributions are summarized as: • We propose MedAgent-Pro, the first agentic paradigm that presents systematic, evidence-based reasoning for accurate and reliable medical diagnosis. By aligning with the principle of modern medical workflow, our paradigm transforms the empirical, ready-made outputs of prior methods into more rigorous logical reasoning and a well-founded conclusion. • MedAgent-Pro presents a hierarchical structure consisting of disease-level and patient-level reasoning. A RAG-based method ensures that disease-level planning aligns with medical guidelines, while quantitative analysis and evidence-based reasoning are devised at the patient level to ensure professionalism and reliability of the step-by-step analysis. • We validate MedAgent-Pro comprehensively across 10+ imaging modalities, 20+ anatomies, and 50+ diseases, surpassing mainstream VLMs, medical agentic systems, and even task-specific models. Notably, it outperforms GPT-4o by 34% and 22% on glaucoma and heart disease diagnosis. Clinician evaluations further highlight diagnostic quality and reliability of our MedAgent-Pro. 2 Related work Multi-modal Medical Diagnosis Developing AI techniques for multi-modal medical diagnosis has become a primary research objective. [ 34 , 35 , 36 , 37 ] . Prior research focused on medical imaging assessment, including classification [ 24 , 38 , 39 , 40 , 41 ] , detection [ 42 , 43 , 44 , 45 , 46 ] , and segmentation [ 47 ,

Read the complete text