Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learning
Yichun Feng, Jiawei Wang, Lu Zhou, Yikai Zheng, Zhen Lei, Yixue Li · 2025
Read Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learning by Yichun Feng, Jiawei Wang, Lu Zhou, Yikai Zheng, Zhen Lei, Yixue Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19630.txt.
Introduction
Yichun Feng and co-authors train a doctor agent that asks its way to a diagnosis in this 2025 paper. Real consultations unfold over turns, but most medical chatbots demand everything at once. Their framework treats each visit as decisions under uncertainty. Several agents cooperate, and reinforcement learning rewards good questioning, not just right answers.
The team built a multi-turn English dataset of medical dialogues for training. Their doctor runs on a seven-billion-parameter open model. In blinded human reviews and trials with real patients it beat frontier models. It matched the exact diagnosis in about seven of ten cases. The authors pitch it as screening help that frees doctors for harder cases.
Catalog record
| CREATOR | Yichun Feng, Jiawei Wang, Lu Zhou, Yikai Zheng, Zhen Lei, Yixue Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19630
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learning Yichun Feng, Jiawei Wang, Lu Zhou, Yikai Zheng, Zhen Lei, Yixue Li Abstract Large language models (LLMs) struggle in real-world clinical consultations. Single-turn consultation systems require patients to describe all symptoms at once, which often leads to unclear complaints and vague diagnoses. Traditional dialogue models, constrained by static supervised learning, are limited to superficially imitating existing dialogue patterns and lack the ability to actively construct understanding in dynamic interactions, thus failing to achieve genuine clinical reasoning. To address these challenges, we propose DoctorAgent-RL, a reinforcement learning (RL)-based multi-agent collaborative framework, and train a doctor agent on Qwen2.5-7B-Instruct using this framework. Within this framework, a medical consultation is modeled as a dynamic decision-making process under uncertainty. The core intelligence of the doctor agent is shifted from knowing the answer to learning and mastering a questioning methodology aimed at achieving an optimal diagnosis. Through strategic questioning, it guides the progressive emergence of key patient information in multi-turn dialogues. To support this high-fidelity simulation of the real diagnostic process, we constructed MTMedDialog, a novel English multi-turn medical consultation dataset designed for dynamic, interactive training. To validate its real-world effectiveness, rigorous evaluations including blinded human assessments and trials with real patients were conducted. DoctorAgent-RL outperformed frontier models and achieved a 70% exact diagnostic match rate, confirming its potential as a collaborative tool. By handling initial screenings, it can free clinicians to focus on complex cases, thereby addressing critical issues like physician shortages and misdiagnosis risks while alleviating the strain on healthcare resources. keywords Large Language Models, Medical Multi-Turn Consultation, Reinforcement Learning, Multi-Agent System † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Introduction Large language models (LLMs) like ChatGPT Achiam et al. (2023) , LLaMA Touvron et al. (2023) , and ChatGLM GLM et al. (2024) show significant potential for assisting in clinical decision-making Kopka et al. (2025) . This capability is urgently needed in the context of a global doctor shortage and high-pressure healthcare environments Agyeman-Manu et al. (2023) . In these settings, time pressure on clinicians often leads to rapid diagnoses based on incomplete information, increasing misdiagnosis risks ALQahtani et al. (2016) , while routine consultations consume resources that could be allocated to more critical cases Arndt et al. (2024) . Therefore, developing an intelligent system to improve primary care efficiency holds significant practical importance for optimizing resource allocation and alleviating healthcare strain Al Kuwaiti et al. (2023) . The limitations of current systems are evident on multiple levels. First, single-turn question-answering systems, such as MedAlpaca Han et al. (2023) and BioMistral Labrak et al. (2024) , require patients to provide a comprehensive description of their condition all at once Wu et al. (2025) . This contradicts the clinical reality of complex situations where chief complaints are unclear and information is vague Laban et al. (2025) , often leading to overly broad or even risky diagnostic suggestions Auerbach et al. (2024) . Second, even traditional models that support multi-turn dialogue, like Bianque Chen et al. (2023) and DialoGPT Zhang et al. (2019) , rely heavily on static supervised learning, which amounts to a superficial imitation of existing dialogue transcripts Liu et al. (2025) . These models lack a dynamic decision-making mechanism and cannot intelligently adjust their questioning strategies based on the real-time progress of the conversation to probe for key information. Even systems that integrate knowledge graphs Wang et al. (2023) ; Zhang et al. (2024) ; Feng et al. (2025) fundamentally rely on predefined dialogue paths, failing to achieve genuine clinical reasoning. This fundamentally solidifies the role of LLM as a passive knowledge reproducer, whose intelligence remains limited to reproducing existing knowledge rather than actively exploring the unknown. In recent years, research has begun to explore more dynamic approaches. For instance, systems like AMIE Tu et al. (2025) optimize the diagnostic dialogue itself through self-play simulations, while multi-agent frameworks such as Agent Hospital Li et al. (2024) , AI Hospital Fan et al. (2024) , and MAC Chen et al. (2025) simulate complex hospital environments or multi-role collaboration protocols. Additionally, EvoMDT Liu et al. (2026) introduces a self-evolving multi-agent system that automatically optimizes prompts and retrieval scopes based on feedback. However, these methods fundamentally rely on prompt-level optimizations and the synthesis of static information. They still lack the capability to optimize proactive inquiry strategies when facing the inherent uncertainty of patient-led interactions. Concurrently, the application of reinforcement learning (RL) in the medical field is transitioning toward such dynamic paradigms. MedVLM-R1 Pan et al. (2025) and Med-R1 Lai et al. (2025b) employ the Group Relative Policy Optimization (GRPO) framework Shao et al. (2024) to enhance diagnostic accuracy, while HuatuoGPT-o1 Chen et al. (2024) improves clinical reasoning through verifiable question generation. MedRIA Zou et al. (2024) and PPME Sun et al. (2025) further explore inquiry efficiency and experience replay. Despite these efforts, existing approaches lack robust frameworks for modeling dynamic doctor-patient interactions and validating performance in real clinical settings with high complexity and uncertainty. Figure 1 : Overview of DoctorAgent-RL Framework and Performance. The MTMedDialog Dataset: A high-fidelity multi-turn medical dialogue dataset supporting dynamic patient simulation, where realistic patient responses are generated based on comprehensive medical profiles. DoctorAgent-RL Training Strategy: A multi-agent reinforcement learning framework combining reasoning distillation and policy optimization, where the doctor agent learns strategic questioning through interaction with the patient agent under guidance from a consultation evaluator. Results: DoctorAgent-RL achieves state-of-the-art on the MTMedDialog task. To address these challenges, we propose DoctorAgent-RL—a multi-agent collaborative reinforcement learning framework that reformulates clinical reasoning as a Markov Decision Process (MDP), as shown in Figure 1 . Within this framework: (1) A high-fidelity patient agent based on LLMs, which generates pathologically consistent responses while mimicking the diversity of real-world communication; (2) A doctor agent initialized by cloning medical actions from real consultation records and refined through RL to master effective questioning strategies; and (3) A consultation evaluator that provides multi-dimensional rewards based on diagnostic accuracy, patient information responsiveness, and the standardization of questions. Through this framework, we train the DoctorAgent-RL model based on Qwen2.5-7B-Instruct Xu et al. (2025) , which redefines the diagnostic process through reward-based strategic policy optimization: the doctor agent continuously optimizes its questioning strategy, obtaining immediate feedback through multi-turn interactions with the patient agent, and dynamically adjusting the information-gathering path based on comprehensive rewards from the consultation evaluator. Here, the core intelligence of the doctor agent no longer lies in knowing the answer, but rather in learning and mastering a questioning methodology aimed at achieving optimal diagnosis through strategic questioning that guides the gradual emergence of key patient information throughout multi-turn dialogues. This end-to-end RL fine-tuning mechanism enables LLMs to autonomously evolve interaction strategies that align with clinical reasoning logic, rather than simply imitating surface patterns in existing dialogue data. While existing datasets provide valuable examples of multi-turn medical conversations, they consist of static transcripts. This limits their use for training agents that need to learn adaptive questioning strategies through interactive simulation. To address this gap, we constructed MTMedDialog, the first English dataset specifically designed to support dynamic, stateful patient simulations. Each dialogue is grounded in a comprehensive patient