Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support

Chen Zhan, Xihe Qiu, Xiaoyu Tan, Xibing Zhuang, Gengchen Ma, Yue Zhang, Shuo Li, Peifeng Liu, Xiaoxiao Ge, Liang Liu, Lu Gan · 2026

Read Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support by Chen Zhan, Xihe Qiu, Xiaoyu Tan, Xibing Zhuang, Gengchen Ma, Yue Zhang, Shuo Li, Peifeng Liu, Xiaoxiao Ge, Liang Liu, Lu Gan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22047.txt.

Introduction

Chen Zhan and co-authors study active evidence-seeking in large language models for clinical decision support. Models do well on static medical exams, yet real diagnosis needs iterative evidence gathering under uncertainty. The team builds an OSCE-inspired patient simulator and a controlled benchmark for active diagnostic inquiry. In 468 cases and 15 models, multi-turn evidence seeking cuts diagnostic accuracy by 12.75 percent and supporting-evidence quality by 24.36 percent versus full-context evaluation, linked to premature closure and poor questioning. Static benchmarks may thus inflate interactive performance.

Catalog record

CREATORChen Zhan, Xihe Qiu, Xiaoyu Tan, Xibing Zhuang, Gengchen Ma, Yue Zhang, Shuo Li, Peifeng Liu, Xiaoxiao Ge, Liang Liu, Lu Gan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79594
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22047

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support Chen Zhan, Xihe Qiu, Xiaoyu Tan, Xibing Zhuang, Gengchen Ma, Yue Zhang, Shuo Li, Peifeng Liu, Xiaoxiao Ge, Liang Liu, Lu Gan Abstract Large language models perform well on static medical examinations, yet clinical diagnosis often requires iterative evidence gathering under uncertainty. Building on prior interactive evaluation efforts, we introduce an OSCE-inspired standardized patient simulator and a controlled, reproducible benchmark for active diagnostic inquiry. Across 468 cases and 15 models in our protocol, we observe that multi-turn evidence seeking reduces diagnostic accuracy by 12.75% and lowers supporting-evidence quality by 24.36% relative to full-context evaluation; error analyses associate these drops with premature diagnostic closure and inefficient questioning. Together, these results suggest that static full-context benchmarks may overestimate performance in interactive evidence-seeking settings, motivating complementary interactive assessment for safer clinical decision support. keywords Large language models, Clinical decision support, Standardized patient simulation, Interactive diagnostic reasoning † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Introduction In recent years, general-purpose large language models (LLMs) have demonstrated strong capabilities in medical knowledge retrieval and reasoning ( 25 ; 17 ; 33 ) . Results from static medical benchmarks such as MedQA-USMLE, PubMedQA, and MedMCQA suggest that models ranging from ChatGPT to medically specialized variants such as Med-PaLM 2 can meet or exceed passing thresholds on standardized licensing-style examinations ( 19 ; 20 ; 31 ; 34 ; 30 ; 3 ) . These achievements have motivated growing interest in using LLMs for Clinical Decision Support Systems. However, many widely used evaluation settings remain largely passive and full-context: complete case information, including patient history, symptoms, and test results, is provided to the model simultaneously before it generates a response ( 26 ; 2 ) . This setup is well-suited for assessing summary interpretation given complete information, but it does not directly test how models decide what evidence to request when information is initially sparse. As a result, it can blur the distinction between using provided facts and actively reasoning under uncertainty in iterative clinical workflows. In practice, clinical diagnosis is a dynamic, multi-turn interactive process governed by the hypothetico-deductive model ( 5 ) . Clinicians typically do not receive all relevant data points instantaneously, especially early in a workup. Instead, they must actively formulate initial hypotheses and seek evidence through successive exchanges involving history taking, physical examinations, and laboratory testing to dynamically confirm or refute differential diagnoses ( 8 ) . Medical education often uses the Objective Structured Clinical Examination (OSCE) to assess clinicians during such active reasoning processes ( 14 ; 23 ) . By utilizing standardized patient simulations, the OSCE evaluates key competencies including active inquiry, information acquisition strategies, and the ability to synthesize evolving data ( 1 ) . This creates a partial mismatch between real diagnostic workflows and many existing evaluation setups: static information feeds can underrepresent the role of active evidence acquisition and iterative decision-making. Several recent studies have moved toward more clinically grounded evaluations by introducing interactive inquiry or multi-step diagnostic procedures ( 44 ; 18 ; 41 ) . For example, MediQ proposed a simulated clinical dialogue framework to assess information gathering through question-answer interactions, highlighting that under incomplete initial information even large models may struggle to generate efficient and clinically meaningful follow-up questions ( 24 ; 25 ) . Other efforts structure evaluation into staged diagnosis such as preliminary diagnosis, differential diagnosis, and final diagnosis to better reflect sequential decision making. Nonetheless, these approaches involve practical trade-offs, and it is not always straightforward to isolate information acquisition strategies from diagnostic reasoning performance: performance can be influenced by open-ended dialogue variability, prompting choices, and scoring criteria, which can make it difficult to determine whether errors stem from suboptimal evidence seeking or from evidence synthesis ( 4 ; 32 ; 36 ; 38 ) . In addition, work examining conversational quality and interpretability further suggests that static question-answer benchmarks do not adequately characterize performance during realistic clinical interactions. To better approximate authentic diagnostic processes, some initiatives have explored system designs that support cyclical inquiry and dialogue management ( 29 ) . For instance, the Articulate Medical Intelligence Explorer, known as AMIE, studied an optimized conversational diagnostic system to evaluate performance across multiple dimensions including history collection, diagnostic accuracy, and communication skills ( 37 ) . Another representative work, Ask Patients with Patience, structured multi-turn inquiry into iterative dialogues driven by medical guidelines ( 46 ; 39 ) . While these studies demonstrate the value of dialogue-oriented designs, they are often optimized for specific systems, and their evaluation protocols may not directly provide a standardized, reusable benchmark. These systems often rely on open-ended conversations, which can complicate reproducibility and objective grading without human intervention ( 45 ) . Consequently, there remains a need for benchmarks that combine controlled information release, reproducibility, and objective scoring-especially for separating evidence acquisition from evidence synthesis for systematically evaluating model capabilities in active evidence seeking and clinical reasoning within a controlled and reproducible environment that can isolate specific failure modes ( 22 ; 12 ) . To complement existing static and dialogue-based evaluations, we introduce an OSCE-inspired interactive framework, ROUNDS-Bench. This framework incorporates a Standardized Patient Simulator and controls information release through a multi-turn inquiry design, providing evidence only when the model submits logical clinical requests ( 9 ; 28 ) . Compared with open-ended chat evaluations, this request–response protocol improves controllability and reproducibility. It resembles common query workflows (e.g., electronic health record retrieval), enabling measurement of questioning efficiency and diagnostic performance across turns during both the information acquisition and reasoning phases separately ( 18 ) . Using 468 diverse cases covering six major clinical systems, we designed a dual-task evaluation structure to quantify the performance gap between full-context and active inquiry settings. Task 1, Full-Context Diagnosis, establishes the upper bound of model performance under conditions of complete information, whereas Task 2, Active Evidence Seeking, requires the model to autonomously drive a multi-turn diagnostic process starting with only the chief complaint. A series of systematic evaluations on current state-of-the-art Large Language Models, including GPT-4o and Qwen ( 16 ) , revealed a significant capability gap. When transitioning from the passive diagnostic task to the active diagnostic task, diagnostic accuracy decreases by 12.75% on average in our setting. Supporting-evidence quality decreases by 24.36% under active inquiry. This gap matters because high scores under full-context evaluation may not fully reflect whether a model can elicit and cite discriminative evidence during interactive diagnostic workflows. Specifically, models may output correct diagnoses without consistently eliciting, selecting, or citing the key evidence needed to justify the decision. Figure 1 : Overview of the ROUNDS-Bench framework for evaluating active clinical reasoning. A Paradigm Comparison. Contrasts the authentic OSCE clinical workflow with the passive Full-Context baseline and the proposed Active Evidence-Seeking interactive protocol. B Technical Pipeline. Illustrates the workflow integrating dataset structuring, dual-task evaluation (Task 1 vs. Task 2), and multi-dimensional grading. C Dataset Stratification. Anatomical visualization of the six clinical systems covered (top) and the distribution of

Read the complete text