The Optimization Paradox in Clinical AI Multi-Agent Systems Conference: ; ;

Suhana Bedi, Iddah Mlauzi, Daniel Shin, Sanmi Koyejo, Nigam H. Shah · 2025

Read The Optimization Paradox in Clinical AI Multi-Agent Systems Conference: ; ; by Suhana Bedi, Iddah Mlauzi, Daniel Shin, Sanmi Koyejo, Nigam H. Shah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06574.txt.

Introduction

Bedi and four co-authors report a surprising result about teams of AI agents in hospital diagnosis, in this two thousand twenty-five paper. Splitting the work among specialized agents sounds sensible, but the team asked whether the best parts actually make the best whole.

They tested single agents against multi-agent teams on two thousand four hundred real patient cases covering four abdominal diseases, breaking diagnosis into gathering facts, interpreting them, and listing possibilities. Multi-agent teams usually won, yet the team of the individually best components, despite excellent process scores, diagnosed far worse than the top team. The lesson is that how agents pass information matters as much as their solo skill, so clinical AI needs whole-system testing.

Catalog record

CREATORSuhana Bedi, Iddah Mlauzi, Daniel Shin, Sanmi Koyejo, Nigam H. Shah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06574

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Optimization Paradox in Clinical AI Multi-Agent Systems Conference: ; ; Suhana Bedi, Iddah Mlauzi, Daniel Shin, Sanmi Koyejo, Nigam H. Shah Abstract . Multi-agent artificial intelligence systems are increasingly deployed in clinical settings, yet the relationship between component-level optimization and system-wide performance remains poorly understood. We evaluated this relationship using 2,400 real patient cases from the MIMIC-CDM dataset across four abdominal pathologies (appendicitis, pancreatitis, cholecystitis, diverticulitis), decomposing clinical diagnosis into information gathering, interpretation, and differential diagnosis. We evaluated single agent systems (one model performing all tasks) against multi-agent systems (specialized models for each task) using comprehensive metrics spanning diagnostic outcomes, process adherence, and cost efficiency. Our results reveal a paradox: while multi-agent systems generally outperformed single agents, the component-optimized or Best of Breed system with superior components and excellent process metrics (85.5% information accuracy) significantly underperformed in diagnostic accuracy (67.7% vs. 77.4% for a top multi-agent system). This finding underscores that successful integration of AI in healthcare requires not just component level optimization but also attention to information flow and compatibility between agents. Our findings highlight the need for end to end system validation rather than relying on component metrics alone. 1. Introduction Artificial intelligence (AI) is rapidly transforming healthcare across diagnosis ( zhou2024largelanguagemodelsdisease ) , treatment planning ( wilhelm2023large ) , and patient management ( neo2024use ) . As AI systems grow in complexity, the focus has shifted from single-model solutions toward networks of specialized models (“agents”) ( sapkota2025aiagentsvsagentic ) that collaboratively handle different aspects of patient care. Recent studies, including Google DeepMind’s AMIE ( tu2025towards ) , have demonstrated agent-based systems exceeding primary care physicians’ performance in randomized clinical settings, while frameworks like MASH ( moritz2025coordinated ) and CRAFT-MD ( johri2024craftmd ) have explored both the potential and pitfalls of multi-agent approaches. Multi-agent AI systems mirror interdisciplinary healthcare teams, where specialists such as radiologists, pathologists, and physicians collaborate to synthesize comprehensive diagnoses. This modular approach can improve interpretability ( gordon2024ai ) , simplify troubleshooting, and enable task-specific optimization ( shu2025unlocking ) . However, a critical challenge arises from interactions among individually optimized agents ( eliot2024multiagent ) . We term this the Optimization Paradox: the phenomenon where excellent performance at the individual agent or component level does not necessarily translate to high overall system performance. This misalignment between individual and system-level effectiveness poses risks to patient safety and clinician trust. This study addresses the Optimization Paradox within clinical decision support systems by examining three essential components of the diagnostic process: information gathering (ordering appropriate clinical tests), interpretation (analyzing lab results), and differential diagnosis (identifying potential medical conditions) (Figure 1 ). We compare multi-agent systems, where specialized agents manage each task, to single-agent systems, where one model performs all tasks. Our evaluation uses the MIMIC-CDM dataset comprising 2,400 real patient cases across four common abdominal pathologies ( hager2024evaluation ) . We introduce clinically relevant evaluations extending beyond diagnostic accuracy to include process metrics (appropriate test ordering and accurate lab value interpretation) and cost efficiency metrics (clinical resource utilization and computational demands). Our findings reveal that while certain multi-agent systems achieve impressive process-level performance, this does not always translate into clinical effectiveness. The component-optimized or Best of Breed system exemplifies this paradox: despite achieving 85.5% accuracy in lab interpretation, its overall diagnostic accuracy (67.7%) was significantly lower than a top performing multi-agent system (77.4%; McNemar’s test, p < 0.001) without component optimization. This 10-percentage accuracy drop poses clinically significant risks, potentially increasing misdiagnoses and compromising patient outcomes when AI systems are deployed solely based on component-level validation ( Macaluso2012AbdominalPain ) . Our study underscores the necessity of rigorous, end-to-end validation of AI systems prior to clinical implementation, emphasizing that effective patient outcomes depend on careful system-wide integration rather than isolated component excellence. Figure 1. Overview of experimental methodology and the Optimization Paradox. The Clinical Decision Support Task Decomposition (left) breaks the diagnostic process into three specialized components. The Experimental Workflow (center) shows Phase 1 component selection and Phase 2 system comparison. The Optimization Paradox (right) illustrates the counterintuitive finding: while the Best of Breed system was constructed from top-performing components, it achieved poor diagnostic accuracy compared to alternative systems. 2. Methods This study used the MIMIC-CDM dataset, a curated subset of the MIMIC-IV database containing 2,400 real patient cases from Beth Israel Deaconess Medical Center ( hager2024evaluation ) . We focused on four common abdominal pathologies: appendicitis (957 cases), pancreatitis (538 cases), cholecystitis (648 cases), and diverticulitis (257 cases). These pathologies were selected based on three criteria: high prevalence at the emergency department (ED) ( DiSaverio2020WSES ; cervellin2016epidemiology ) , high diagnostic complexity due to overlapping clinical presentations ( vanBredaVriesman2006Mimics ) , and availability of extensive clinical data within MIMIC-CDM for robust case development. This combination creates an ideal testbed for evaluating AI-based differential diagnosis systems. All patients in this cohort presented to the ED with acute abdominal pain and received one of these four conditions as their primary diagnosis. Each case includes comprehensive clinical data: patient history of present illness, physical examination findings, laboratory results (138,788 lab values from 480 unique tests), imaging reports (5,959 reports including abdominal CT, ultrasound, and X-rays), and procedural information. All available data was de-identified, with any mentions of the primary diagnosis replaced with underscores to prevent trivial pattern matching. 2.1. Decomposition of the Clinical Decision Support Task We adapted MIMIC-CDM’s evaluation framework and decomposed the diagnostic workflow into three distinct tasks: • Information Gathering: Requesting relevant clinical information based on patient presentation, including physical examination findings, laboratory tests, and imaging studies. • Information Interpretation: Processing and interpreting raw clinical data, such as classifying laboratory results as “high,” “normal,” or “low” relative to reference ranges. • Differential Diagnosis: Synthesizing gathered and interpreted information to generate a ranked list of potential diagnoses through clinical reasoning. We compared two system designs: single-agent systems where one LLM performed all three tasks end-to-end, versus multi-agent systems that divided these tasks among specialized agents. To maintain data integrity, we employed a Retriever LLM that processed information gathering requests and retrieved only specified tests from patient records. GPT-4o was selected for its cost-effectiveness and 100% retrieval accuracy on the preliminary set. The retrieval prompt is in Appendix 5.1 . 2.2. Data Splits We set aside 20 cases (5 per pathology) as a pilot set for prompt development and pipeline testing during initial data preparation. We excluded these cases from all subsequent model evaluations. Appendix 5.1 details the prompts we developed. We then divided the remaining 2,380 cases into two datasets: • Development Set: We used 1,190 cases (50%), stratified by pathology, for Phase 1 component selection. This identified the best agent for each role to construct the Best of Breed system. No training or fine-tuning was performed. • Test Set: We used the remaining 1,190 cases (50%), stratified by pathology, as our held-out test set for Phase 2 evaluation of all systems to assess final performance. 2.3. Model Implementation We created agents using LLMs from multiple families including GPT (GPT-4o, GPT-4.1), Claude (Claude-3.5-Sonnet), Gemini (Gemini-1.5-Pro, Gemini-2.0-Flash), Llama (Llama-3.3-70b), and reasoning models (o3-mini, DeepSeek-R1). All experiments were conducted on a PHI-compliant shared cluster. API calls to all models were made through the institution’s secure Azure instance, ensuring patient data remained within

Read the complete text