MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning
Hongjun Liu, Yinghao Zhu, Yuhui Wang, Yitao Long, Zeyu Lai, Lequan Yu, Chen Zhao · 2025
Read MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning by Hongjun Liu, Yinghao Zhu, Yuhui Wang, Yitao Long, Zeyu Lai, Lequan Yu, Chen Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-24314.txt.
Introduction
A shaky first read of evidence can send medical reasoning down a wrong branch that ends in a confident error. Hongjun Liu and 6 co-authors built MedMMV, a multi-agent framework meant to keep clinical reasoning stable and checkable. Short, varied rollouts diversify the first steps instead of locking onto one path.
Intermediate steps are grounded in a structured evidence graph watched by a hallucination detector. A combined uncertainty scorer then merges candidate paths. Across six medical benchmarks, accuracy rose by up to about an eighth. Blind physician reviews found the reasoning more truthful without loss of content. The design offers a robust route for high-stakes clinical support.
Catalog record
| CREATOR | Hongjun Liu, Yinghao Zhu, Yuhui Wang, Yitao Long, Zeyu Lai, Lequan Yu, Chen Zhao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-24314
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning Hongjun Liu, Yinghao Zhu, Yuhui Wang, Yitao Long, Zeyu Lai, Lequan Yu, Chen Zhao Abstract Recent progress in multimodal large language models (MLLMs) has demonstrated promising performance on medical benchmarks and in preliminary trials as clinical assistants. Yet, our pilot audit of diagnostic cases uncovers a critical failure mode: instability in early evidence interpretation precedes hallucination, creating branching reasoning trajectories that cascade into globally inconsistent conclusions. This highlights the need for clinical reasoning agents that constrain stochasticity and hallucination while producing auditable decision flows. We introduce MedMMV, a controllable multimodal multi-agent framework for reliable and verifiable clinical reasoning. MedMMV stabilizes reasoning through diversified short rollouts, grounds intermediate steps in a structured evidence graph under the supervision of a Hallucination Detector, and aggregates candidate paths with a Combined Uncertainty scorer. On six medical benchmarks, MedMMV improves accuracy by up to 12.7% and, more critically, demonstrates superior reliability. Blind physician evaluations confirm that MedMMV substantially increases reasoning truthfulness without sacrificing informational content. By controlling instability through a verifiable, multi-agent process, our framework provides a robust path toward deploying trustworthy AI systems in high-stakes domains like clinical decision support. 1 Introduction Recent frontier multimodal large language models (MLLMs), such as Claude-Sonnet-4 ( Anthropic 2025 ) and GPT-5 ( OpenAI 2025 ) , are beginning to translate strong general reasoning abilities into healthcare applications. When combined with techniques like chain-of-thought (CoT) prompting ( Wei et al. 2023 ) , these systems have achieved state-of-the-art performance on medical question answering benchmarks, narrowing the gap to human expert performance ( Singhal et al. 2025 ; Bhayana et al. 2023 ; Sandmann et al. 2025 ; Bedi et al. 2025 ) . Beyond static benchmarks, MLLMs are increasingly being evaluated as agentic clinical assistants in realistic settings, including randomized, double-blind standardized-patient trials, with considerable utility gains ( Tu et al. 2024 ; Tu et al. 2025 ; Schmidgall et al. 2025 ; Qiu et al. 2024 ; Zhu et al. 2025b ) . Despite this progress, deploying agent-based systems in real-world healthcare is challenging due to the need for extreme reliability ( Zhu et al. 2025a ) . As illustrated in Figure 1 , existing systems face two primary issues that are largely overlooked by benchmarks focused on final-answer accuracy: (1) instability , where decisions are highly sensitive to noisy or context-dependent data, and minor perturbations can lead to different outcomes ( Singhal et al. 2023a ) ; and (2) hallucinated reasoning chains , where models generate fabricated or unsupported facts to justify their conclusions. To investigate this, we conducted a pilot audit of 100 clinical diagnostic cases, generating multiple responses per case and applying two probes: the random guess measure (RGM) for decision instability and the cross-modal hallucination rate (CMHR) to quantify evidence fabrication. Our audit revealed a critical failure pattern: instability often precedes hallucination. Specifically, when a model’s reasoning was unstable, indicated by high prior entropy across runs, it frequently switched its majority-voted reasoning path. Such a switch increased the immediate likelihood of hallucination by over 13%, ultimately leading to globally inconsistent conclusions. Figure 1: An illustration of MedMMV compared to standard CoT reasoning. Top-left : An example patient case as input. Top-right : Performance comparison between MedMMV and CoT. Bottom-left : Direct CoT produces unstable trajectories, randomly switching among divergent paths and incorporating hallucinated evidence, leading to fragile and unreliable diagnoses. Bottom-right : MedMMV mitigates these issues through a three-stage workflow centered on parallel exploration and evidence-graph-grounded verification, ensuring verifiable and clinically reliable reasoning. This empirical evidence motivates our central research question: How can we design clinical agent systems that explicitly constrain stochasticity and hallucination while exposing auditable representations of their decision-making process? We present MedMMV, a multimodal multi-agent framework that directly addresses the error modes observed in our audit. Rather than committing to a single reasoning trajectory early, MedMMV explores diverse diagnostic hypotheses through multi-round short rollouts, reducing path instability at uncertain decision points. Each path is then refined under the supervision of a Hallucination Detector, which grounds reasoning steps in an evidence graph and prevents local errors from cascading. The refined candidates are finally aggregated by a Combined Uncertainty (CU) Scorer, yielding diagnoses that are not only accurate but also verifiably supported. To validate the effectiveness of MedMMV, we conduct extensive experiments on six public medical benchmarks spanning both multimodal VQA and text-based QA. As shown in Figure 1 , MedMMV consistently improves accuracy, achieving gains of 7.9% on MedXpert-MM and 12.7% on MedFrameQA when using GPT-5 as the executor. Reliability-oriented metrics further highlight that MedMMV improves truthfulness (TRUE) while maintaining informativeness (INFO), yielding TRUE \times INFO gains of 8–12% across datasets. Beyond automated evaluation, physician studies confirm MedMMV’s clinical reliability, with our framework achieving a TRUE score of 4.36 versus 3.49 for CoT. Ablation studies demonstrate that hallucination control and uncertainty-aware aggregation are the main contributors; removing the CU scorer drops accuracy by 11% and TRUE \times INFO by 13%. In summary, our contributions are threefold: (1) Empirical Analysis : We identify a critical failure mechanism where reasoning instability serves as a direct precursor to hallucination, showing how early-stage stochasticity in evidence interpretation cascades into global inconsistency. (2) MedMMV Framework : We propose a controllable, multimodal multi-agent reasoning system that integrates diversified rollouts, evidence-grounded refinement, and uncertainty-aware aggregation to mitigate these errors. (3) Comprehensive Evaluation : Through experiments on six benchmarks, physician studies, and ablations, we demonstrate state-of-the-art reliability and provide new insights into building trustworthy clinical reasoning systems. 2 Related Work Consistency and hallucination in multimodal clinical reasoning. Ensuring process fidelity is especially challenging in high-stakes, multimodal clinical settings, where sparse or misaligned evidence can trigger random guessing and cross-modal hallucinations ( Liu et al. 2024 ; Bai et al. 2025b ) . Existing medical MLLMs like LLaVA-Med ( Li et al. 2023a ) and Med-Flamingo ( Moor et al. 2023 ) show progress on image-text QA but still lack robust uncertainty calibration and step-level verifiability. While self-consistency reduces variance, it cannot guarantee evidence-faithful reasoning ( Wang et al. 2023 ) . Benchmarks such as POPE ( Li et al. 2023c ) , HallusionBench ( Guan et al. 2024 ) , Med-HALT ( Pal et al. 2023 ) , and ConBench ( Zhang et al. 2024 ) expose hallucination patterns and assess trajectory-level consistency. Distinct from these probes, our analysis highlights how early stochasticity in evidence interpretation branches reasoning trajectories and cascades into global inconsistency, echoing the “snowballing” effect in multimodal hallucinations ( Zhong et al. 2024 ) . To capture this process, we employ dispersion across paths (RGM) and cross-modal hallucination rate (CMHR) not as end goals but as early-warning signals of this cascading dynamic. Controllable reasoning and process-level regulation. A growing body of literature shows that process-level supervision and verification, rather than outcome-only scoring, improves reliability by checking intermediate steps and rationales ( Lightman et al. 2023 ; Stiennon et al. 2022 ) . In parallel, uncertainty estimation enables models to “know when they know” ( Kadavath et al. 2022 ) , providing a principled basis for gating and adjudication. Exploration strategies such as CoT ( Wei et al. 2023 ) , self-consistency ( Wang et