MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering

1 st Hai-Dang Nguyen, 2 nd Minh-Anh Dang, 3 rd Minh-Tan Le, 4 th Minh-Tuan Le · 2025

Read MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering by 1 st Hai-Dang Nguyen, 2 nd Minh-Anh Dang, 3 rd Minh-Tan Le, 4 th Minh-Tuan Le (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22803.txt.

Introduction

Doctors need clear reasons before they trust AI answers. Hai-Dang Nguyen, Minh-Anh Dang, Minh-Tan Le, and Minh-Tuan Le address that need in 2025. Their system is called MedXplain-VQA. It answers questions about medical images. Five explainable parts work together in the frame.

A tuned BLIP-2 model forms the backbone. Query reformulation sharpens each question. Better attention maps highlight key regions. A language model then gives structured reasoning steps. The team built a clinical scoring frame. It checks terminology, structure, and attention relevance. On five hundred pathology samples, scores rose a lot over baselines. Each case flagged three to five key regions.

Catalog record

CREATOR1 st Hai-Dang Nguyen, 2 nd Minh-Anh Dang, 3 rd Minh-Tan Le, 4 th Minh-Tuan Le
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22803

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering Thanks: GPU support: Mr. Trong, https://ThueGPU.vn Source code available at: https://github.com/dangindev/medxplain-vqa 1 st Hai-Dang Nguyen, 2 nd Minh-Anh Dang, 3 rd Minh-Tan Le, 4 th Minh-Tuan Le Abstract Explainability is critical for the clinical adoption of medical visual question answering (VQA) systems, as physicians require transparent reasoning to trust AI-generated diagnoses. We present MedXplain-VQA, a comprehensive framework integrating five explainable AI components to deliver interpretable medical image analysis. The framework leverages a fine-tuned BLIP-2 backbone, medical query reformulation, enhanced Grad-CAM attention, precise region extraction, and structured chain-of-thought reasoning via multi-modal language models. To evaluate the system, we introduce a medical-domain-specific framework replacing traditional NLP metrics with clinically relevant assessments, including terminology coverage, clinical structure quality, and attention region relevance. Experiments on 500 PathVQA histopathology samples demonstrate substantial improvements, with the enhanced system achieving a composite score of 0.683 compared to 0.378 for baseline methods, while maintaining high reasoning confidence (0.890). Our system identifies 3-5 diagnostically relevant regions per sample and generates structured explanations averaging 57 words with appropriate clinical terminology. Ablation studies reveal that query reformulation provides the most significant initial improvement, while chain-of-thought reasoning enables systematic diagnostic processes. These findings underscore the potential of MedXplain-VQA as a robust, explainable medical VQA system. Future work will focus on validation with medical experts and large-scale clinical datasets to ensure clinical readiness. I INTRODUCTION The integration of artificial intelligence in medical diagnostics has reached a critical juncture where technical capability must align with clinical acceptance standards. While AI systems demonstrate impressive performance on medical image analysis tasks, their adoption in healthcare settings remains limited by the fundamental requirement for transparent, explainable decision-making processes that medical professionals can understand, validate, and trust [ 1 , 2 ] . Medical visual question answering represents a particularly challenging domain where this explainability gap becomes pronounced. Unlike general computer vision applications, medical VQA systems must satisfy stringent clinical requirements: regulatory compliance for diagnostic tools, educational value for medical training, and transparent reasoning that enables physician validation of AI conclusions [ 3 , 4 ] . Current approaches, however, primarily optimize for answer accuracy while treating explainability as a secondary consideration. Existing medical VQA systems exhibit critical limitations that impede clinical deployment. Most systems function as ”black boxes,” providing diagnostic conclusions without systematic explanation of the underlying reasoning process [ 5 ] . Traditional evaluation frameworks borrowed from natural language processing fail to capture the clinical relevance and educational value essential for healthcare applications [ 6 ] . While recent advances in foundation models [ 7 ] and structured reasoning [ 8 ] show promise, these techniques lack the medical domain adaptation and systematic integration necessary for comprehensive clinical explainability. This fundamental challenge requires moving beyond post-hoc explainability approaches toward systems designed with transparency as a core architectural principle. We address these challenges through MedXplain-VQA, a comprehensive framework that systematically integrates multiple explainable AI components designed specifically for medical applications. Our approach represents a paradigm shift from accuracy-focused systems toward explainability-first design, combining domain-adapted foundation models with medical context enhancement, sophisticated attention mechanisms, and structured diagnostic reasoning. Our primary contributions include: (1) Multi-Component Explainable Architecture: A systematic framework integrating five complementary AI techniques designed specifically for transparent medical image analysis and diagnostic reasoning. (2) Medical-Domain Evaluation Methodology: Novel assessment framework addressing the limitations of traditional NLP metrics through clinically relevant measurements of medical terminology, reasoning quality, and attention precision. (3) Systematic Component Integration Analysis: Comprehensive evaluation revealing the synergistic effects and individual contributions of query enhancement, visual attention, and structured reasoning in medical VQA applications. (4) Clinical Transparency Standards: Establishment of evaluation protocols that address medical education requirements and clinical decision support transparency standards. The remainder of this paper reviews related work in medical VQA and explainable AI (Section II), presents our comprehensive methodology (Section III), evaluates the system through systematic experiments (Section IV), discusses findings and clinical implications (Section V), and concludes with future research directions (Section VI). II RELATED WORK II-A Domain Adaptation in Medical VQA Medical visual question answering requires combining visual and textual modalities under domain-specific constraints. General-purpose vision-language models like BLIP-2 [ 7 ] achieve strong results on open-domain VQA, but models pre-trained on natural images struggle with medical images due to distribution shifts and specialized visual patterns. Recent approaches address this through domain adaptation. LLaVA-Med [ 9 ] fine-tunes multimodal models on biomedical data, enabling sophisticated medical image understanding and outperforming prior supervised methods. Text-only medical LLMs like ChatDoctor [ 10 ] improve domain knowledge via fine-tuning but lack visual components. These works highlight the need for adapting multimodal systems to medical terminology and data scarcity, though most focus on accuracy rather than explainability. MedXplain-VQA builds on multimodal foundation models by incorporating medical domain adaptation alongside systematic explainability mechanisms, ensuring the model both understands specialized inputs and transparently conveys reasoning in medical contexts. II-B Chain-of-Thought Reasoning in Medical AI Explaining how models arrive at answers is crucial in healthcare, where traditional VQA systems produce direct answers without rationale, hindering trust. Large language models demonstrate that step-by-step reasoning significantly improves complex question answering [ 11 ] , with chain-of-thought prompting eliciting intermediate reasoning steps for more transparent solutions. In medical domains, Med-PaLM 2 [ 12 ] combines medical fine-tuning with advanced reasoning strategies to achieve near-expert performance, while ChatDoctor [ 10 ] demonstrates that infusing clinical knowledge enhances answer accuracy. However, these text-based models lack visual integration. Recent work extends explainable reasoning to VQA. MedThink [ 13 ] introduced ”medical chain of thought” paradigm, augmenting VQA datasets with intermediate reasoning steps. Such methods show that multi-step explanations clarify decision processes and improve performance, though prior approaches either ignore images or add rationales without ensuring visual grounding. MedXplain-VQA integrates chain-of-thought reasoning within the VQA pipeline, generating answers with stepwise explanations that reference image findings, effectively merging visual analysis with logical reasoning for clinical applications. II-C Visual Attention and Grounding Visual grounding techniques pinpoint where models focus when answering questions, providing interpretability through attention mechanisms that highlight important image regions. MedFuseNet [ 14 ] employed attention-based fusion for medical VQA with interpretable attention maps, while post-hoc methods like Grad-CAM highlight critical regions influencing CNN-based medical predictions [ 15 ] . A key challenge is that attention maps can appear plausible without being faithful [ 16 ] — looking convincing while not truly reflecting decision processes. Models might attend to correct regions visually yet rely on spurious cues. Recent VQA grounding studies propose metrics requiring that answers change when relevant regions are masked, ensuring both ”faithful” and ”plausible” grounding. Most existing medical VQA systems do not enforce this consistency between highlighted regions and actual model influences. MedXplain-VQA enhances visual grounding by combining attention-based saliency maps with bounding box extraction, feeding these regions into the reasoning module. Generated explanations reference highlighted areas, promoting stronger alignment between visual evidence and textual justification compared to standalone attention visualization. II-D Evaluation of Medical Explainability Evaluating explainability remains complex, especially without ground-truth rationales for medical images. Traditional VQA metrics (accuracy, BLEU) inadequately assess explanation quality [ 17 ] , as correct answers don’t guarantee sound reasoning usable by clinicians. Recent approaches compare generated explanations to reference texts using NLP metrics or measure faithfulness

Read the complete text