Medic-AD : Towards Medical Vision-Language Model’s Clinical Intelligence

Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do · 2026

Read Medic-AD : Towards Medical Vision-Language Model’s Clinical Intelligence by Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27176.txt.

Introduction

Park and co-authors present MEDIC-AD, a medical vision-language model built for clinical usefulness. Learnable anomaly-aware tokens steer the model toward abnormal regions first. Difference tokens then encode how two studies of a patient compare. Those tokens separate worsening, improvement, and stability in disease burden. A final stage trains heatmaps that show the lesion evidence behind each call. The staged design reached state-of-the-art results on detection and segmentation. Longitudinal hospital data confirmed stable, faithful explanations.

Catalog record

CREATORWoohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40377
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27176

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Medic-AD : Towards Medical Vision-Language Model’s Clinical Intelligence Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do Abstract Lesion detection, symptom tracking, and visual explainability are central to real-world medical image analysis, yet current medical Vision-Language Models (VLMs) still lack mechanisms that translate their broad knowledge into clinically actionable outputs. To bridge this gap, we present Medic-AD , a clinically oriented VLM that strengthens these three capabilities through a stage-wise framework. First, learnable anomaly-aware tokens ( <Ano> ) encourage the model to focus on abnormal regions and build more discriminative lesion centered representations. Second, inter-image difference tokens ( <Diff> ) explicitly encode temporal changes between studies, allowing the model to distinguish worsening, improvement, and stability in disease burden. Finally, a dedicated explainability stage trains the model to generate heatmaps that highlight lesion-related regions, offering clear visual evidence that is consistent with the model’s reasoning. Through our staged design, Medic-AD steadily boosts performance across anomaly detection, symptom tracking, and anomaly segmentation, achieving state-of-the-art results compared with both closed source and medical-specialized baselines. Evaluations on real longitudinal clinical data collected from real hospital workflows further show that Medic-AD delivers stable predictions and clinically faithful explanations in practical patient-monitoring and decision-support workflows. Figure 1 : Overall performance of VLMs on Medical Anomaly Detection and Medical Symptom Tracking (MMXU 42 ). 1 Introduction Vision–Language Models (VLMs) have rapidly evolved 40 ; 66 ; 14 from simple image–text tasks such as visual question answering (VQA) 2 and captioning 52 to more advanced capabilities including visual grounding 60 ; 47 and multi–image reasoning 34 ; 3 ; 11 . These advances have inspired the emergence of Medical Foundation VLMs 35 ; 50 ; 41 , which aim to integrate visual and textual medical knowledge for comprehensive diagnostic understanding. Trained on large-scale image-report pairs and multimodal instructions, these models have achieved strong results across tasks such as disease classification, report generation, and Med–VQA, demonstrating the promise of language-driven clinical reasoning. However, most Medical Foundation VLMs remain optimized for broad medical knowledge coverage rather than real clinical application 59 ; 53 . Their training typically relies on long–form captioning, OCR–based instruction tuning, and medical chain–of–thought reasoning that enhance generic reasoning ability, but overlook key properties required for real-world clinical workflows 1 : (1) accurate lesion detection, (2) reliable temporal symptom tracking, and (3) transparent visual explainability of the reasoning process. Addressing these limitations demands a paradigm shift from generalized intelligence toward clinically grounded perception, and understanding. To that end, we explore three research questions guiding the design of a clinically usable medical VLM. RQ1: How can lesion and symptom recognition be improved in VLMs for real clinical settings? Even as medical VLMs expand in knowledge, accurate abnormality detection remains essential for safe deployment. We define an abnormality as any pathological deviation within an image and propose to enhance this recognition through explicit anomaly–aware representations. By injecting learnable <Ano> tokens into the transformer layers, the model highlights abnormal regions and strengthens its discriminative reasoning. Experiments on brain MRI, head CT, and chest X-ray datasets show that this design achieves strong performance in medical anomaly detection. RQ2: How can a VLM disentangle temporal medical images to enable more accurate symptom tracking? Existing foundation models that support multi–image inputs typically concatenate visual features, thus failing to capture the temporal progression between scans. To model clinically meaningful changes, we introduce <Diff> tokens that compare anomaly features extracted from multiple images of the same patient. These representations allow the model to reason about whether a condition has worsened, improved, or remained unchanged. On benchmarks such as MMXU 42 , which assess longitudinal symptom understanding, our approach achieves superior performance, highlighting its effectiveness as a practical clinical tool for patient monitoring. RQ3: How can visual explainability be integrated into medical VLM reasoning? Explainability is indispensable for clinical decision-making. To visually justify model’s predictions, we design a heatmap decoder that fuses anomaly features and visual features to generate visualization maps highlighting regions responsible for each prediction. These region-level explanations enhance transparency by providing visual evidence for both lesion detection and change assessment, ultimately bridging the gap between black-box reasoning and clinical trust. These three research directions culminate in Medic-AD , a stage-wise medical VLM with clinical intelligence, designed to integrate anomaly detection, temporal reasoning, and visual explainability. Medic-AD is trained in three stages: Stage 1: Anomaly Detection. Learn discriminative abnormality embeddings via injected anomaly tokens, <Ano> , adapting contrastive architecture between normal and abnormal regions for enhancing sensitivity to pathological cues. Stage 2: Difference Reasoning. Encode cross-scan variations using <Diff> tokens that disentangle temporal progression of abnormal features and enable fine-grained symptom tracking. Stage 3: Visual Explainability. Generate visual evidence heatmaps that ground textual outputs on abnormal regions, ensuring verifiable reasoning. Through extensive evaluation, Medic-AD consistently demonstrates state-of-the-art (SOTA) performance across diverse medical modalities and tasks as shown in Fig. 1 . It outperforms medical foundation models 35 ; 59 ; 53 , anomaly–specialized models 17 ; 58 , and closed-source counterparts 24 ; 4 in both lesion detection, and temporal symptom reasoning. Moreover, Medic-AD delivers superior visual explainablity generating spatially grounded explanations that align model decisions with clinical evidence. Beyond numerical gains, its stage–wise design encodes the clinical diagnostic workflow—detect, compare, explain—into the model’s learning curriculum, transforming general-purpose vision–language understanding into clinically actionable intelligence. Our main contributions are as follows: • We present a unified, stage-wise framework ( Medic-AD ) that integrates anomaly detection, longitudinal reasoning, and visual grounding to enable explainable medical inference. • We introduce anomaly- and difference-token mechanisms that endow medical VLMs with explicit lesion sensitivity and temporal reasoning capability. • We conduct comprehensive evaluations on multiple medical tasks, as well as on real longitudinal datasets from hospital sites, demonstrating superior reliability and usability compared to both open- and closed-source foundation models, and showcasing deployment readiness for real-world clinical practice workflows. 2 Related Works 2.1 Vision-Language Models for Medical Imaging Vision Language Models (VLMs) 40 ; 66 ; 14 ; 34 ; 3 ; 11 have unified visual and textual reasoning across domains through large-scale contrastive or instruction-tuned learning. Building upon these, medical VLMs have adapted multimodal alignment to clinical imaging and reporting tasks. Early medical VLMs focused on contrastive alignment and report-level representation learning 54 ; 6 , while later instruction-tuned architectures expanded multimodal reasoning through larg scale medical–text pretraining 35 ; 57 ; 49 ; 51 . More recently, Lingshu and Citrus-V 59 ; 53 , built on Qwen-VL 2.5 3 , introduced multi-stage training with shallow/deep alignment, medical instruction tuning, and reinforcement learning with verifiable rewards, achieving state-of-the-art results on single-image medical VQA benchmarks such as SLAKE, PathVQA, VQA-RAD, and OmniMedVQA 51 ; 19 ; 33 ; 22 . These datasets collectively evaluate anatomical localization, factual consistency, and report generation, forming the empirical foundation for single-image medical VLMs. Beyond single-image understanding, medical VLMs have advanced toward difference-aware reasoning, modeling longitudinal changes and disease progression between paired studies. Generic difference captioning frameworks describe visual changes across image pairs 46 ; 61 , whereas medical-specific longitudinal reasoning integrates anatomical or report-based temporal modeling 21 ; 12 . A recent unified framework 15 proposes a Report Generator–Answer

Read the complete text