MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays

Sunghwan Steve Cho, Yunseok Han, Jaeyoung Do, AIDAS Laboratory, ECE & IPAI, Seoul National University {steve97, qicher, jaeyoung.do} @snu.ac.kr · 2026

Read MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays by Sunghwan Steve Cho, Yunseok Han, Jaeyoung Do, AIDAS Laboratory, ECE & IPAI, Seoul National University {steve97, qicher, jaeyoung.do} @snu.ac.kr (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15574.txt.

Introduction

Sunghwan Steve Cho, Yunseok Han, and Jaeyoung Do introduce MI-CXR, a benchmark for reasoning over chest X-ray sequences spanning five patient visits. Its five-choice questions cover locating events in time, reasoning about changes between intervals, and summarizing whole trajectories, without requiring free-form reports. Fourteen leading vision-language models average only 29.3 percent accuracy, barely above chance, often describing intervals plausibly yet failing to enforce time constraints or combine evidence into globally consistent decisions.

Catalog record

CREATORSunghwan Steve Cho, Yunseok Han, Jaeyoung Do, AIDAS Laboratory, ECE & IPAI, Seoul National University {steve97, qicher, jaeyoung.do} @snu.ac.kr
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109664
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.15574

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays Sunghwan Steve Cho, Yunseok Han, Jaeyoung Do, AIDAS Laboratory, ECE & IPAI, Seoul National University {steve97, qicher, jaeyoung.do} @snu.ac.kr Abstract Longitudinal chest X-ray (CXR) interpretation requires reasoning over disease evolution across multiple patient visits, yet most existing medical VQA benchmarks focus on single images or short-horizon image pairs. We introduce MI-CXR , a benchmark for standardized evaluation of M ulti- I nterval longitudinal reasoning over multi-visit CXR sequences, without requiring free-form report generation or additional clinical context. MI-CXR comprises five-way multiple-choice questions over five-visit patient timelines and instantiates three complementary task families: Temporal Event Localization , Interval-wise Change Reasoning , and Global Trajectory Summarization , which assess clinically grounded visual reasoning over time. Evaluating 14 state-of-the-art vision–language models (VLMs) shows low overall performance (29.3% accuracy), only modestly above random guessing. Using stage-wise diagnostic probing, we find that models often produce locally plausible interval descriptions but fail to enforce temporal constraints or compose evidence into globally consistent decisions over the full timeline. These findings reveal key limitations of current VLMs and establish MI-CXR as a principled benchmark for longitudinal medical reasoning. The benchmark is available at https://github.com/AIDASLab/MI-CXR . † † footnotetext: † Corresponding author 1 Introduction Figure 1: Overview of longitudinal medical visual question answering and MI-CXR. Clinical image interpretation requires integrating evidence across multiple patient visits (top). We formalize longitudinal medical VQA into three core reasoning capabilities—Temporal Event Localization (TEL), Interval-wise Change Reasoning (ICR), and Global Trajectory Summarization (GTS)—and evaluate them over multi-visit CXR sequences using a diagnostic stage-wise decomposition (bottom). Despite rapid progress in vision–language models (VLMs) for medical image understanding, most existing evaluations adopt simplified problem formulations that diverge from real clinical workflows 25 ; 12 ; 37 . In chest X-ray (CXR) interpretation, diagnostic reasoning rarely relies on isolated images; instead, clinicians routinely compare examinations acquired across multiple visits to assess disease onset, progression, response to treatment, and recurrence 39 ; 1 ; 19 . However, current CXR medical benchmarks predominantly focus on restricted settings, such as single-image recognition 25 ; 14 or pairwise image comparison 37 ; 12 ; 58 . While these formulations capture important sub-problems, they fail to support many clinically meaningful questions that are inherently longitudinal , including when an abnormality first appears, whether it recurs after resolution, and how disease evolves over time 49 . A key challenge is that longitudinal interpretation imposes constraints absent in single-image or pairwise settings. Clinical decisions must remain globally consistent across temporally ordered visits, resolving mutually exclusive event hypotheses and composing local changes into coherent trajectory-level conclusions 2 ; 24 ; 55 ; 60 ; 20 . As a result, even when VLMs can describe local interval-level changes 46 ; 59 ; 44 ; 42 ; 29 , they may still fail at temporal diagnostic reasoning that requires structured decision-making over extended, dependent evidence. To address this mismatch, we formalize medical VQA as a problem of multi-interval longitudinal reasoning over multi-visit CXR sequences. Rather than treating longitudinal understanding as a straightforward extension of pairwise comparison, we decompose it into three core reasoning capabilities that naturally arise in clinical workflows and jointly stress different aspects of temporal reasoning (Figure 1 ). Specifically, Temporal Event Localization (TEL) requires identifying when clinically meaningful events—such as abnormality emergence, resolution, or recurrence—occur along the timeline, emphasizing decisive reasoning under temporal ordering and exclusivity constraints 57 ; 35 . Interval-wise Change Reasoning (ICR) focuses on interpreting visual changes between consecutive visits, isolating local interval-level perception that underlies longitudinal interpretation 15 . Global Trajectory Summarization (GTS) further requires integrating evidence across all visits to characterize the overall disease course, making decisions sensitive to cumulative context and error propagation 16 ; 48 . Based on this formulation, we introduce MI-CXR , a benchmark for evaluating M ulti- I nterval longitudinal reasoning over multi-visit CXR sequences. The benchmark consists of curated multi-visit patient timelines paired with questions that explicitly target the above reasoning capabilities (i.e., TEL, ICR, and GTS). Crucially, each question is constructed such that correct answers require aggregating information across multiple visits, rather than relying on cues from any single image or isolated image pair. This enables a principled and fine-grained assessment of whether models can reason over extended visual evidence in a clinically meaningful manner. Our evaluation under 14 state-of-the-art VLMs indicates that current VLMs remain far from reliable for longitudinal medical reasoning, with an average overall accuracy of only 29.3% across task categories. We also employ a stage-wise task decomposition that separates interval-level evidence articulation from final decision making, enabling a principled examination of how different task structures stress distinct aspects of model reasoning. Through this analysis, we show that while many models are capable of articulating local interval-level changes when appropriately prompted, they frequently fail to enforce exclusivity, bind events into ordered temporal structures, or compose interval-level observations into coherent global trajectories. These findings highlight a fundamental limitation of current VLMs: the bottleneck lies not only in visual perception, but also in structured temporal decision-making over extended sequences. In summary, our contributions are threefold: • We introduce MI-CXR , a benchmark that systematically evaluates Temporal Event Localization , Interval-wise Change Reasoning , and Global Trajectory Summarization over multi-interval CXR sequences. • We formalize longitudinal CXR interpretation as a global reasoning problem grounded in realistic clinical workflows, emphasizing temporally structured constraints (ordering, exclusivity, and trajectory-level consistency). • We present a stage-wise diagnostic framework that characterizes local and global reasoning failures in current VLMs, revealing a systematic gap where locally correct observations do not reliably yield correct longitudinal decisions. 2 Related works 2.1 Medical Visual Question Answering for Chest X-ray Medical VQA for chest X-ray images has been widely studied as a benchmark for multimodal understanding in clinical imaging 32 ; 63 ; 9 ; 5 ; 4 ; 8 . Early datasets such as VQA-RAD 25 and PathVQA 14 focus on single-image settings, evaluating snapshot-level recognition of abnormalities, anatomical structures, and image attributes. Subsequent work extends this paradigm to pairwise comparison settings, with benchmarks such as MIMIC-Diff-VQA 12 , MMXU 37 , and TemMed-Bench 58 targeting local changes between two visits. Beyond two-image settings, a few recent benchmarks have begun to incorporate multi-visit CXR data, though with substantially different objectives from longitudinal reasoning evaluation. For example, LUNGUAGE 36 focuses on report generation over image sequences, while CXReasonBench 27 introduces limited multi-timepoint question answering. However, these benchmarks are not designed to explicitly evaluate long-horizon longitudinal reasoning over temporally ordered visual evidence. Taken together, prior medical VQA benchmarks for chest X-rays fall short in evaluating long-horizon longitudinal reasoning: single-image and pairwise datasets are limited to short-term reasoning, while existing multi-visit benchmarks emphasize report generation or presence–absence judgments without probing temporal event ordering, recurrence, or resolution. Moreover, they do not disentangle different stages of temporal reasoning, making it difficult to analyze where longitudinal inference breaks down. 2.2 Longitudinal Modeling and Multi-visit Reasoning in Chest X-ray Recent studies have explored longitudinal modeling for CXR analysis by incorporating prior images, historical reports, and clinical context to improve diagnostic fidelity and report quality 62 ; 10 ; 17 ; 43 ; 64 . Many of these approaches focus on radiology report generation, such as PriorRG 33 ,

Read the complete text