Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation
Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei · 2026
Read Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation by Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11533.txt.
Introduction
Xiang and six colleagues introduce C2A, a dataset for turning multimodal clinical check-up reports into Action Cards for patients. Routine reports combine labs, body checks, imaging and visual information, but rarely say what to do next. C2A holds 2,000 scrubbed reports, each card naming one issue, its priority, department, follow-up window, explanation and questions for clinicians. Scoring covers coverage, precision, priority consistency, department and timing accuracy, complexity, usefulness, readability and safety. No model leads on every dimension, though experts judged 84 percent of outputs fully sound.
Catalog record
| CREATOR | Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39854 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11533
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation Sike Xiang, Shuang Chen, Kevin Qinghong Lin, Jialin Yu, Yijia Sun, Philip Torr, Amir Atapour-Abarghouei Abstract Clinical check-up reports are multimodal documents that combine page layouts, tables, numerical biomarkers, abnormality flags, imaging findings, and domain-specific terminology. Such heterogeneous evidence is difficult for laypersons to interpret and translate into concrete follow-up actions. Although large language models show promise in medical summarisation and triage support, their ability to generate safe, prioritised, and patient-oriented actions from multimodal check-up reports remains under-benchma-rked. We present Checkup2Action , a multimodal clinical check-up report dataset and benchmark for structured Action Card generation. Each card describes one clinically relevant issue and specifies its priority, recommended department, follow-up time window, patient-facing explanation, and questions for clinicians, while avoiding diagnostic or treatment-prescriptive claims. The dataset contains 2,000 de-identified real-world check-up reports covering demographic information, physical examinations, laboratory tests, cardiovascular assessments, and imaging-related evidence. We formulate checkup-to-action generation as a constrained structured generation task and introduce an evaluation protocol covering issue coverage and precision, priority consistency, department and time recommendation accuracy, action complexity, usefulness, readability, and safety compliance. Experiments with general-purpose and medical large language models reveal clear trade-offs between issue coverage, action correctness, conciseness, and safety alignment. Checkup2Action provides a new multimodal benchmark for evaluating patient-oriented reasoning over clinical check-up reports. BMVC Author Guidelines † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † affiliation: Durham University Durham, UK † † affiliation: University of Oxford Oxford, UK 1 Introduction Routine clinical check-ups generate multimodal reports that combine visual document layouts, tabular laboratory results, numerical biomarkers, abnormality flags, specialised symbols, and imaging-related findings. Unlike single-section clinical notes, these reports are often multi-page documents in which evidence is distributed across heterogeneous regions, including tables, structured examination blocks, scanned pages, and imaging summaries. For laypersons, such dense and visually structured clinical artefacts are difficult to interpret: abnormal values, reference ranges, arrows, and templated medical phrases must be read together before a meaningful follow-up decision can be made. This creates a substantial “interpretability gap” between multimodal check-up evidence and concrete patient action [ 12 , 25 , 16 , 21 ] . Figure 1: Real-world motivation for Checkup2Action. Patients often receive multimodal clinical check-up reports containing structured measurements, abnormality flags, and imaging-related findings, but may struggle to decide what to do next. Checkup2Action converts such reports into prioritised, patient-facing Action Cards that support appropriate follow-up consultations and concrete next steps. In real-world practice, patients often read check-up reports by scanning visually salient cues, such as “abnormal” labels, \uparrow/\downarrow symbols, positive test indicators, highlighted reference-range violations, or concluding phrases in imaging and laboratory sections. However, these cues are distributed across multimodal report regions and do not by themselves indicate which findings are clinically important, which deviations are mild, or what follow-up action should be taken [ 19 ] . Prior work shows that people with limited health literacy struggle to interpret heterogeneous clinical evidence, including laboratory values, radiology conclusions, and phrases such as “correlate clinically” or “mildly elevated”, which can lead either to disproportionate anxiety around isolated cues or to missed high-risk findings that require intervention [ 14 , 21 ] . Even when reports contain an “overall conclusion” or “summary opinion”, these sections are usually descriptive rather than action-oriented, leaving patients uncertain about priority, department referral, or follow-up timing and often dependent on further clarification [ 20 ] . This motivates a benchmark for converting multimodal check-up reports into structured, patient-facing next-step plans. Recent advances in large language models and conversational AI have enabled clinical summarisation, patient-facing report simplification, and triage assistance [ 21 , 15 ] . However, most existing systems still produce free-form explanations or acuity labels rather than structured, patient-oriented action plans. They rarely evaluate whether a model can organise multimodal check-up evidence into explicit priorities, recommended departments, follow-up time windows, and concrete questions for clinicians [ 3 ] . On the evaluation side, prior work often relies on text similarity metrics or small-scale expert ratings, leaving open whether systems consistently identify clinically relevant issues, rank them appropriately, and provide safe next-step guidance for patients [ 23 ] . In this context, we construct the C2A (Checkup2Action) benchmark dataset (Section 3 ) for multimodal check-up-to-action generation. The dataset contains de-identified real-world check-up reports and supports evaluation of whether a system can transform visually structured, multi-section clinical evidence into prioritised patient-facing Action Cards. We further introduce an evaluation framework that combines structured metrics (problem recall, priority accuracy, department accuracy, time accuracy, and action complexity bias) with human-centred ratings (problem relevance, safety, usefulness, clarity, and tone), yielding ten complementary metrics that capture both system performance and user-perceived quality. Built on this benchmark, we instantiate Checkup2Action (Section 4 ) as a constrained baseline workflow for generating structured “Action Cards”. Each card focuses on a single issue and specifies its priority, recommended department, suggested follow-up time window, patient-facing explanation, and questions to ask a clinician. Figure 1 illustrates the real-world workflow: after a check-up, the patient receives a report, and Checkup2Action converts it into action cards that help the patient prepare appropriate next steps. We deliberately constrain the system’s scope to interpretation and action planning, organising existing findings into follow-up recommendations without issuing new diagnostic labels or medication plans. Our primary contributions are thus as follows: (i) We introduce C2A , a real-world multimodal clinical check-up report dataset and benchmark containing 2,000 de-identified reports with expert annotations for patient-oriented Action Card generation. (ii) We formulate checkup-to-action generation as a structured multimodal report understanding task and provide an evaluation protocol that jointly measures issue coverage and precision, prioritisation consistency, department and follow-up time recommendation quality, action complexity, usefulness, readability, and safety compliance. (iii) We instantiate Checkup2Action , a constrained baseline workflow that converts multi-section check-up evidence into ordered, patient-facing Action Cards while avoiding diagnostic and treatment-prescriptive outputs. 2 Related Work We review related work on multimodal health check-up report understanding and datasets (Section 2.1 ), followed by medical AI agents for clinical summarisation, patient-facing communication, and triage support (Section 2.2 ). 2.1 Health Check-up Reports and Related Datasets Routine health check-ups typically include multiple examination types, such as vital signs, laboratory tests, functional tests, and imaging or ultrasound examinations. Although check-up packages vary across settings, they commonly centre on cardiometabolic risk indicators such as blood pressure, cholesterol, adiposity measures, and, where appropriate, glucose-related testing [ 1 , 6 ] . Their reports are usually multimodal clinical documents: numeric tables, reference ranges, abnormality flags, structured examination blocks, imaging-related summaries, and free-text conclusions are arranged across visually distinct report sections [ 25 ] . In laboratory and imaging sections, structured templates and standardised terminology can improve documentation consistency, but they also create comprehension barriers for non-professional users [ 5 ] . Patients and health care providers further report that access to test results through web portals often requires additional explanation and guidance to support appropriate follow-up actions [ 16 ] . Simply presenting numerical results with reference ranges does not ensure interpretability, and the limitations of reference intervals can lead to confusion or misinterpretation [ 24