MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging
Mahmoud Soliman a , Islam Osman a , Mohamed S. Shehata a and Rasika Rajapakshe a,b,c · 2025
Read MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging by Mahmoud Soliman a , Islam Osman a , Mohamed S. Shehata a and Rasika Rajapakshe a,b,c (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-04016.txt.
Introduction
Mahmoud Soliman, Islam Osman, Mohamed S Shehata, and Rasika Rajapakshe present MedDChest, a foundation vision transformer pretrained specifically for thoracic imaging rather than adapted from natural images. Fine-tuning backbones trained on everyday photos leaves a domain gap that limits medical performance, the team argue.
Pretraining from scratch used over one point two million chest images, X-ray and computed tomography, gathered from ten public sources. A guided cropping strategy biases training samples toward anatomically relevant regions instead of random crops. Fine-tuned on varied diagnostic tasks, MedDChest beat strong publicly available ImageNet-pretrained models clearly. Large in-domain pretraining with domain-aware augmentation gives a better starting point for chest imaging work.
Catalog record
| CREATOR | Mahmoud Soliman a , Islam Osman a , Mohamed S. Shehata a and Rasika Rajapakshe a,b,c |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-04016
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging Mahmoud Soliman a , Islam Osman a , Mohamed S. Shehata a and Rasika Rajapakshe a,b,c Abstract The performance of vision models in medical imaging is often hindered by the prevailing paradigm of fine-tuning backbones pre-trained on out-of-domain natural images. To address this fundamental domain gap, we propose MedDChest, a new foundational Vision Transformer (ViT) model optimized specifically for thoracic imaging. We pre-trained MedDChest from scratch on a massive, curated, multimodal dataset of over 1.2 million images, encompassing different modalities including Chest X-ray and Computed Tomography (CT) compiled from 10 public sources. A core technical contribution of our work is Guided Random Resized Crops, a novel content-aware data augmentation strategy that biases sampling towards anatomically relevant regions, overcoming the inefficiency of standard cropping techniques on medical scans. We validate our model’s effectiveness by fine-tuning it on a diverse set of downstream diagnostic tasks. Comprehensive experiments empirically demonstrate that MedDChest significantly outperforms strong, publicly available ImageNet-pretrained models. By establishing the superiority of large-scale, in-domain pre-training combined with domain-specific data augmentation, MedDChest provides a powerful and robust feature extractor that serves as a significantly better starting point for a wide array of thoracic diagnostic tasks. The model weights will be made publicly available to foster future research and applications. keywords Foundational Models; Medical Imaging; Self-Supervised Learning; Vision Transformer; Data Augmentation † † articletype: RESEARCH ARTICLE † † affiliation: a Department of Computer Science, Mathematics, Physics and Statistics, The University of British Columbia, Kelowna, BC, Canada; b Medical Physics, BC Cancer, Kelowna, BC, Canada; c Department of Surgery, The University of British Columbia, Vancouver, BC, Canada 1 Introduction Deep learning has become a cornerstone of modern medical image analysis, with Vision Transformers (ViTs) [ 11 ] pushing the boundaries of what is possible by capturing long-range dependencies within images [ 35 ] . The dominant paradigm involves fine-tuning models pre-trained on large-scale, out-of-domain datasets like ImageNet [ 9 ] . However, this approach is fundamentally limited by the significant domain gap between natural and medical images. Natural images are typically 3-channel (RGB) photographs, whereas thoracic scans are often single-channel grayscale representations of complex anatomy. This discrepancy in feature distribution and color space forces the model to expend significant capacity to adapt, limiting its ultimate performance on downstream medical tasks. We posit that pre-training from scratch on a massive, in-domain dataset is a more effective strategy for building truly foundational models for medicine. To address this challenge, we introduce MedDChest , a new foundational ViT model specifically optimized for thoracic imaging. Our central thesis is that large-scale, self-supervised pre-training on in-domain data provides a superior feature representation. To build MedDChest, we aggregated a massive, multimodal dataset of over 1.2 million images from 10 public sources, encompassing Chest X-rays and CT scans. This large-scale effort, similar in spirit to recent work [ 14 ] , allows MedDChest to learn a rich visual vocabulary directly relevant to thoracic anatomy. Our most significant technical contribution is a novel data augmentation strategy, Guided Random Resized Crop, designed to solve a critical flaw in applying standard self-supervised learning (SSL) to medical scans. Methods like Masked Autoencoders (MAE) [ 16 ] are inefficient on medical images, which often contain large, non-informative backgrounds. Our two-stage algorithm first identifies the content’s bounding box and then biases the crop sampling process towards this region, dramatically improving the efficiency and quality of pre-training. This paper introduces MedDChest, a powerful foundational model for thoracic imaging, and details the large-scale dataset curated for its pre-training. We present our novel content-aware augmentation technique, Guided Random Resized Crop, and provide rigorous empirical validation of its effectiveness. By demonstrating significant performance gains over strong ImageNet-pretrained baselines, we establish MedDChest as a superior starting point for downstream tasks, and we will release our model weights to facilitate future research. 2 Related Work Our research builds upon established work in transfer learning, self-supervised learning (SSL), and the application of Vision Transformers (ViTs) to medicine. While fine-tuning ImageNet-pretrained models like ResNet [ 17 ] and DenseNet [ 18 ] has been highly successful on medical tasks [ 13 , 33 ] , the domain gap between natural and medical images remains a known limitation [ 32 , 3 ] . This has motivated a shift towards creating foundational models pre-trained directly on in-domain medical data, primarily through SSL. SSL allows models to learn from vast unlabeled datasets, which is critical in medicine. While early contrastive methods like SimCLR [ 7 ] were effective, recent work has favored masked image modeling approaches like Masked Autoencoders (MAE) [ 16 ] , especially for ViTs [ 11 ] . The success of this paradigm has spurred large-scale medical pre-training initiatives. For instance, RadImageNet [ 29 ] aggregated millions of radiology images for pre-training CNNs, while MedMAE [ 14 ] proposed a generalist backbone for diverse medical tasks, both validating the benefits of in-domain pre-training. Our work is distinct from these efforts in two crucial ways. First, we focus specifically on the thoracic region to develop a more potent, specialized feature set. Second, and most importantly, we identify and solve a key methodological flaw in applying SSL to medical scans. Standard augmentation like Random Resized Crop is inefficient for medical images, which often contain large empty backgrounds. While some have used computationally heavy saliency maps to guide cropping, our work introduces an efficient, algorithm-driven solution, Guided Random Resized Crop . By performing a rapid content analysis before sampling, we ensure the model consistently learns from relevant anatomy, thereby maximizing the effectiveness of the pre-training process. 3 Methodology Input Medical Image Guided Multi-Crop Augmentation Figure 1: Multi-crop augmentation process for medical image analysis. The input medical image is processed to generate multiple views: 2 global crops at higher resolution and 8 local crops at lower resolution for comprehensive feature extraction. Local Crop Global Crop Student ViT Encoder Teacher ViT Encoder Student Projection Head Teacher Projection Head (No Gradient Updates) (Gradient Updates) Total Loss (DINO + iBOT) MedD Architecture Figure 2: DINOv2 Self-Supervised Learning Architecture in MedD. The asymmetric data augmentation strategy feeds global crops to both student and teacher networks, while local crops are only fed to the student network. The teacher network is updated via exponential moving average (EMA) and receives no gradient updates. Our methodology is designed to build a powerful foundational model, MedDChest, through a meticulously adapted self-supervised learning pipeline tailored to the unique challenges of medical imaging. This section provides a comprehensive account of our approach. We first outline the advanced self-supervised framework that serves as the basis of our work. We then introduce our primary technical contribution, a domain-specific data augmentation strategy named Content-Guided Multi-Crop Augmentation. Finally, we describe the large-scale dataset curated for pre-training and our implementation and evaluation protocols. 3.1 Self-Supervised Pre-training Framework We select the DINOv2 [ 30 ] framework as the foundation for our pre-training regimen, chosen for its demonstrated ability to learn robust, general-purpose visual features without supervision. The framework’s architecture, illustrated in Figure 2 , integrates a student-teacher learning paradigm with a compound loss function derived from the principles of both DINO [ 6 ] and iBOT [ 43 ] . 3.1.1 Student-Teacher Architecture The learning