Google-MedGemma Based Abnormality Detection in Musculoskeletal radiographs
Soumyajit Maity, Pranjal Kamboj, Sneha Maity, Rajat Singh, Sankhadeep Chatterjee · 2025
Read Google-MedGemma Based Abnormality Detection in Musculoskeletal radiographs by Soumyajit Maity, Pranjal Kamboj, Sneha Maity, Rajat Singh, Sankhadeep Chatterjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05600.txt.
Introduction
Soumyajit Maity and colleagues use a medical foundation model from Google to spot abnormalities in bone and joint X-rays. Their system builds on MedGemma, whose vision encoder was pretrained on many kinds of medical images. X-rays are first cleaned, then turned into rich feature vectors by that encoder.
A small neural classifier reads those vectors and labels each image normal or abnormal. In tests the system beat older convolutional and autoencoder pipelines. The authors credit the foundation model's transfer learning for stronger generalization and simpler feature work. They suggest such systems could speed up clinical triage of radiographs and extend to other medical image tasks.
Catalog record
| CREATOR | Soumyajit Maity, Pranjal Kamboj, Sneha Maity, Rajat Singh, Sankhadeep Chatterjee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05600
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Google-MedGemma Based Abnormality Detection in Musculoskeletal radiographs Soumyajit Maity, Pranjal Kamboj, Sneha Maity, Rajat Singh, Sankhadeep Chatterjee Abstract This paper proposed a MedGemma based framework for automatic abnormality detection in musculoskeletal radiographs. Departing from conventional autoencoder and neural network based pipelines, the proposed method leverages the MedGemma foundation model, incorporating a SigLIP derived vision encoder pretrained on diverse medical imaging modalities. Preprocessed X-ray images are encoded into high-dimensional embeddings using the MedGemma vision backbone, which are subsequently passed through a lightweight multilayer perceptron for binary classification. The experimental assessment reveals that the MedGemma driven classifier exhibits strong performance, exceeding conventional, convolutional and autoencoder based performance metrics. Additionally, the model leverages MedGemma’s transfer learning capabilities, enhancing generalization and optimizing the feature engineering process. The integration of a modern medical foundation model not only enhances representation learning but also facilitates modular training strategies, such as selective encoder block unfreezing for efficient domain adaptation. The findings suggest that MedGemma powered classification systems can advance clinical radiograph triage by providing scalable and accurate abnormality detection, which has potential for a broader application in automated medical image analysis. 1 Introduction Automated abnormality detection in musculoskeletal radiographs is a pivotal challenge in the advancement of computer-aided diagnosis systems. [ 1 ] The fast growth of large, annotated datasets, like MURA [ 2 ] , which has more than 40,000 upper extremity X-ray studies, has helped this field move forward. However, making strong and generalizable models is still hard because of things like image diversity, subtle diagnostic cues, and a big class imbalance. Convolutional architectures or autoencoder frameworks are often used in traditional methods to get discriminative features from medical images. These methods work, but they usually need a lot of network engineering and labeled data to work best. Recent advancements in medical artificial intelligence have highlighted the promise of foundation models large, pretrained networks optimized on diverse medical imaging [ 3 ] corpora for downstream diagnostic tasks. MedGemma [ 4 ] exemplifies this new generation of models, integrating a SigLIP [ 5 ] derived vision encoder that is proficient in extracting domain-specific representations from a range of biomedical modalities. By harnessing the transfer learning capabilities inherent in MedGemma, it becomes feasible to efficiently adapt model weights to domain-specific challenges with minimal fine-tuning. This work proposes a MedGemma based classification pipeline for abnormality detection in the MURA dataset, replacing bespoke feature engineering and standalone encoders with a unified, pretrained backbone. The approach aims not only to streamline the training process, but also to enhance diagnostic accuracy. This study presents the implementation, and evaluation of this methodology, establishing MedGemma’s applicability for musculoskeletal abnormality detection and outlining directions for its broader impact in automated radiology. 2 Literature Review Machine learning has come a long way in the field of medical image analysis, with a growing focus on automated diagnostic support for radiology. In the past, most methods used handcrafted features [ 6 ] and traditional classifiers [ 7 ] . These didn’t work very well because clinical imaging is very complex and changes a lot. With the advent deep learning, particularly convolutional neural networks (CNNs) [ 8 ] , this field has changed a lot. CNNs can now directly extract hierarchical and discriminative image representations from pixel data. The authors [ 9 ] proposed a CNN based pneumonia detection from chest X-rays with data augmentation, transfer learning with optimized network complexity and targeted layer retraining outperformed SVMs and capsule networks on labeled datasets. This study [ 10 ] utilizes deep learning to show that transfer-learned CNNs combined with early fusion multiview features and downstream SVM/KNN classifiers attain enhanced accuracy in classifying COVID-19, viral pneumonia, and normal lungs from medical images.But these models usually need a lot of network design, big annotated datasets, and careful tuning to work with different anatomical sites and imaging protocols. To fix the problems that come up when there are too many classes in medical data, techniques like sampling techniques [ 11 ] and cost-sensitive loss functions [ 12 ] have been used. However, finding the best calibration is still a problem. Newer studies have looked into unsupervised and self-supervised frameworks, such as autoencoders. In this paper [ 13 ] , they represent a Variational Autoencoder driven pipeline that maps chest X - ray images to a latent space, applies established resampling to correct severe class imbalance, and then trains conventional classifiers to detect COVID - 19, confirming significant performance gains and methodological robustness. This work proposes ViT - AE++ [ 14 ] , a self - supervised ViT autoencoder that augments masked - patch pretraining with a structure - aware reconstruction loss and a contrastive objective from dual masked views, and extends the framework to 3D volumes, yielding consistently stronger representations on natural and medical images. Here the authors introduces SDDA - MAE [ 15 ] , which directly pre - trains and fine - tunes on target medical datasets using a Dual Attention Transformer backbone and a self - distillation scheme that transfers global context from the decoder to the encoder, yielding state - of - the - art segmentation and classification performance without large upstream pretraining. The emergence of foundation models, large-scale neural networks pre-trained on heterogeneous medical image corpora promises a paradigm shift for medical AI. This paper depicts BiomedCLIP [ 16 ] , pretrained on the extensive PMC - 15M corpus of 15 million image–text pairs extracted from 4.4 million biomedical articles, demonstrates the power of domain specific multimodal pretraining for generalist biomedical AI. The authors here [ 17 ] , unified visual and textual reasoning, visual–language foundation models such as Med - Flamingo and LLaVA - Med promise improved diagnosis, reporting, and longitudinal monitoring results into trustworthy multimodal healthcare AI. This study [ 18 ] introduces a multimodal contrastive learning framework that aligns 3D brain MRI with radiology reports and incorporates tumor - location cues to learn clinically grounded representations, yielding substantially improved explainability—evidenced by a 31.1% Dice overlap between attention maps and manual segmentations—and 87.7% test accuracy for pediatric low - grade glioma genetic marker classification, thereby supporting integration into radiology workflows. Although CNNs and autoencoder based pipelines have established the groundwork for automated abnormality classification, the incorporation of medical AI foundation models provides unique benefits in scalability and adaptability. These developments set the groundwork for the methodology presented in this work, wherein MedGemma’s pretrained medical vision encoder is harnessed for robust, efficient radiograph abnormality detection. 3 Proposed Method In this work we have proposes a MedGemma based abnormality classification pipeline for musculoskeletal radiographs that replaces bespoke encoders and handcrafted features with a unified, medically pretrained vision backbone and a lightweight classification head. The design leverages a SigLIP derived medical vision encoder from the MedGemma family to obtain pooled image embeddings, which are subsequently fine-tuned with a small multilayer perceptron (MLP) head under a selective unfreezing regime. The methodology is tailored to the MURA benchmark and emphasizes rigorous data handling, calibrated optimization, and clinically aligned evaluation to ensure both accuracy and robustness for radiograph triage scenarios. 3.1 Dataset The dataset we have used for this experiment is MURA(musculoskeletal radiographs) [ 19 ] , it includes 14,863 studies from 12,173 individuals, totaling 40,561 upper-extremity radiographs that clinical specialists have