Applications of Large Models in Medicine
YunHe Su, Zhengyang Lu, Junhui Liu, Ke Pang, Haoran Dai, Sa Liu, Yuxin Jia, Lujia Ge, Jing-min Yang * · 2025
Read Applications of Large Models in Medicine by YunHe Su, Zhengyang Lu, Junhui Liu, Ke Pang, Haoran Dai, Sa Liu, Yuxin Jia, Lujia Ge, Jing-min Yang * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17132.txt.
Introduction
YunHe Su and co-authors survey large AI models in medicine in 2025. Large models span text, vision, three dimensional, and mixed forms. The survey maps how each kind is reshaping care. Uses include disease prediction, diagnostic help, tailored treatment plans, and drug discovery.
Graph based large models earn a special note for mapping biomedical links. Vision language and three dimensional models aid scan reading, body modeling, and prosthetic design. The authors also flag hurdles that slow clinic uptake. The field is setting new marks for accuracy while still proving safety at the bedside.
Catalog record
| CREATOR | YunHe Su, Zhengyang Lu, Junhui Liu, Ke Pang, Haoran Dai, Sa Liu, Yuxin Jia, Lujia Ge, Jing-min Yang * |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-17132
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Applications of Large Models in Medicine YunHe Su, Zhengyang Lu, Junhui Liu, Ke Pang, Haoran Dai, Sa Liu, Yuxin Jia, Lujia Ge, Jing-min Yang * Abstract This paper explores the advancements and applications of large-scale models in the medical field, with a particular focus on Medical Large Models (MedLMs). These models, encompassing Large Language Models (LLMs), Vision Models, 3D Large Models, and Multimodal Models, are revolutionizing healthcare by enhancing disease prediction, diagnostic assistance, personalized treatment planning, and drug discovery. The integration of graph neural networks in medical knowledge graphs and drug discovery highlights the potential of Large Graph Models (LGMs) in understanding complex biomedical relationships. The study also emphasizes the transformative role of Vision-Language Models (VLMs) and 3D Large Models in medical image analysis, anatomical modeling, and prosthetic design. Despite the challenges, these technologies are setting new benchmarks in medical innovation, improving diagnostic accuracy, and paving the way for personalized healthcare solutions. This paper aims to provide a comprehensive overview of the current state and future directions of large models in medicine, underscoring their significance in advancing global health. 1 The First Clinical Medical School, MuDanJiang Medical University, Heilongjiang, China [email protected] 2 School of Design, Jiangnan University, Wuxi, China [email protected] 3 Kunming, Yunnan, China [email protected] 4 Department of Anesthesiology, Third Xiangya Hospital, Central South University, Changsha, Hunan, China [email protected] 5 Nanjing Medical University, Nanjing, China [email protected] 6 University of California, Davis, California, USA [email protected] 7 Brown School, Washington University in St. Louis, Louis, Missouri, USA [email protected] 8 School of Basic Medical Sciences, Capital Medical University, Beijing, China [email protected] Correspondence: Jing-min Yang, WestChina Biomedical Big Data Center, WestChina Hospital, Sichuan University, Chengdu, China [email protected] † These authors contributed equally to this work. Keywords: Medical Models, Large Language Models, Vision Models, Multimodal Models, Drug Discovery Figure 1 : The overall structure of the survey. 1 Introduction Medical Large Models (MedLMs) refer to a class of large-scale artificial intelligence models specifically trained to handle and analyze various types of medical-related data, such as clinical text [ 98 ] , imaging data and genetic information. They are typically based on deep learning and neural network technologies, enabling them to perform a variety of tasks in the medical field, including disease prediction, diagnostic assistance, personalized treatment planning, and drug development. The core advantage of MedLMs lies in their powerful data processing capabilities and their ability to learn from vast amounts of data. MedLMs can be categorized into distinct types, each serving different applications based on the data they handle. Large Language Models (LLMs) are designed primarily for processing clinical textual data (like MedPaLM), such as electronic health records (EHRs) such as [ 143 , 142 , 141 , 65 ] . These models excel at extracting pertinent information from a wide range of medical texts, including patient histories, symptoms, and treatment instructions. This capability reduces the manual workload for healthcare professionals and provides crucial decision-making support. Furthermore, LLMs are instrumental in generating clinical pathways, assisting in the creation of personalized treatment plans by analyzing a vast corpus of medical literature and integrating the latest clinical guidelines [ 126 ] with practical clinical validations [ 91 , 17 ] . Another major category is Vision Models, which primarily deal with medical imaging data. These models, typically based on convolutional neural networks (CNNs), have proven effective in tasks such as detecting early-stage cancers through the analysis of medical images. For example, CNNs have been applied to detect skin cancer with dermatologist-level accuracy and are also used in the detection of lung and breast cancers, providing physicians with fast and reliable diagnostic support. The ability to detect even minute abnormalities in images, with accuracy comparable to that of experienced specialists, has made vision models a powerful tool in modern medical practice [ 29 ] . 3D Large Models, which focus on volumetric data analysis, represent another critical category within MedLMs. These models utilize 3D convolutional neural networks (CNNs) to handle medical images in three-dimensional formats, such as CT or MRI scans. By accurately segmenting tumors and analyzing their spatial characteristics, 3D models assist in determining tumor locations and volumes, essential for surgical planning. Additionally, these models contribute to virtual surgery simulations, allowing medical practitioners to plan surgeries more effectively and mitigate potential risks . The integration of 3D models into clinical practice is enhancing both the precision of tumor detection and the planning of surgical interventions [ 139 ] . In addition to these single-modal models, Multimodal Models integrate multiple types of data, such as clinical text, imaging, and genomic data, to provide a more comprehensive understanding of a patient’s condition. By combining diverse data sources, these models enhance diagnostic accuracy and the ability to develop personalized treatment plans. For instance, multimodal models have been successfully used to improve the early diagnosis of lung cancer by combining CT images with clinical records, thus providing more accurate diagnostic insights . These models also play a crucial role in personalizing treatment plans for complex conditions, such as breast cancer, by integrating genomic data, imaging, and clinical histories [ 129 ] . Graph Large Models (Graph Neural Networks, GNNs) are another key type of MedLM, particularly in the field of genomics. GNNs are designed to analyze the relationships between genes, diseases, and therapeutic targets. By studying gene interaction networks, GNNs can predict disease risk factors and identify potential biomarkers, offering novel insights into early diagnosis and potential treatment options. Graph Large Models leveraging transformers with large parameters and trained with large biomedical datasets. These models have shown great promise in areas such as cancer risk prediction, where they analyze the complex interactions between genes and disease. Medical large models (MedLMs) are bringing new possibilities to healthcare. They offer applications in disease prediction, diagnostics, treatment planning, and drug development. These models analyze extensive medical data and identify patterns that may indicate disease risks. For example, genetic interaction analysis can reveal potential cancer risks, supporting early preventive strategies [ 125 ] . MedLMs also contribute to improving diagnostic processes. By analyzing imaging data, they assist in identifying abnormalities that might be missed in manual reviews. Some models trained on dermatological imaging have been shown to classify skin conditions with a high degree of accuracy, providing clinicians with additional tools for decision-making. In treatment planning, MedLMs combine imaging, genomic, and historical patient data. This combination supports the development of personalized strategies for managing complex conditions. For diseases such as breast cancer, integrating these data sources has been associated with better-aligned treatment options for patients. Drug discovery is another area where MedLMs show promise. They assist in predicting protein structures, which is a critical step in therapeutic development. Tools like AlphaFold have been used to reduce the time and effort required for molecular design, helping researchers identify drug candidates more efficiently [ 55 ] . Medical large models (MedLMs) are experiencing widespread adoption across healthcare systems globally. Their utilization is increasing rapidly, particularly in areas such as diagnostic assistance, disease prediction, personalized treatment planning, and drug discovery. These models have proven their value in enhancing the accuracy and efficiency of medical practices. Notably, many healthcare platforms have now integrated these models as standard tools, embedding them deeply into their operations. For instance, Baidu’s Lingyi Medical