A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications
Hongyuan Zhang, Yuheng Wu, Mingyang Zhao, Zhiwei Chen, Rebecca Li, Fei Zhu, Haohan Zhao, Xiaohua Yuan, Meng Yang, Chunli Qiu, Xiang Cong, Haiyan Chen, Lina Luan, Randolph H.L. Wong, Huai Liao, Colin A Graham, Shi Chang, Guowei Tao, Dong Yi, Zhen Lei, Nassir Navab, Sebastien Ourselin, Jiebo Luo, Hong · 2025
Read A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications by Hongyuan Zhang, Yuheng Wu, Mingyang Zhao, Zhiwei Chen, Rebecca Li, Fei Zhu, Haohan Zhao, Xiaohua Yuan, Meng Yang, Chunli Qiu, Xiang Cong, Haiyan Chen, Lina Luan, Randolph H.L. Wong, Huai Liao, Colin A Graham, Shi Chang, Guowei Tao, Dong Yi, Zhen Lei, Nassir Navab, Sebastien Ourselin, Jiebo Luo, Hong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11752.txt.
Introduction
Hongyuan Zhang and twenty-four co-authors release a foundation model for medical ultrasound in this 2025 biomedical AI paper. Ultrasound AI has been held back by small labeled datasets and by models that each solve one narrow task, so the team trained a generalist on a very large pooled collection.
Their dataset gathers four and a half million ultrasound images from more than twenty-three countries on five continents, across many devices and populations, and the model, EchoCare, learns from it by self-supervision. A hierarchical classifier lets it learn pixel-level detail and whole-image representation together, capturing anatomy at both scales. With minimal extra training it beat state-of-the-art rivals across ten benchmarks covering diagnosis, segmentation, organ detection, landmarks, measurement, enhancement, and report writing. Code and the pretrained model are fully public.
Catalog record
| CREATOR | Hongyuan Zhang, Yuheng Wu, Mingyang Zhao, Zhiwei Chen, Rebecca Li, Fei Zhu, Haohan Zhao, Xiaohua Yuan, Meng Yang, Chunli Qiu, Xiang Cong, Haiyan Chen, Lina Luan, Randolph H.L. Wong, Huai Liao, Colin A Graham, Shi Chang, Guowei Tao, Dong Yi, Zhen Lei, Nassir Navab, Sebastien Ourselin, Jiebo Luo, Hong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-11752
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications Hongyuan Zhang, Yuheng Wu, Mingyang Zhao, Zhiwei Chen, Rebecca Li, Fei Zhu, Haohan Zhao, Xiaohua Yuan, Meng Yang, Chunli Qiu, Xiang Cong, Haiyan Chen, Lina Luan, Randolph H.L. Wong, Huai Liao, Colin A Graham, Shi Chang, Guowei Tao, Dong Yi, Zhen Lei, Nassir Navab, Sebastien Ourselin, Jiebo Luo, Hongbin Liu, Gaofeng Meng Abstract Abstract. The inherent safety and versatility of ultrasound imaging have made it widely accessible in modern clinical settings for disease diagnosis and health management. Artificial intelligence (AI) that can effectively learn ultrasound representations by integrating multi-source data holds significant promise for advancing clinical care. However, the scarcity of large labeled datasets in real-world clinical environments and the limited generalizability of task-specific models have hindered the development of generalizable clinical AI models for ultrasound applications. In this study, we present EchoCare, a novel ultrasound foundation model for generalist clinical use, developed via self-supervised learning on our curated, publicly available, large-scale dataset EchoCareData. EchoCareData comprises 4.5 million ultrasound images, sourced from over 23 countries across 5 continents and acquired via a diverse range of distinct imaging devices, thus encompassing global cohorts that are multi-center, multi-device, and multi-ethnic. Unlike prior studies that adopt off-the-shelf vision foundation model architectures, we introduce a hierarchical classifier into EchoCare to enable joint learning of pixel-level and representation-level features, capturing both global anatomical contexts and local ultrasound characteristics. With minimal training, EchoCare outperforms state-of-the-art comparison models across 10 representative ultrasound benchmarks of varying diagnostic difficulties, spanning disease diagnosis, lesion segmentation, organ detection, landmark prediction, quantitative regression, imaging enhancement and report generation. The code and pretrained model are publicly released, rendering EchoCare accessible for fine-tuning and local adaptation, supporting extensibility to additional applications. EchoCare provides a fully open and generalizable foundation model to boost the development of AI technologies for diverse clinical ultrasound applications. Contents 1 Introduction Ultrasound imaging stands as a cornerstone of modern medicine, celebrated for its unique combination of real-time assessment, cost-effectiveness, and inherent safety. This non-invasive and radiation-free modality allows for the dynamic visualization of physiological processes, securing its indispensable role in a wide range of clinical practices [ 28 ] . Despite these advantages, ultrasound diagnostic is heavily reliant on the skill of the sonographer and the specialized expertise required to interpret the complex, often subtle, visual information. This inherent complexity, coupled with the ubiquity and versatility of ultrasound, has spurred significant interest in leveraging artificial intelligence (AI) to advance its use. As ultrasound imaging expands to new anatomical regions and clinical applications, there is a growing demand for versatile and generalizable AI models that can adapt to diverse clinical tasks and organs with minimal reliance on new labeled data. Meeting this demand will not only broaden the application of ultrasound analysis but also accelerate the deployment of smart healthcare solutions, making high-quality diagnostics more accessible and efficient. Recent advances in foundation models (FM) using self-supervised learning have opened new frontiers in medical AI [ 36 , 33 , 8 , 27 , 20 , 21 ] . These models learn general-purpose feature representations directly from raw data, eliminating dependence on extensive expert annotations. Upon completion of pretraining, these models can be effectively adapted to a wide array of downstream clinical tasks with minimal or no additional fine-tuning. This paradigm represents a significant advantage over conventional medical AI approaches, which are typically limited to specific anatomical structures or require extensive retraining when adapted to each new clinical application. However, pretraining of foundation models requires large-scale and diverse datasets, making data acquisition and rigorous curation essential for developing clinically reliable and generalizable systems. Building on the success of vision foundation models, researchers have started adapting these approaches to ultrasound imaging analysis [ 15 , 16 , 31 ] . Although initial results show promise, several critical challenges could limit their potential clinical impact. First, the scale of available ultrasound datasets remains relatively small, undermining the reliability of models for clinical deployment. Moreover, much of the pretraining data employed in previous studies is private, creating barriers to reproducibility, broader research and application. Second, current collections often focus on narrow anatomical regions, which is insufficient to fully capture the diversity of whole-body regions. This limitation restricts their utility in comprehensive clinical workflows. Third, most approaches rely on off-the-shelf vision foundation model frameworks [ 13 ] , failing to systematically explore network architecture optimizations tailored to the morphological complexity and spatial hierarchies of anatomical structures. This oversight limits the model’s ability to capture anatomical relationships across scales and organs during pre-training. Finally, existing research mainly focuses on a few downstream tasks such as image classification or segmentation, leaving open questions about model capabilities for more diverse clinical applications. In this work, we introduce EchoCare, a novel foundation model for ultrasound images, accompanied by a systematic investigation of its utility across a diverse spectrum of clinical tasks. EchoCare is pre-trained on EchoCareData, our newly curated large-scale and openly accessible dataset comprising 4.5 million ultrasound images. Collected from multi-center, multi-device, multi-modality, and multi-ethnic global sources, EchoCareData ensures diverse data representation. EchoCareData covers 9 major regions and 52 anatomical organs of the human body, supporting models pretrained on it to generalize effectively across comprehensive whole-body ultrasound clinical applications. We have also optimized the architecture of the vision foundation model to better capture hierarchical anatomical structures, from broad ultrasound regions ( e.g. , abdomen) to specific organs ( e.g. , liver, kidney), enabling the model to mimic human-like clinical diagnostic reasoning. Extensive evaluations across eight categories of core ultrasound clinical tasks of varying diagnostic difficulties, such as lesion segmentation, organ detection, disease diagnosis, and quantitative regression, reveal that EchoCare significantly outperforms state-of-the-art general-domain foundation models, underscoring the critical need for ultrasound-specific models. Compared with leading ultrasound-focused foundation models, EchoCare also demonstrated superior performance, highlighting the advantages of pretraining on large, diverse data. We will release both EchoCare and the EchoCareData to promote clinical AI development in ultrasound images upon publication. Figure 1 : Overview of this study. Caption on next page. (Previous page.) Figure 1: Overview of this study. a. The ultrasound data from over 20 countries and 5 continents are collected, encompassing over 4 million ultrasound images. b. The constructed ontology shows a hierarchy of object types that are used to unify semantic concepts across datasets. Bar plots showing the number of images containing that object type. c. Flowchart of EchoCare. EchoCare takes a masked image as input and then outputs the reconstructed ultrasound image. To capture both global anatomical contexts and local ultrasound characteristics, EchoCare also incorporates a novel hierarchical classifier branch. d. Performance evaluation across diverse clinical applications, spanning different organs and a wide range of diagnostic difficulties. EchoCare achieved state-of-the-art performance across all downstream clinical tasks. Image was created with BioRender.com. 2 Results Figure 2 : Evaluation on disease diagnostic classification and anatomical segmentation. a-f. EchoCare consistently outperforms previous state-of-the-art (SOTA) models (w/o FM) and other existing foundation models (RadImageNet [ 22 ] , UltraSAM [ 23 ] , CLIP [ 25 ] , BiomedCLIP [ 35 ] , DINO [ 34 ] , SimMIM [ 30 ] , USFM [ 15 ] ) across different classification and segmentation tasks. Specifically, for classification, we