Challenging DINOv3 Foundation Model under Low Inter-Class Variability: A Case Study on Fetal Brain Ultrasound

Edoardo Conti, Riccardo Rosati, Lorenzo Federici, Adriano Mancini, Maria Chiara Fiorentino · 2025

Read Challenging DINOv3 Foundation Model under Low Inter-Class Variability: A Case Study on Fetal Brain Ultrasound by Edoardo Conti, Riccardo Rosati, Lorenzo Federici, Adriano Mancini, Maria Chiara Fiorentino (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01915.txt.

Introduction

Edoardo Conti and four colleagues give a vision foundation model its first thorough examination in fetal ultrasound. DINOv3 transfers impressively across medical imaging. Fetal brain scans pose a special trap, though. Doctors must tell apart three standard planes: transthalamic, transventricular, and transcerebellar. They look almost identical. Mistaking them undermines biometric checks.

To test fairly, the team curated every public fetal ultrasound dataset into one multi-site benchmark, FetalUS-188K, with more than 188,000 annotated images from varied acquisition settings. They pretrained DINOv3 self-supervised on the ultrasound data and compared adaptation by linear probing and full fine-tuning, against initialization from natural-image weights. Models pretrained on fetal ultrasound consistently beat the natural-image starts, showing that domain pretraining matters most exactly where classes differ least.

Catalog record

CREATOREdoardo Conti, Riccardo Rosati, Lorenzo Federici, Adriano Mancini, Maria Chiara Fiorentino
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01915

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Challenging DINOv3 Foundation Model under Low Inter-Class Variability: A Case Study on Fetal Brain Ultrasound Edoardo Conti, Riccardo Rosati, Lorenzo Federici, Adriano Mancini, Maria Chiara Fiorentino Abstract Purpose: This study provides the first comprehensive evaluation of foundation models in fetal ultrasound (US) imaging under low inter-class variability conditions. While recent vision foundation models such as DINOv3 have shown remarkable transferability across medical domains, their ability to discriminate anatomically similar structures has not been systematically investigated. We address this gap by focusing on fetal brain standard planes—transthalamic (TT), transventricular (TV), and transcerebellar (TC)—which exhibit highly overlapping anatomical features and pose a critical challenge for reliable biometric assessment. Methods: To ensure a fair and reproducible evaluation, all publicly available fetal ultrasound datasets were curated and aggregated into a unified multicenter benchmark, FetalUS-188K, comprising more than 188,000 annotated images from heterogeneous acquisition settings. DINOv3 was pretrained in a self-supervised manner to learn ultrasound-aware representations. The learned features were then evaluated through standardized adaptation protocols, including linear probing with frozen backbone and full fine-tuning, under two initialization schemes: (i) pretraining on FetalUS-188K and (ii) initialization from natural-image DINOv3 weights. Results: Models pretrained on fetal ultrasound data consistently outperformed those initialized on natural images, with weighted F1-score improvements of up to 20%. Domain-adaptive pretraining enabled the network to preserve subtle echogenic and structural cues crucial for distinguishing intermediate planes such as TV. Conclusion: Results demonstrate that generic foundation models fail to generalize under low inter-class variability, whereas domain-specific pretraining is essential to achieve robust and clinically reliable representations in fetal brain ultrasound imaging. 2025 keywords Standard-plane detection, Foundation models, Ultrasound imaging, Fetal ultrasound, DINOv3 1 Introduction Fetal standard planes—such as abdominal, brain, and cardiac views—are fundamental in prenatal ultrasound (US) imaging, as they provide the basis for fetal biometry and enable the detection of conditions like growth restriction and congenital anomalies migliorelli2024use ; fiorentino2022review . Traditionally, identifying these planes has relied on the expertise of trained sonographers. However, the manual interpretation of US images is inherently subjective, which may compromise the consistent and accurate localization of standard planes. This has motivated the adoption of deep learning (DL) techniques to automate plane identification, thereby enhancing both efficiency and precision in fetal biometry bai2024psfhs . Early DL approaches for standard plane detection predominantly relied on single-center datasets, which, while useful for proof-of-concept studies, suffer from limited generalizability due to restricted population diversity and site-specific imaging protocols kiserud2017world . The release of the first publicly available datasets, such as those introduced by burgos2020evaluation and subsequently employed in krishna2024standard ; migliorelli2024use ; krishna2023automated , marked an important step toward reproducible research in this domain. However, these datasets still originated from limited clinical settings, raising concerns about model robustness when deployed across diverse healthcare environments. Recognizing these limitations, the research community has progressively shifted toward multicenter studies sendra2023generalisability ; krishna2024deep ; fiorentino2025contrastive , which aggregate data from multiple hospitals and populations to better capture the heterogeneity of real-world clinical practice. More recently, this evolution has culminated in the development of generalizable foundation models ambsdorf2025general , which leverage large-scale pretraining on diverse vision tasks to achieve transferable representations for medical imaging. While these models demonstrate robust performance across standard planes with high inter-class variability—such as abdominal circumference and four-chamber heart views, where anatomical structures are distinctly different—a critical gap remains in their evaluation on more challenging scenarios. Specifically, their ability to discriminate between anatomically similar planes has not been systematically investigated. Brain planes such as transventricular (TV), transcerebellar (TC), and transthalamic (TT) views exemplify this challenge: they share overlapping structural features, including similar echogenic patterns and the presence of key landmarks like the midline, lateral ventricles, and posterior fossa structures. This low inter-class variability makes their differentiation particularly difficult, yet accurate classification is essential for reliable biometric measurements—such as head circumference and biparietal diameter—and for the detection of neurological anomalies gallery2020isuog . In this work, we address this gap by conducting the first comprehensive evaluation of foundation models on fetal brain planes with low inter-class variability. To ensure a fair and reproducible assessment that reflects real-world clinical diversity, we aggregate all publicly available fetal US datasets from the literature, creating the largest and most heterogeneous benchmark to date for this specific task. Our contributions are fourfold: (i) we present, to the best of our knowledge, the first application of DINOv3 to US imaging, considering its potential to better capture domain-specific characteristics compared to previous versions; (ii) we establish a unified and curated multi-center benchmark (FetalUS-188K) that consolidates and harmonizes fragmented public datasets through a standardized cleaning and filtering procedure, enabling rigorous comparison across methods and promoting reproducibility in future research; (iii) we provide the first systematic analysis of foundation model performance on anatomically similar brain planes, revealing critical limitations in handling subtle inter-class differences that are not apparent when evaluating on standard, high-variability plane classifications; and (iv) we demonstrate that while foundation models exhibit strong feature extraction capabilities, targeted fine-tuning strategies are necessary to achieve clinically acceptable discrimination between TV, TC, and TT views. Figure 1 : Overview of the proposed pipeline. (1) All publicly available fetal ultrasound datasets are aggregated and curated to build the FetalUS-188K dataset, ensuring heterogeneous and diverse ultrasound acquisitions. (2) A self-supervised training strategy based on the DINOv3 pipeline is employed to learn a feature extractor specifically aware of fetal ultrasound characteristics. (3) The learned representations are transferred to the downstream task of fetal brain standard plane classification. 2 Method To assess the capability of foundation models in scenarios characterized by low inter-class variability, such as fetal brain planes, we have chosen DINOv3 as our baseline model liu2025does . DINOv3, a state-of-the-art self-supervised framework based on Vision Transformers (ViTs), introduces significant improvements over its predecessor DINOv2. Its backbone enables the extraction of scalable and cross-domain features, incorporating techniques like multi-crop strategies, Gram anchoring, and patch-level latent reconstruction. This is crucial where challenges like domain shifts and noise are common, as seen in US acquisitions. The overall architecture is showed in Figure 1 . DINOv3 is based on a composite self-supervised objective, combining a global consistency loss with a patch-level latent reconstruction loss. Given any US image \mathbf{x} , multiple views (two global crops \mathbf{x}1 , \mathbf{x}2 and several local crops) are obtained via stochastic transformations. Both the student network f\theta and the teacher network f{\theta’} extract features from each view. The student is then optimized to align its global outputs with those of the teacher, while simultaneously reconstructing patch-level latent representations from masked local crops. A preliminary training objective is: \mathcal{L}_{\mathrm{Pre}}=\mathcal{L}_{\text{DINO}}+\mathcal{L}_{\text{iBOT}}+0.1\cdot\mathcal{L}_{\mathrm{Koleo}} where \mathcal{L}_{\text{DINO}} enforces alignment between global student and teacher predictions, \mathcal{L}_{\text{iBOT}} is a patch-level latent reconstruction loss across masked local views, and \mathcal{L}_{\mathrm{Koleo}} encourages uniform spread of features in the embedding space. Outputs of all networks are softmax-normalized, controlled by temperature parameter \tau , while multi-crop heads use LayerNorm for stabilization. After the initial training, DINOv3 introduces a Gram anchoring phase, where the model is further trained to preserve local spatial correspondences by regularizing the Gram matrix similarity between patch embeddings at different checkpoints. This process effectively maintains dense feature quality at scale. The teacher parameters are updated using an exponential moving average: \theta’\leftarrow m\theta’+(1-m)\theta , where m\in[0,1] represents the momentum. Unsupervised pre-training is performed

Read the complete text