Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation

Chunzheng Zhu, Yijun Wang, Jianxin Lin, Feng Wang, Hongwei Wang, Lei Zhao, Shengli Li, Kenli Li · 2026

Read Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation by Chunzheng Zhu, Yijun Wang, Jianxin Lin, Feng Wang, Hongwei Wang, Lei Zhao, Shengli Li, Kenli Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25402.txt.

Introduction

Chunzheng Zhu and colleagues shift ultrasound representation learning from generic image regions to anatomical structures that matter. Their ANAUS framework uses a tuned prompt engine plus single adaptation on public image and mask pairs to delineate anatomy without manual labels at scale. A dual training scheme then aligns features within the same anatomy while separating other structures, and predicts damaged core regions to capture fine detail. Evaluations on six public datasets show consistent gains over current leading methods. The approach keeps computation lean enough for clinical deployment.

Catalog record

CREATORChunzheng Zhu, Yijun Wang, Jianxin Lin, Feng Wang, Hongwei Wang, Lei Zhao, Shengli Li, Kenli Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28424
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25402

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation Chunzheng Zhu, Yijun Wang, Jianxin Lin, Feng Wang, Hongwei Wang, Lei Zhao, Shengli Li, Kenli Li Abstract Self-supervised pre-training paradigm has gained increasing prominence for learning transferable representations in medical imaging, yet existing methods for ultrasound (US) images operate at the image or frame level, overlooking the anatomical context for clinical-aligned representation learning. In this work, we propose an A natomy- A nchored U ltra S ound Self-Supervision framework ( AnaUS ) that shifts representation learning from generic visual regions to clinically meaningful anatomical structures. Utilizing a learnable latent prompt engine alongside a one-time domain adaptation on existing public image–mask pairs, we empower the LP-SAM module to achieve annotation-free anatomy delineation at scale. Building upon this anatomical grounding, we propose a dual-policy self-supervised learning paradigm consisting of inter-view semantics-aware anatomy-separating alignment and contextual core-region prediction to enhance representation learning. Specifically, the former enforces feature invariance within identical anatomical regions while promoting discriminability across distinct structures; the latter compels the model to reconstruct corrupted regions, thereby capturing fine-grained structural details. Extensive evaluations on six public datasets demonstrate that AnaUS consistently outstrips current state-of-the-art methods while maintaining the computational efficiency essential for clinical deployment. Code is available at https://github.com/zhcz328/ANAUS . 1 Introduction Ultrasound (US) imaging is a cornerstone of clinical diagnostics owing to its real-time capability, portability, and non-ionizing nature [ 4 , 33 ] . Deep neural networks have demonstrated considerable promise in automating US image analysis [ 11 , 8 , 18 ] , yet their data-hungry training regimes remain at odds with the limited availability of annotated clinical data. Self-supervised learning (SSL) addresses this bottleneck by deriving supervision from the data itself, enabling pre-training on large unlabeled corpora followed by efficient fine-tuning [ 16 , 6 , 14 ] . Current SSL strategies for ultrasound broadly fall into two categories. Frame-level approaches leverage temporal coherence in US video to construct positive pairs [ 8 , 2 , 26 ] , yet they treat each frame holistically and risk encoding background artifacts and speckle noise rather than diagnostically relevant anatomy. Region-level methods borrowed from natural images [ 17 , 28 ] attempt finer-grained learning, but their reliance on generic segmentation ( e.g. , K-means clustering or image-computable proposals) fails to isolate meaningful anatomical structures in US images, where boundaries are characteristically low-contrast and speckle-corrupted [ 3 ] . Neither paradigm aligns with clinical reasoning, which is inherently anatomy-centric : diagnostic interpretation focuses on the morphology, texture, and spatial relationships of specific anatomical landmarks. To ensure congruence with clinical reasoning, the US representation learning paradigm must prioritize anatomical structures over stochastic image patches as fundamental contrastive units. Attaining such structural awareness requires reconciling the inherent domain disparity between natural and medical imagery. To this end, our proposed AnaUS performs self-supervised pre-training directly at the anatomy level. This approach is powered by LP-SAM , a domain-specialized module that utilizes a learnable latent prompt engine and lightweight fine-tuning to bridge the domain gap, thereby achieving automated anatomy discovery. Utilizing this delineation capability, AnaUS integrates two synergistic learning mechanisms. The first, view-invariant anatomical contrastive matching, regularizes the representation space to ensure invariance for the same structure under diverse views while maximizing separation between distinct anatomical entities. Such a formulation introduces explicit inter-anatomy discriminability, a property typically absent in generic contrastive frameworks. Complementing this, an anatomical contextual prediction task targets the reconstruction of corrupted core regions, compelling the encoder to internalize both local textural intricacies and global structural coherence. These dual objectives operate across a multi-scale feature hierarchy, ensuring that the learned representations encapsulate anatomical traits at hierarchical granularities. Figure 1 : Overview of the AnaUS framework. (i) LP-SAM provides anatomical masks \mathcal{M}(x){=}\{m_{k}\}_{k=1}^{K} for unlabeled US images via a learnable latent prompt engine (Sec. 2.1 ). (ii) The encoder f_{\theta} is jointly optimized by an anatomy-level contrastive learning objective across augmented view pairs (\mathbf{v}_{1},\mathbf{v}_{2}) (Sec. 2.2 ) and a contextual prediction objective over corrupted core regions (Sec. 2.3 ). Comprehensive evaluations across six public ultrasound benchmarks, encompassing lung, breast, thyroid, and cardiac imaging, validate the multi-task efficacy of AnaUS in various downstream clinical scenarios. The proposed framework sets a new state-of-the-art, consistently surpassing both general-purpose SSL methods and previous ultrasound-specific pre-training approaches. As a result, our work underscores the pivotal role of anatomically meaningful priors in guiding the representation learning process for medical imagery. Our contributions can be summarized as: (i) We propose AnaUS , the first anatomy-level self-supervised pre-training framework designed explicitly for ultrasound imaging, shifting the contrastive unit from generic regions to clinically relevant structures. (ii) We introduce LP-SAM with a learnable latent prompt engine that empowers fully automated, expert-free structural discovery in ultrasound imagery through targeted latent-space adaptation. (iii) We demonstrate consistent state-of-the-art performance across diverse US datasets and tasks, establishing a new benchmark for ultrasound self-supervised pre-training. 2 Method Let \mathcal{X}{=}\{x_{i}\}_{i=1}^{N} be a corpus of unlabeled US images. AnaUS (Fig. 1 ) seeks an encoder f_{\theta}{:}\,\mathcal{X}{\to}\mathcal{F} whose representations are invariant within anatomical structures yet discriminative across them, without any quality or segmentation annotations. Given x , LP-SAM produces a per-image mask set \mathcal{M}(x){=}\{m_{k}\}_{k=1}^{K} , m_{k}\in\{0,1\}^{H\times W} . Two augmented views \mathbf{v}_{1}{=}t(x) and \mathbf{v}_{2}{=}t^{\prime}(x) , t,t^{\prime}{\sim}\mathcal{T} , are fed to an asymmetric online–target encoder pair optimized jointly for representation. Figure 2 : (i) Left : The overall training pipeline of AnaUS . Phase 1 fine-tunes LP-SAM adapters on paired US image–mask data; Phase 2 trains the latent prompt engine (LPE) with LP-SAM frozen. The resulting LP-SAM is then frozen to guide SSL pre-training on unlabeled data. (ii) Right : The LP-SAM architecture. In LPE, Cross-Perception Attention (CPA) fuses global ViT and local CNN features; bidirectional cross-attention iteratively refines query tokens into latent prompt embedding T^{\prime}_{\mathrm{last}} for annotation-free mask generation. 2.1 Latent-Prompted Domain Adaptation of LP-SAM SAM [ 19 ] exhibits zero-shot segmentation capability on natural images, yet its direct application to US data yields suboptimal masks owing to the domain gap. LP-SAM (Fig. 2 ) resolves this via a two-stage lightweight adaptation that preserves SAM’s capacity while injecting domain-specific inductive biases. Stage 1: Adapter-based domain specialization. Following [ 29 ] , we insert learnable adapters into each ViT block of SAM’s image encoder while keeping the pre-trained weights frozen. Each adapter implements a bottleneck projection \phi{:}\,\mathbb{R}^{d}{\to}\mathbb{R}^{d^{\prime}} ( d^{\prime}{\ll}d ) with a residual bypass, trained on existing US image–mask pairs [ 23 , 30 , 20 ] via \mathcal{L}{=}(1{-}\alpha)\mathcal{L}_{\mathrm{BCE}}+\alpha\mathcal{L}_{\mathrm{Dice}} . The Dice term proves critical for small-structure US targets with severe foreground–background imbalance. Stage 2: Latent prompt engine. Annotation-free deployment demands prompt-free mask generation. Let F_{v}\in\mathbb{R}^{c\times h\times w} and F_{c}\in\mathbb{R}^{c\times h\times w} denote the global ViT feature and local CNN feature, respectively. A Cross-Perception Attention (CPA) module fuses complementary representations as: F_{vc}=\mathit{LN}\!\left(\mathit{MHCA}(F_{v},\,F_{c},\,F_{c})+F_{v}\right), (1) where \mathrm{MHCA}(\cdot) is multi-head cross-attention [ 27 ] with F_{v} as query and F_{c} as key and value. A learnable query token T\in\mathbb{R}^{k\times c} is then iteratively refined over N rounds of bidirectional cross-attention: \displaystyle F_{v}^{\prime} \displaystyle=\mathit{MLP}\!\left(\mathit{LN}\!\left(\mathit{MHCA}(F_{vc},\,T,\,T)+F_{vc}\right)\right), (2) \displaystyle T^{\prime} \displaystyle=\mathit{MLP}\!\left(\mathit{LN}\!\left(\mathit{MHCA}(T,\,F_{vc},\,F_{vc})+T\right)\right), (3) where each round feeds the updated (F_{v}^{\prime},T^{\prime}) and original F_{c} as new inputs. The final token T^{\prime}_{\mathrm{last}} serves as the task-conditioned prompt embedding for SAM’s mask decoder, enabling fully annotation-free anatomical mask generation. During Phase 2, SAM and

Read the complete text