Multiscale Switch for Semi-Supervised and Contrastive Learning in Medical Ultrasound Image Segmentation Jingguo Qu, Xinyang Han, Yao Pu, Man-Lik Chui, Simon Takadiyi Gunda, Ziman Chen, Jing Qin, Ann Dorothy King, Winnie Chiu-Wing Chu, Jing Cai, Michael Tin-Cheung Ying Abstract Medical ultrasound image segmentation faces significant challenges due to limited labeled data and characteristic imaging artifacts including speckle noise and low-contrast boundaries. While semi-supervised learning (SSL) approaches have emerged to address data scarcity, existing methods suffer from suboptimal unlabeled data utilization and lack robust feature representation mechanisms. In this paper, we propose Switch, a novel SSL framework with two key innovations: (1) Multiscale Switch (MSS) strategy that employs hierarchical patch mixing to achieve uniform spatial coverage; (2) Frequency Domain Switch (FDS) with contrastive learning that performs amplitude switching in Fourier space for robust feature representations. Our framework integrates these components within a teacher-student architecture to effectively leverage both labeled and unlabeled data. Comprehensive evaluation across six diverse ultrasound datasets (lymph nodes, breast lesions, thyroid nodules, and prostate) demonstrates consistent superiority over state-of-the-art methods. At 5% labeling ratio, Switch achieves remarkable improvements: 80.04% Dice on LN-INT, 85.52% Dice on DDTI, and 83.48% Dice on Prostate datasets, with our semi-supervised approach even exceeding fully supervised baselines. The method maintains parameter efficiency (1.8M parameters) while delivering superior performance, validating its effectiveness for resource-constrained medical imaging applications. The source code is publicly available at https://github.com/jinggqu/Switch . I Introduction Medical imaging is an important technique in clinical diagnostics, providing the advantage of non-invasive visualization of internal body structures. Common techniques include computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound (US) imaging. US is commonly utilized for diagnosing superficial organs and tissues, such as breast lesions [ 5 ] , thyroid nodules [ 39 ] , and lymph nodes [ 52 , 2 , 1 ] because of its real-time imaging capability, non-invasiveness, and cost-effectiveness. By employing high-frequency US waves, US generates detailed images of internal structures and is widely adopted in clinical settings. The current identification process of regions of interest (ROIs) in US images is performed manually by radiologists, which is time-consuming and prone to variations in accuracy and consistency, depending on the expertise of the radiologists. Segmentation of US images presents significant challenges due to three primary factors. First, the quality of US images is often lower than CT and MRI because of the presence of speckle noise, which is a granular pattern that degrades image quality. More specifically, the ambiguity of ROI boundaries contributes to low contrast and artifacts caused by noise in US images. Second, the shape, size, and position of ROIs may vary significantly across images obtained from different scan planes and different patients. Third, US images frequently exhibit inconsistent brightness, resolution, and quality, which are influenced by variations in imaging settings and operator practices. These factors make the manual annotation process labor-intensive and time-consuming, highlighting the necessity for automated systems that are capable of accurately identifying ROIs to enhance both the accuracy and efficiency of diagnostic workflows. Fig. 1: A comparison of model performance between previous semi-supervised SOTAs and the proposed Switch method on the lymph node dataset (LN-INT). Our method achieved the best results across five data labeling ratios (i.e. models were trained using only part of the ground truth). Deep learning has achieved remarkable success in image and signal processing tasks [ 28 , 56 , 13 , 26 ] . Building upon these advances, researchers have increasingly applied deep learning to medical image processing [ 38 , 59 , 36 , 20 , 37 ] , which typically requires a substantial amount of well-annotated data. This rigorous requirement may be alleviated by the advent of semi-supervised learning (SSL) [ 41 ] . SSL models effectively leverage a limited amount of labeled data in conjunction with a vast quantity of unlabeled data to learn representations while maintaining coherence. This approach offers a more viable solution for medical image segmentation. Despite numerous SSL studies aiming to mitigate this constraint, the application of SSL in US image segmentation remains relatively limited in scope. Current SSL studies of US image segmentation mainly focus on specific lesion locations, such as breast [ 19 , 55 , 15 , 24 ] and thyroid [ 43 , 8 ] . Attention-based generative adversarial networks (GANs) [ 19 , 55 ] are designed to handle individual variance in breast lesions and improve the distinction between lesions and background through the discriminator of GANs. PH-Net [ 24 ] partitions the input image into multiple equal-size patches, introducing adaptive patch augmentation and hard patch shielding strategies with high average entropy for further model training. SABR-Net [ 8 ] proposes a boundary refinement module to tackle the challenge of unclear edges within US images while introducing computational complexity. Although the aforementioned methods exhibit superior performance compared to state-of-the-art (SOTA) approaches across various US datasets, their generalizability remains uncertain due to the absence of thorough validation on other US datasets with comparable characteristics. To address this issue, this study proposes a simple and effective SSL framework (named Switch ), which is based on the teacher-student model [ 40 ] for superficial US image segmentation. Our approach integrates three key components: multiscale switch (MSS), frequency domain switch (FDS), and contrastive learning (CL) modules. First, a pair of US images with or without manual annotations are partially switched by MSS to integrate coarse and fine representations. Second, the frequency domains of the above images decomposed by Fourier transformation are partially exchanged to reconstruct images that incorporate the frequency information of unlabeled images. This process generates positive and negative sample pairs for CL. Finally, the CL module is employed to reinforce the representation of the student network by maximizing the coherence between positive pairs in feature space, and vice versa. The proposed framework is evaluated on six superficial US datasets: the in-house lymph node US datasets (LN-INT and LN-EXT), the breast US image segmentation (BUSI) dataset [ 3 ] , thyroid datasets DDTI [ 35 ] and TN3K [ 17 ] , and the Prostate [ 23 ] dataset. As shown in Figure 1 , the proposed Switch consistently outperformed previous SOTAs on the LN-INT dataset across five labeling ratios, where each ratio denotes the percentage of annotated samples used for training. The key contributions of this study are outlined as follows: • We propose a novel SSL framework for superficial US image segmentation with similar essence, which fuse the coarse and fine knowledge to strengthen the consistency within image pairs. • We develop a CL module with frequency domain switch strategy to boost the uniformity between the original and reconstructed images. • We conduct extensive experiments on six superficial US datasets with an external testing set and demonstrate the effectiveness and generalizability of the proposed framework. The remainder of this paper is organized as follows: Section II reviews the related work on medical image segmentation, SSL, and CL. Section III presents the detailed methodology of the proposed Switch framework, including the MSS, the FDS for CL, consistency regularization, augmentations, loss function, and training strategy. Section IV describes the experimental setup, datasets, evaluation metrics, implementation details, and provides comprehensive comparisons with SOTA methods along with ablation studies. Section V provides comprehensive ablation studies, embedding space analysis, and discusses the clinical implications and limitations of our approach. Finally, Section VI concludes the paper. II Related Work II-A Medical Image Segmentation Medical image segmentation refers to the classification process of 3D volumes or 2D images to extract ROIs at the pixel level, which is a fundamental task in medical image analysis and clinical applications. Many deep learning-based methods have been proposed for this task, including U-Net [ 38 ] and its variants [ 31 , 61 , 12 ] . Other methods have also achieved SOTA performance as the backbone of segmentation networks, such as DeepLab [ 9 ] , PSPNet [ 58 ] , and HRNet [ 42 ] . II-B Semi-Supervised Learning SSL methods aim to learn global representations across the entire dataset by utilizing both labeled and unlabeled data. There are two main categories of SSL methods: consistency regularization and pseudo labeling. Consistency regularization methods attempt to minimize the difference between predictions on the same input with different augmentations or views, while pseudo labeling methods aim to generate high-quality pseudo labels for unlabeled data and combine them with labeled data to strengthen the model. Many approaches based on consistency regularization [ 40 , 54 , 53 , 34 , 32 , 47 , 57 , 60 , 4 , 11 ] and pseudo labeling [ 7 , 50 , 45 , 14 ] have been proposed to address SSL. Particularly, Mean Teacher [ 40 ] enhances the performance of the student network by enforcing coherence between the predictions of student and teacher networks through an exponential moving average (EMA) strategy. UA-MT [ 53 ] introduces Monte Carlo dropout to estimate and minimize the uncertainty between the student and teacher networks. CCT [ 34 ] applies multiple auxiliary segmentation heads with randomly perturbed encoded features to reduce the discrepancy between these two networks. Copy-Paste (CP) [ 54 ] design has also been utilized for SSL, where BCP [ 4 ] adopts a bidirectional CP approach to further increase the uniformity within the dataset. In addition, large-scale pre-trained vision models such as the Segment Anything Model (SAM) [ 27 ] have been incorporated to generate high-quality pseudo labels for unlabeled samples [ 14 ] . Recent advances include PH-Net [ 25 ] which introduces patch-wise hardness estimation for breast lesion segmentation, and ABD [ 11 ] which proposes adaptive bidirectional displacement strategies. Furthermore, \beta -FFT [ 22 ] presents nonlinear interpolation in frequency domain for enhanced training strategies. II-C Contrastive Learning CL methods were proposed to learn global representations in a self-supervised manner by maximizing and minimizing the similarity between positive and negative pairs, respectively [ 18 , 49 , 10 , 21 , 44 ] . Many efforts have been made to improve the performance of SSL by incorporating CL. CDCL [ 46 ] constructs negative pairs from feature patches with large disparity to enhance the discrimination capability of the segmentation model. MMS [ 30 ] introduces independent classifiers and projectors to conduct supervised and unsupervised CL in feature space according to spatial correspondence at the pixel level. In addition, U 2 PL [ 45 ] filters out unreliable pseudo labels predicted by the teacher model through entropy estimation as negative samples, and then these negative samples are pushed into a memory bank [ 49 ] to provide consistent and continually updated support for the student model. Similarly, PH-Net [ 25 ] shields patches with high entropy to avoid them being altered through CutMix [ 54 ] operations at the patch level. A separate projector and memory bank are also associated with the high reliability patch selection, sampling, and CL processes. These current CL methods for semantic segmentation mainly focus on the overall variability between labeled and unlabeled data, while ignoring the information cohesion between them. Following this observation, we propose a novel sample pair construction method for CL by employing frequency domain switching to enhance the local harmony and uniformity within US image pairs. II-D Summary Despite promising results in general medical image segmentation, existing SSL and CL methods face two critical limitations for US image segmentation: (1) Geometric Adaptation Challenges : Fixed-patch methods like Copy-Paste [ 16 ] and BCP [ 4 ] inadequately handle US ROIs with highly variable shapes and positions across scan planes; (2) Feature Representation Limitations : Current CL approaches emphasize overall variability while neglecting information cohesion between labeled and unlabeled data. To address these limitations, we propose Switch which integrates multiscale spatial switching with frequency domain manipulation for enhanced US image segmentation. The detailed methodology is presented in the following section. III Method Fig. 2: Overview of our proposed method. f_{\mathrm{s}} and f_{\mathrm{t}} are the student and teacher network, respectively. Stage 1 : The system splits labeled data into two subsets ( x_{1} , x_{2} ) and unlabeled data into two subsets ( u_{1} , u_{2} ), generates pseudo-labels for unlabeled images using f_{\mathrm{t}} , creates spatial masks for patch-based mixing, and combines unlabeled base images with labeled patches to create mixed training samples for semi-supervised learning. Stage 2 : The system performs amplitude switching between labeled and unlabeled image batches in the frequency domain, creates new mixed images using the same spatial masks from MSS, and applies CL to align feature representations between original and frequency-switched mixed images, while enforcing consistency regularization between their model predictions to enhance semi-supervised learning robustness. More details of MSS and FDS modules can be found in Figure 3 and Figure 4 . The overall structure of our proposed method is shown in Figure 2 , which is based on the Mean Teacher framework [ 40 ] . Our approach employs a teacher-student architecture consisting of two neural networks with identical U-Net [ 38 ] architectures that collaborate to leverage both labeled and unlabeled data effectively. In this framework, the student network f_{\mathrm{s}} serves as the primary learning model that is actively trained using gradient descent on both labeled and unlabeled data. It receives mixed samples generated through our multiscale switch mechanism and learns to predict segmentation masks under supervision from both ground truth labels and pseudo labels. The teacher network f_{\mathrm{t}} , in contrast, acts as a stable reference model that generates reliable pseudo labels for unlabeled data. Crucially, the teacher network is not directly trained through gradient descent; instead, it maintains a temporally averaged version of parameters derived from the student network through exponential moving average (EMA) updates. The EMA update mechanism can be mathematically expressed as: \theta_{\mathrm{t}}^{(k+1)}=\alpha\theta_{\mathrm{t}}^{(k)}+(1-\alpha)\theta_{\mathrm{s}}^{(k+1)} (1) where \theta_{\mathrm{t}}^{(k)} and \theta_{\mathrm{s}}^{(k)} represent the parameters of teacher and student networks at iteration k , respectively, and \alpha is the momentum coefficient (typically set to 0.99). For ease of description, we hereby make the following mathematical definitions. Given a medical US dataset \mathcal{D}=\{\mathcal{D}_{l}\cup\mathcal{D}_{u}\} , \mathcal{D}_{l}=\{x_{i},y_{i}\}_{i=1}^{N} consists of N images with ground truth annotations and \mathcal{D}_{u}=\{u_{i}\}_{i=1}^{M} only includes M ( N\ll M ) unlabeled images, where x_{i} , u_{i} and y_{i} are the input image with and without annotation and corresponding ground truth label, respectively. III-A Multiscale Switch In CP series methods [ 54 , 4 ] , only fixed-area arbitrary regions of input images were used to fuse sample pairs as input to the student model, which may lack the concentration for US images with variable ROI size and location. To address this issue and inspired by SwAV [ 6 ] and BCP [ 4 ] , we propose an MSS mechanism to incorporate partial unlabeled information into labeled samples. First, a binary mask \mathcal{M} consisting of p coarse patches and q fine patches is randomly generated for sample batches. The mask generation algorithm can be formalized as: \mathcal{M}=\bigcup_{i=1}^{p}\mathcal{P}_{\mathrm{coarse}}^{(i)}\cup\bigcup_{j=1}^{q}\mathcal{P}_{\mathrm{fine}}^{(j)} (2) where \mathcal{P}_{\mathrm{coarse}}^{(i)} and \mathcal{P}_{\mathrm{fine}}^{(j)} represent the i -th coarse patch (size s_{c}\times s_{c}=128\times 128 ) and j -th fine patch (size s_{f}\times s_{f}=32\times 32 ), respectively. Each patch is positioned according to: \displaystyle\mathcal{P}_{\mathrm{coarse}}^{(i)} \displaystyle=\{(x,y):x_{i}\leq x