HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation
Tran Quoc Khanh Le, Nguyen Lan Vi Vu, Ha-Hieu Pham, Xuan-Loc Huynh, Tien-Huy Nguyen, Minh Huu Nhat Le, Quan Nguyen, Hien D. Nguyen · 2025
Read HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation by Tran Quoc Khanh Le, Nguyen Lan Vi Vu, Ha-Hieu Pham, Xuan-Loc Huynh, Tien-Huy Nguyen, Minh Huu Nhat Le, Quan Nguyen, Hien D. Nguyen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09876.txt.
Introduction
Tran Quoc Khanh Le and seven co-authors present a 2025 paper on segmenting fetal ultrasound images when labeled examples are scarce. Outlining the cervix in these scans is hard: contrast is low, shadows intrude, and borders blur. Fully supervised networks need large annotated sets that clinics rarely have. Their framework, HDC, learns from unlabeled scans too, using a single teacher model rather than several.
A hierarchical distillation scheme guides the student with two losses. One aligns feature representations between teacher and student. The other uses mutual information to steady the student against noisy guidance. The design cuts computing cost while fighting the confirmation bias that plagues teacher-student setups. On two fetal ultrasound data sets the method matched multi-teacher rivals with less overhead. The authors target practical semi-supervised segmentation for clinical imaging.
Catalog record
| CREATOR | Tran Quoc Khanh Le, Nguyen Lan Vi Vu, Ha-Hieu Pham, Xuan-Loc Huynh, Tien-Huy Nguyen, Minh Huu Nhat Le, Quan Nguyen, Hien D. Nguyen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-09876
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation Tran Quoc Khanh Le, Nguyen Lan Vi Vu, Ha-Hieu Pham, Xuan-Loc Huynh, Tien-Huy Nguyen, Minh Huu Nhat Le, Quan Nguyen, Hien D. Nguyen Abstract Transvaginal ultrasound is a critical imaging modality for evaluating cervical anatomy and detecting physiological changes. However, accurate segmentation of cervical structures remains challenging due to low contrast, shadow artifacts, and indistinct boundaries. While convolutional neural networks (CNNs) have demonstrated efficacy in medical image segmentation, their reliance on large-scale annotated datasets presents a significant limitation in clinical ultrasound imaging. Semi-supervised learning (SSL) offers a potential solution by utilizing unlabeled data, yet existing teacher-student frameworks often encounter confirmation bias and high computational costs. In this paper, a novel semi-supervised segmentation framework, called HDC, is proposed incorporating adaptive consistency learning with a single-teacher architecture. The framework introduces a hierarchical distillation mechanism with two objectives: Correlation Guidance Loss for aligning feature representations and Mutual Information Loss for stabilizing noisy student learning. The proposed approach reduces model complexity while enhancing generalization. Experiments on fetal ultrasound datasets, FUGC and PSFH, demonstrate competitive performance with reduced computational overhead compared to multi-teacher models. 1 Introduction Figure 1 : An intuitive overview of our Hierarchical Distillation and Consistency scheme. Transvaginal ultrasound is a widely used imaging technique for assessing cervical anatomy, providing essential insights into muscle structure and function. Accurate segmentation of cervical ultrasound images is crucial for evaluating physiological changes and guiding clinical decision-making. However, the segmentation process is challenging due to common ultrasound imaging issues such as shadow artifacts, low contrast, and indistinct boundaries [ 1 , 41 ] . Convolutional Neural Networks (CNNs) have demonstrated strong performance in medical image segmentation by capturing local spatial features. However, traditional supervised learning approaches that rely on CNNs require large-scale annotated datasets to be generalized effectively [ 32 ] . In medical imaging, particularly in transvaginal ultrasound, obtaining pixel-wise labeled data is both time-consuming and costly, limiting the scalability of supervised CNN models [ 34 ] . Moreover, CNNs primarily focus on local dependencies, which may lead to segmentation errors such as over-segmentation or under-segmentation, particularly in complex anatomical structures. To overcome these limitations, semi-supervised learning (SSL) has emerged as a promising approach by leveraging both labeled and unlabeled data to improve segmentation performance. Among various SSL frameworks, the Mean Teacher [ 40 ] model has gained significant attention that ensures smooth training dynamics and improves student and teacher prediction consistency. Recent advancements, such as Dual Teacher [ 8 ] , introduce multiple teacher networks that periodically switch during training to enhance model robustness and mitigate confirmation bias. These techniques highlight the promise of multi-teacher SSL frameworks but add complexity or intricate update mechanisms. Figure 2 : Our proposed semi-supervised pipeline for transvaginal ultrasound segmentation. The teacher generates pseudo-labels to guide the student, which processes both raw and noisy inputs. The dashed arrow (- - -) represents strong augmentation, the double slash arrow (//) denotes feature comparisons for L_{CG} and L_{MI} , black arrows indicate forward propagation, and red arrows represent pixel-level consistency loss. Existing dual-teacher and multi-student methods [ 8 , 52 ] , despite their advantages, often suffer from high computational costs due to the need for training and maintaining multiple models. Some approaches address confirmation bias by incorporating strong-weak augmentation strategies or contrastive learning mechanisms. While these techniques can enhance segmentation performance, they frequently demand careful hyperparameter tuning and extensive computational resources, making them less practical for real-world medical imaging applications. To address the challenges of semi-supervised medical image segmentation—particularly confirmation bias and computational inefficiency—this work introduces a novel framework that combines adaptive consistency training with a single-teacher architecture. Unlike conventional approaches that rely on multiple teacher-student models [ 8 , 52 ] , our method enhances training efficiency and generalization through structured noise injection and carefully designed objective functions .Most existing teacher-student methods focus primarily on transferring knowledge through pseudo-labeling based solely on output predictions. In contrast, we propose a hierarchical distillation mechanism that enriches the learning signal and improves feature alignment, inspired by theoretical insights from prior works on knowledge distillation [ 49 , 25 ] . This is achieved through two novel loss functions: (1) Correlation Guidance Loss , which encourages consistency between the feature representations of the teacher and the main student branch, and (2) Mutual Information Loss , which aligns the representations between the main student and a noisy student branch to enhance intra-batch stability. These mechanisms are designed to bridge the feature distribution gap between teacher and student networks while stabilizing student representations in the presence of noise. HDC is evaluated on two challenging fetal ultrasound datasets, FUGC and PSFH [ 1 , 21 ] , where it achieves state-of-the-art segmentation performance with significantly reduced computational overhead compared to existing methods under various evaluation metrics. 2 Related Works Semi-Supervised Learning (SSL). The core challenge of semi-supervised learning is to maximize model accuracy while minimizing human annotation effort by effectively leveraging abundant unlabeled data in conjunction with a small set of high-quality labeled samples. Foundational works in SSL can be categorized into two dominant branches: Pseudo-labeling [ 37 , 2 ] and consistency regularization [ 33 , 28 , 30 ] . Pseudo-labeling follows an iterative training process. The most up-to-date optimized model generates pseudo-labels from unlabeled data and combines them with labeled samples to refine prediction over time. On the other hand, consistency regularization-based methods rely on the smoothness assumption [ 16 ] , stemming from the goal of training models that remain invariant to various perturbations applied to either the input data or the models themselves. FixMatch [ 38 ] integrates the strength of both strategies by employing a weak-to-strong consistency mechanism. In this research, pseudo labels are derived from weakly augmented samples and used to supervise the training of their strongly augmented counterparts. FlexMatch [ 50 ] takes into account different learning difficulties across categories and creates different thresholds for each class. Semi-Supervised Semantic Segmentation (S4). Benefits from the advances in deep neural networks [ 17 , 45 , 11 ] and robust SSL algorithms, many works adopt them to build powerful semantic segmentation baselines [ 19 , 27 ] . In medical image analysis, SSL also advances medicine significantly, with many noticeable works [ 30 , 23 , 29 ] . On top of these fundamental designs, Mean-Teacher-based methods have achieved remarkable results across benchmarks. The Mean Teacher (MT) framework [ 40 ] introduces a teacher-student architecture in which the teacher model, serving as a stable version of the student, is updated using the exponential moving average (EMA) of the student’s weights to guide the student during training. Building upon MT, PS-MT [ 20 ] proposes a stricter teacher that prevents model collapse. AD-MT [ 52 ] employs a dual-teacher framework with different perspectives in an adaptive switching manner to mitigate confirmation bias. On the other hand, co-training has also shown promising results in S4 by leveraging multi-view references to improve model perception and enhance pseudo-label reliability. Cross-pseudo supervision (CPS) [ 5 ] enforces consistency between the outputs of two networks initialized with different weights to foster diverse feature learning. CCVC [ 47 ] suggests learning richer semantic information from competing