Head and Neck Tumor Segmentation of MRI from Pre- and Mid-radiotherapy with Pre-training, Data Augmentation and Dual Flow UNet
Litingyu Wang, Wenjun Liao, Shichuan Zhang, Guotai Wang✉ · 2024
Read Head and Neck Tumor Segmentation of MRI from Pre- and Mid-radiotherapy with Pre-training, Data Augmentation and Dual Flow UNet by Litingyu Wang, Wenjun Liao, Shichuan Zhang, Guotai Wang✉ (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-14846.txt.
Introduction
Litingyu Wang and co-authors build a tool that outlines head and neck tumors on MRI scans before and during radiation care. Their paper from 2024 notes that exact outlines drive care planning and outlook sums. Hand drawn outlines at pixel level are slow, so a sound auto method is in demand.
For scans before care, the team used full training, then added pre trained weights and a data mixing trick to boost results. For scans during care, they built a network with two input paths, so the prior scan and its labels guide the new outline step by step. The best models from each fold were averaged for the final test. Scores reached about eighty two percent before care and seventy three during care.
Catalog record
| CREATOR | Litingyu Wang, Wenjun Liao, Shichuan Zhang, Guotai Wang✉ |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-14846
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Head and Neck Tumor Segmentation of MRI from Pre- and Mid-radiotherapy with Pre-training, Data Augmentation and Dual Flow UNet Litingyu Wang, Wenjun Liao, Shichuan Zhang, Guotai Wang✉ Abstract Head and neck tumors and metastatic lymph nodes are crucial for treatment planning and prognostic analysis. Accurate segmentation and quantitative analysis of these structures require pixel-level annotation, making automated segmentation techniques essential for the diagnosis and treatment of head and neck cancer. In this study, we investigated the effects of multiple strategies on the segmentation of pre-radiotherapy (pre-RT) and mid-radiotherapy (mid-RT) images. For the segmentation of pre-RT images, we utilized: 1) a fully supervised learning approach, and 2) the same approach enhanced with pre-trained weights and the MixUp data augmentation technique. For mid-RT images, we introduced a novel computational-friendly network architecture that features separate encoders for mid-RT images and registered pre-RT images with their labels. The mid-RT encoder branch integrates information from pre-RT images and labels progressively during the forward propagation. We selected the highest-performing model from each fold and used their predictions to create an ensemble average for inference. In the final test, our models achieved a segmentation performance of 82.38% for pre-RT and 72.53% for mid-RT on aggregated Dice Similarity Coefficient (DSC) as HiLab. Our code is available at https://github.com/WltyBY/HNTS-MRG2024_train_code . 1 Introduction Head and neck (H&N) cancers are among the most prevalent types of cancer. Imaging in H&N cancer serves multiple purposes, including quantitative assessment of tumors, evaluation of nodal disease, and differentiation between recurrent tumors and post-treatment changes [ 11 ] . In recent years, convolutional neural networks (CNNs) within the realm of deep learning have profoundly impacted medical image analysis [ 2 , 4 ] . The detection and segmentation of H&N tumors and metastatic lymph nodes are crucial for diagnosis and treatment. In addition, deep learning aids in these processes by segmenting target areas to facilitate treatment and surgical planning. During radiation treatment planning, computed tomography (CT) is commonly utilized [ 12 ] . However, CT images often lack clear delineation between lymph nodes and surrounding tissues [ 3 , 5 ] . And, unfortunately, most existing datasets are based on CT, contrast-enhanced CT, or positron emission computed tomography (PET) [ 1 , 9 ] . In contrast, the HNTS-MRG2024 Challenge provides a magnetic resonance imaging (MRI) dataset for H&N images, representing an uncommon imaging modality in H&N cancer diagnosis and capturing a significant scenario of adaptive radiotherapy. This competition focuses on segmenting gross tumor volumes of primary tumors (GTVp) and lymph nodes (GTVn) on both pre-radiotherapy (pre-RT) and mid-radiotherapy (mid-RT) MRI images, designated as Task-1 and Task-2, respectively. Our methods and innovations are summarized as follows. We designed a novel framework, incorporating model pre-training, multiple data augmentations, and network architectures. Firstly, under the competition rules allowing the use of external public datasets, we pre-trained our model on a CT dataset due to the absence of an appropriate H&N MRI dataset. Given the significant differences in image intensities between CT and MRI, we used histogram matching strategy to align the intensity distributions in the preprocessing stage and employed nonlinear intensity transformations in the model pre-training stage. Secondly, to address the scarcity of foreground voxels in our dataset, we employed the MixUp [ 18 ] data augmentation technique. This approach not only expanded the training dataset, but also enhanced the model’s generalizability. Furthermore, different from the conventional single encoder-decoder network structure commonly used in segmentation tasks, we adopted a dual encoder-decoder architecture to leverage per-RT images to guide the segmentation in mid-RT images. Attention blocks were introduced to this innovative structure, allowing for the effective fusion of information from disparate encoder streams. After conducting a rigorous five-fold cross-validation, we identified the top-performing model in each fold. The selected models demonstrated impressive performance, with an average aggregated Dice Similarity Coefficient (DSC) of 80.65% for Task-1 and 74.68% for Task-2 across five-fold cross-validation. Furthermore, in the final test, they scored 82.38% for Task-1 and 72.53% for Task-2. Figure 1: Two architectures used for training. (a) An encoder-decoder network, named basic segmentation network, with MixUp diagram. (b) Two separate encoders with one decoder named Dual Flow UNet (DFUNet). 2 Methods We proposed multiple training strategies for the two tasks, and selected the best-performing model in each of the five-fold cross-validation divisions. For the segmentation of pre-RT images in Task-1, we employed: \bullet Fully supervised learning utilizing an encoder-decoder architecture, as depicted in Fig. 1 (a). \bullet The same fully supervised learning approach, enhanced with pre-trained weights and the MixUp data augmentation method. For Task-2, we introduced an additional training method: \bullet Fully supervised learning employing the Dual Flow UNet (DFUNet) architecture, illustrated in Fig. 1 (b). 2.1 Networks We designed two network architectures in this study. The first network is basic segmentation network, shown as Fig. 1 (a)), which is an encoder-decoder architecture. For task-1, the input of basic segmentation network is the single-channel pre-RT images. For Task-2, the input of basic segmentation network is three-channel images, which are consisted of mid-RT image, registered pre-RT image and its mask. The second network is DFUNet, shown as Fig. 1 (b), which is only used for Task-2. The DFUNet comprises two encoders, whose inputs are a single-channel mid-RT image and a two-channel input (registered pre-RT image and its mask), respectively. To facilitate information fusion, the information from the pre-RT image and its mask is integrated into the mid-RT stream at various encoder stages using a CNN-based cross attention block [ 13 ] . This CNN-based attention mechanism differs from the attention blocks described in [ 14 ] by employing both spatial and channel attention [ 16 ] . This approach enhances or fuses features while significantly reducing the computational demands of training and inference. The operational details of the cross attention block are illustrated in Fig. 2 . Details of the network’s configuration are elaborated in Section 3.3 . Figure 2: CNN-based cross attention block to integrate secondary information f_{pre} into primary information f_{mid} . Figure 3: Raw image (the first column) and corresponding augmented images using Bézier Curve [ 8 ] . 2.2 Model Pre-training We utilized the SegRap2023 Challenge Dataset [ 9 ] and the basic segmentation network depicted in Fig. 1 (a) for fully supervised pre-training. However, since SegRap2023 is based on CT images, we encountered challenges due to the inherent intensity differences between CT and MRI data. To minimize these differences, we employed histogram matching during preprocessing to align the intensity distributions of CT and MRI as closely as possible. Despite this effort, intensity variations across various anatomical structures remained a challenge. To further address these issues, we incorporated techniques from domain generalization [ 19 ] , applying nonlinear transformations to the image intensities. This approach was designed to reduce the model’s reliance on specific intensity values, thereby enhancing its generalization capabilities. The effectiveness of these nonlinear transformations is visualized in Fig. 3 . 2.3 MixUp Augmentation Strategy Since our training methodology is based on patches, patches in one mini-batch inevitably contain negative samples, which entirely belong to the background category. To mitigate the impact of these negative samples and to encourage the network to learn the distribution across different classes, we employ the MixUp [ 18 ]