DpDNet: An Dual-Prompt-Driven Network for Universal PET-CT Segmentation

Xinglong Liang, Jiaju Huang, Luyi Han, Tianyu Zhang, Xin Wang, Yuan Gao, Chunyao Lu, Lishan Cai, Tao Tan, Ritse Mann · 2025

Read DpDNet: An Dual-Prompt-Driven Network for Universal PET-CT Segmentation by Xinglong Liang, Jiaju Huang, Luyi Han, Tianyu Zhang, Xin Wang, Yuan Gao, Chunyao Lu, Lishan Cai, Tao Tan, Ritse Mann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07126.txt.

Introduction

Xinglong Liang and co-authors segment tumors on combined PET-CT scans with prompts that know their cancer, in this 2025 paper. Most systems train one network on all cancers as a single task, ignoring each type's habits. Their network, DpDNet, carries two prompt sets: specific ones that capture a cancer's character, and common ones that hold shared knowledge.

Special heads after the decoder keep the prompts from being forgotten. On a four-cancer dataset, the method beat leading rivals. Its volume and uptake measures also fed a breast cancer survival analysis, pointing toward personal risk sorting.

Catalog record

CREATORXinglong Liang, Jiaju Huang, Luyi Han, Tianyu Zhang, Xin Wang, Yuan Gao, Chunyao Lu, Lishan Cai, Tao Tan, Ritse Mann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07126

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DpDNet: An Dual-Prompt-Driven Network for Universal PET-CT Segmentation Xinglong Liang, Jiaju Huang, Luyi Han, Tianyu Zhang, Xin Wang, Yuan Gao, Chunyao Lu, Lishan Cai, Tao Tan, Ritse Mann Abstract PET-CT lesion segmentation is challenging due to noise sensitivity, small and variable lesion morphology, and interference from physiological high-metabolic signals. Current mainstream approaches follow the practice of one network solving the segmentation of multiple cancer lesions by treating all cancers as a single task. However, this overlooks the unique characteristics of different cancer types. Considering the specificity and similarity of different cancers in terms of metastatic patterns, organ preferences, and FDG uptake intensity, we propose DpDNet, a Dual-Prompt-Driven network that incorporates specific prompts to capture cancer-specific features and common prompts to retain shared knowledge. Additionally, to mitigate information forgetting caused by the early introduction of prompts, prompt-aware heads are employed after the decoder to adaptively handle multiple segmentation tasks. Experiments on a PET-CT dataset with four cancer types show that DpDNet outperforms state-of-the-art models. Finally, based on the segmentation results, we calculated MTV, TLG, and SUVmax for breast cancer survival analysis. The results suggest that DpDNet has the potential to serve as a valuable tool for personalized risk stratification, supporting clinicians in optimizing treatment strategies and improving outcomes.Code is available at https://github.com/XinglongLiang08/DpDNet . 1 Introduction PET-CT (Positron emission tomography and computed tomography) is widely used for cancer diagnosis and prognosis, offering both anatomical and metabolic insights [ 7 , 4 ] . PET-CT typically covers whole-body imaging while the lesions of interest are usually small, and fluorodeoxyglucose (FDG) uptake is not confined to tumor tissues but also appears physiologically in areas like the urinary tract and myocardium [ 8 ] , making PET-CT interpretation a time-consuming task for radiologists. Therefore, developing an accurate PET-CT-based automated lesion segmentation model to assist radiologists in diagnosis, evaluating patient treatment response, and assessing prognosis is valuable. In recent years, deep learning-based models have significantly advanced medical image segmentation [ 12 , 11 , 23 , 9 , 10 , 3 ] . However, whole-body PET-CT segmentation still faces three major challenges. (1) Lesions in PET-CT range from small metastatic foci to large tumors, exhibiting diverse shapes, sizes, and locations, which makes automatic segmentation challenging. (2) The scarcity of PET-CT data leads to training on mixed PET-CT images without differentiating between cancer types, overlooking the specific characteristics of different cancers. (3) The lack of clinical evaluation of segmentation results, such as their impact on diagnostic accuracy and prognosis prediction, may prevent these models from gaining the trust of healthcare professionals and being adopted in practice. Prompt engineering has the potential to be a promising tool for addressing these challenges in PET-CT segmentation. On one hand, it helps address the challenges posed by the small size and partially labeled nature of datasets [ 16 ] . On the other hand, prompt engineering integrates prior knowledge into models, enabling them to more effectively identify and segment regions of interest. A typical example is CLIP-driven universal models [ 16 , 27 , 28 ] , which incorporate the text embeddings of all labels as external knowledge, obtained by feeding medical prompts into CLIP. However, CLIP, trained on natural language text, has limited capacity for understanding medical terminology, and even a slight change in wording can significantly impact its performance [ 29 ] . Another groundbreaking work, DoDNet [ 25 , 21 ] can perform various segmentation tasks by utilizing task encoding and a controller to generate dynamic convolutions, but it suffers from delayed task awareness and the inability to leverage correlations among tasks. UniSeg [ 23 , 22 ] introduces a learnable universal prompt to capture correlations among tasks. This prompt is converted into a task-specific prompt and fed into the decoder, making the model task-aware early and improving task-specific performance. However, prematurely introducing prompts may lead to information forgetting, and using a shared segmentation head limits task-specific adaptability. In this paper, we propose a dual-prompt-driven model for universal whole-body PET-CT segmentation. Previous methods overlook the heterogeneity among different cancer types, such as variations in metastatic patterns, metabolic activity as reflected by FDG uptake, and anatomical preferences for metastases. For instance, certain cancers like breast cancer often exhibit a distinct pattern of bone metastases [ 13 ] , while others, such as lung cancer, may preferentially spread to the brain or adrenal glands [ 20 ] . These differences in biological behavior and imaging characteristics could impact the performance and generalizability of lesion segmentation models. On the other hand, metastases from different cancers are also interrelated, as they may share similar biological mechanisms of dissemination and exhibit overlapping imaging characteristics [ 2 ] . Based on these two considerations, we designed a dual-prompt strategy, comprising task-specific prompts and a common prompt. The specific prompts are tailored to capture the unique characteristics of metastases associated with each cancer type, while the common prompt leverages shared features and interrelations across different cancers to enhance the generalization of the model. These two prompts are fed into the decoder to introduce prompt information to the model early in the process. Additionally, we further designed prompt-aware heads to mitigate information forgetting and adaptively handle multiple segmentation tasks while avoiding a significant increase in model parameters. The head incorporates channel attention and multi-scale branches to enhance small-target feature extraction. Finally, we applied the segmentation results to breast cancer survival analysis to explore its potential clinical applications. Our contributions are three-fold: (1) We design a dual-prompt strategy with specific and common prompts to capture cancer-specific metastatic patterns while leveraging shared features across different cancer types. (2) Prompt-aware heads are employed to replace the shared segmentation head, enabling adaptive task handling and enhancing small-target feature extraction via channel attention and multi-scale branches. (3) Predefined parameters are automatically calculated from segmentation results on a large dataset for breast cancer survival analysis, demonstrating our model’s generalizability and its potential for risk stratification and survival prediction. Figure 1: Overview of the proposed DpDNet architecture for universal whole-body PET-CT segmentation. Cancer-specific information is captured using N (the number of cancer types) learnable prompts, which are processed via the FUSE Module to generate specific prompts. Simultaneously, common prompt is formed by N Gated Fusion Modules using fixed-order inputs from other cancer types. Both prompts are integrated into the shared decoder for segmentation. The architecture also incorporates prompt-aware heads, which are specialized segmentation heads tailored to each cancer type. 2 Methodology 2.1 Problem Definition Let \{C_{1},C_{2},\dots,C_{N}\} be N datasets of different cancer types. Here, C_{i}=\{X_{ij},Y_{ij}\}_{j=1}^{n_{i}} represents that the i -th dataset has a total of n_{i} image-label pairs, and X_{ij} and Y_{ij} are the image and the corresponding ground truth, respectively. Typically, PET-CT-based cancer segmentation models are trained either by treating all cancer datasets as a single task or by focusing on a specific cancer type. The former approach fails to capture the distinct characteristics of different cancers, while the latter is constrained by limited data availability. To address these challenges, we propose a universal PET-CT segmentation framework, termed DpDNet. The overall architecture of DpDNet is shown in Fig. 1 , with details introduced in the following subsections. 2.2 Encoder-decoder backbone The backbone of DpDNet is based on STU-Net [ 11 ] ,

Read the complete text