Policy to Assist Iteratively Local Segmentation: Optimising Modality and Location Selection for Prostate Cancer Localisation

Xiangcen Wu, Shaheer U. Saeed, Yipei Wang, Ester Bonmati Coll, Yipeng Hu · 2025

Read Policy to Assist Iteratively Local Segmentation: Optimising Modality and Location Selection for Prostate Cancer Localisation by Xiangcen Wu, Shaheer U. Saeed, Yipei Wang, Ester Bonmati Coll, Yipeng Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03953.txt.

Introduction

Xiangcen Wu and co-authors build a recommender that tells a segmentation model where to look next in prostate scans. Radiologists hop between scan types and local regions as they read, weighing each view on its own and together. The team's policy network learns to mimic that dance, proposing the best modality and patch for each step.

A pretrained segmentation network then reads only what the policy picks, and its local results feed the next round. The loop runs until the cancers are pinned down as well as possible. Tests used 1,325 labeled multi-sequence MRI scans from prostate patients. The assisted search beat standard segmentation networks, above all on hard cases. Curiously, the agent's own best strategy drifted from the PI-RADS habits radiologists follow.

Catalog record

CREATORXiangcen Wu, Shaheer U. Saeed, Yipei Wang, Ester Bonmati Coll, Yipeng Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-03953

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Policy to Assist Iteratively Local Segmentation: Optimising Modality and Location Selection for Prostate Cancer Localisation Xiangcen Wu, Shaheer U. Saeed, Yipei Wang, Ester Bonmati Coll, Yipeng Hu Abstract Radiologists often mix medical image reading strategies, including inspection of individual modalities and local image regions, using information at different locations from different images independently as well as concurrently. In this paper, we propose a recommend system to assist machine learning-based segmentation models, by suggesting appropriate image portions along with the best modality, such that prostate cancer segmentation performance can be maximised. Our approach trains a policy network that assists tumor localisation, by recommending both the optimal imaging modality and the specific sections of interest for review. During training, a pre-trained segmentation network mimics radiologist inspection on individual or variable combinations of these imaging modalities and their sections - selected by the policy network. Taking the locally segmented regions as an input for the next step, this dynamic decision making process iterates until all cancers are best localised. We validate our method using a data set of 1325 labelled multiparametric MRI images from prostate cancer patients, demonstrating its potential to improve annotation efficiency and segmentation accuracy, especially when challenging pathology is present. Experimental results show that our approach can surpass standard segmentation networks. Perhaps more interestingly, our trained agent independently developed its own optimal strategy, which may or may not be consistent with current radiologist guidelines such as PI-RADS. This observation also suggests a promising interactive application, in which the proposed policy networks assist human radiologists. The code is open-sourced and available at here . 1 Introduction Accurate annotation of tumors in prostate cancer plays a crucial role in targeted biopsy [ 14 , 1 , 8 , 12 ] , focal therapy [ 2 , 17 ] and emerging MRI-targeted radiotherapy [ 9 ] . High-quality annotations help in treatment planning, monitoring disease progression, and training deep learning models for automated detection [ 3 , 11 ] . However, this process is time-consuming and labor-intensive for radiologists. They must analyse multi-parametric MRI scans in multiple aspects, for example, to accurately identify the tumor location and its boundary as well as to select the appropriate imaging modality. Since certain lesions or lesion areas may only be visible in a specific modality, switching between modalities is often necessary for cancer annotation. This is in contrast to existing deep learning models for prostate cancer detection [ 11 ] , which often adopt feed-forward neural networks with one-pass inference unable to model such a dynamic decision making process. Arguably, data-driven approaches may be capable of learning such a complex decision-making. In this application of localising prostate cancer, however, the performance has been highly limited due to practical constraints such as quality and quantity of the available labeled data [ 7 ] . In this work, we focus on using reinforcement learning (RL) to detect cancer on MRI in contrast to conventional supervised cancer segmentation models. Firstly, it models the above-discussed dynamic decision making which optimises the agent’s cancer detection strategy based on changes of state (e.g., here, the images, selected sections, MRI sequences and current segmentation). Secondly, it leverages the optimised trajectories of “actions” for improving interoperability. Thirdly, the agent we train can potentially surpass the supervised segmentation model due to exploration and trial-and-error based learning as opposed to only learning to imitate human labels. Additionally, RL enables multi-step decision processes, such as selecting the most informative imaging modality and annotating cancerous regions portion by portion. In practice, the agent can be integrated to provide actionable guidance to radiologists while labeling cancerous region on MRI, rather than used alone to provide a final prediction. Although not discussed further, potential wider applications include providing feedback in clinician training, interactive assistance and enhancing transparency, trustworthiness and repeatability in clinical practice. Outside of prostate cancer detection, previous work also explored RL approaches for medical image computing applications, such as anatomical segmentation [ 15 ] and pathology detection [ 4 ] . However, most of these existing RL-based algorithms aimed at improving detection performance, without explicitly modeling a human actionable environment. In contrast, we propose a policy network for RL agent, and a second segmentation network, separately developed and trained as a ‘simulated radiologist’. The simulated radiologist is trained to recognise and process different input modalities at various portions of the image volume. These image portions could be represented by image patches or a subset of slices in a volumetric image. These segmentation outputs are then used to give reward to the policy network of our agent and optimises its strategy. We highlight that optimisation of the policy network relies on rewards derived from segmentation accuracy across different modalities and portions, as provided by the simulated radiologist. This process of trial and error allows our agent to explore different paths and therefore potentially discovers different strategies that may result in different and better cancer localisation performance. To summarise, we introduced RL for cancer detection in MRI, contrasting with conventional supervised segmentation approaches, where the iterative learning process enables the agent to discover interesting workflows for modality selection and image portions to be focused on. We proposed a policy network for RL that interacts with a separately trained segmentation network that serves as a surrogate reward function, improving detection accuracy across different MRI modalities. Clinical data is used for a real-world application, highlighting its potential integration into radiology workflows. This iterative learning process enables the agent to discover interesting paths for cancer detection. Open source code is provided for reproducibility and further research here . 2 Method We formulate the prostate cancer localisation problem as a Markov Decision Process (MDP) where a radiologist first selects the imaging modality, and portion of the anatomy, which are likely to yield the best local segmentation. The radiologist then segments the cancer on the selected portion of the anatomy using the selected modality. This process is repeated until all the cancerous regions have been segmented for all portions of the anatomy. We formulate this decision process as a RL problem which aims to find the best patient-specific strategy for imaging modality selection such that cancer segmentation performance is maximised. To this end, we simulate the radiologist’s workflow for segmenting cancer on a given medical image, and discover an optimal strategy using RL. 2.1 Simulated radiologist during training A multi-modal volumetric image sample x\in\mathbb{R}^{H\times W\times D\times C} is composed of C co-registered volumes of size H\times W\times D , each corresponding to a different imaging modality. An image potion 1 1 1 Without loss of generality, an image portion in this study refers to a subset of slices in a cropped single-modality image, cropped to contain at least the prostate gland. Further experimental details are described in Sec 3.1 is denoted as x_{p}\in\mathcal{X}_{p}\in\mathbb{R}^{h\times w\times d\times 1} , where h\times w\times d denote the dimensions of the image portion, with h\leq H , w\leq W and d\leq D , 1 denotes the single modality that is being used for segmentation, and \mathcal{X}_{p} denotes the distribution of image portions. To simulate radiologist segmentation during RL training, we construct a deep neural network f(\cdot;\theta):\mathcal{X}_{p}\rightarrow\mathcal{Y}_{p} , which predicts a segmentation y_{p}\in\mathcal{Y}_{p} for an image portion x_{p}\in\mathcal{X}_{p} , and has

Read the complete text