RadSAM: Segmenting 3D radiological images with a 2D promptable model

Julien Khlaut, Elodie Ferreres, Daniel Tordjman, Hélène Philippe, Tom Boeken, Pierre Manceron, Corentin Dancette · 2025

Read RadSAM: Segmenting 3D radiological images with a 2D promptable model by Julien Khlaut, Elodie Ferreres, Daniel Tordjman, Hélène Philippe, Tom Boeken, Pierre Manceron, Corentin Dancette (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20837.txt.

Introduction

Julien Khlaut and six co-authors present RadSAM, in a 2025 preprint, a way to outline organs in 3D scans while prompting only once. Outlining anatomy in CT and MR images is slow, skilled work where mask accuracy matters. The Segment Anything Model made outlining interactive, but it was trained on ordinary photos and works slice by slice. A 3D organ could need dozens of prompts, and editing tools were thin. RadSAM trains a 2D model that also accepts a rough prior mask as a prompt, alongside boxes and points.

At run time, one prompt starts an iterative pass that rebuilds the 3D mask slice by slice. The authors add a benchmark for single-prompt 3D outlining in CT. On an abdominal organ dataset, RadSAM beat leading rivals. It also carried to new scan types and supports editing afterward.

Catalog record

CREATORJulien Khlaut, Elodie Ferreres, Daniel Tordjman, Hélène Philippe, Tom Boeken, Pierre Manceron, Corentin Dancette
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20837

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RadSAM: Segmenting 3D radiological images with a 2D promptable model Julien Khlaut, Elodie Ferreres, Daniel Tordjman, Hélène Philippe, Tom Boeken, Pierre Manceron, Corentin Dancette Abstract Medical image segmentation is a crucial and time-consuming task in clinical care, where mask precision is extremely important. The Segment Anything Model (SAM) offers a promising approach, as it provides an interactive interface based on visual prompting and edition to refine an initial segmentation. This model has strong generalization capabilities, does not rely on predefined classes, and adapts to diverse objects; however, it is pre-trained on natural images and lacks the ability to process medical data effectively. In addition, this model is built for 2D images, whereas a whole medical domain is based on 3D images, such as CT and MRI. Recent adaptations of SAM for medical imaging are based on 2D models, thus requiring one prompt per slice to segment 3D objects, making the segmentation process tedious. They also lack important features such as editing. To bridge this gap, we propose RadSAM, a novel method for segmenting 3D objects with a 2D model from a single prompt. In practice, we train a 2D model using noisy masks as initial prompts, in addition to bounding boxes and points. We then use this novel prompt type with an iterative inference pipeline to reconstruct the 3D mask slice-by-slice. We introduce a benchmark to evaluate the model’s ability to segment 3D objects in CT images from a single prompt and evaluate the models’ out-of-domain transfer and edition capabilities. We demonstrate the effectiveness of our approach against state-of-the-art models on this benchmark using the AMOS abdominal organ segmentation dataset. 1 Introduction Image segmentation is an essential task for medical imaging [ 13 ] : it allows specialists to compute multiple metrics regarding anatomical and pathological objects useful for clinical care. The manual segmentation process is long, error-prone, subjective, and led to the adoption of simplified criteria such as RECIST [ 4 ] to avoid segmenting whole 3D structures. Models that streamline time-consuming tasks, such as estimating the volume of a liver prior to treatment or quantifying a patient’s cancer burden through extensive tumor segmentation, help physicians in their everyday decision-making and reduce inconsistencies between multiple annotators. Deep-learning-based methods for organs and tumors have been studied extensively; most of them are trained on specific datasets using U-net architectures [ isensee2021nnu , 5 , 11 , 19 ] . However, standard semantic segmentation models are subject to limitations in their clinical usage. First, once the model outputs a segmentation map for a given image, it has no mechanism to correct it. In case of an error, the user has to modify it manually, which is time-consuming. Recently, the Segment Anything Model (SAM) [ 8 ] was introduced and proposed a prompted segmentation framework, allowing users to interactively guide the model with a spatial prompt to obtain an initial segmentation. Users can then provide correction points, helping the model refine the mask. Moreover, being trained on very large and diverse datasets, SAM has the ability to segment objects that are not necessarily present in the initial dataset. It is, therefore, considered a foundation model for segmentation. This framework is promising for medical image segmentation, as it allows the precise segmentation of diverse structures, with more consistency and speed than manual segmentation. However, the original Segment Anything model (SAM) was shown to be unreliable for medical data [ 10 , 18 ] . Thus, Ma et al. proposed MedSAM [ 9 ] , a fine-tuned SAM for medical segmentation. SAM-Med 2D [ 2 ] , Medical SAM Adapter [ 16 ] and SA-Med2D [ 17 ] feature similar approaches. Yet, they lack important SAM features, as shown in Table 1 . For instance, MedSAM only supports bounding-box prompts and does not implement SAM’s editing mode. More importantly, in radiology, high-quality imaging is produced in 3D, with Magnetic resonance imaging (MRI) and Computed Tomography (CT), which requires models to segment 3D structures. Previous SAM-based models only propose a 2D segmentation method, requiring one prompt for each 2D slice to segment full 3D objects, with potentially noncoherent segmentation for consecutive slices. Some work, such as SAM-Med-3D [ 14 ] , propose a model that directly segments 3D medical images from the full volume. However, such a 3D model has high memory GPU constraints, making it hard to train and deploy. Indeed, during training, the model needs to have access to the whole volume, which uses more memory than the 2D-based model. It also outputs a lower resolution mask, 128x128, compared to the standard 256x256 outputted by SAM. We chose to focus on 2D models because they are: easy to implement and train, efficient, and produce higher-resolution masks. In the present work, we introduce RadSAM (for Radiological SAM). This promptable segmentation model can segment 3D structures in CT images from a single prompt, using the memory footprint of a 2D model. Our method is simple yet effective: in addition to points and box prompts, we train a 2D segmentation model with a mask prompt as the first input. The model learns to reconstruct the ground-truth mask from a degraded input. We then introduce an iterative inference method to leverage the novel mask prompt and forward instructions from one slice to the next with minimal information loss. This allows us to segment 3D radiological objects from a single prompt and to add 3D editing capabilities. We benchmark our promptable segmentation model on 3D imaging, using AMOS [ 7 ] and TotalSegmentator [ 15 ] , two CT organ segmentation datasets. We evaluate its capacity to segment from multiple prompt types, transferability to other datasets, and edition performances. We will release our model weights upon acceptance of the paper. Model Domain 3D Prompt Correction SAM Natural ✗ Bbox / Point ✓ MedSAM Medical ✗ Bbox ✗ RadSAM Medical ✓ Bbox / Point † ✓ Table 1: Different settings and characteristics of the SAM-like models. † RadSAM is also trained with mask prompts, but they are not used as the first prompt during inference. 2 Benchmark 2.1 Datasets To produce our benchmark, we consider only models trained on the AMOS dataset [ 7 ] . It comprises a large and diverse collection of clinical CT scans for abdominal organ segmentation and some MRIs. We only use the CT part containing 500 CT scans and voxel-level annotations of 15 abdominal organs. In addition to AMOS, we use another dataset for additional experiments: TotalSegmentator (TS) [ 15 ] : This dataset consists of 1204 CT scans with detailed annotations of 104 anatomical structures (27 organs, 59 bones, 10 muscles, and 8 vessels). 2.2 Input prompts The benchmark evaluates the model’s response to different prompting strategies, as detailed in Figure 1 . We consider 2 approaches. The first one is slice-level prompting, which consists of predicting a mask for each slice of interest by giving a prompt on each slice. The second one is volume-level prompting; this consists of a unique 2D prompt on one slice and 2 boundary annotations indicating the maximum and minimum slices. We also study the models’ ability to respond to edition prompts. For the 2D slice-level prompting, we add a point to each of the slices, whereas

Read the complete text