Assessing the generalization performance of SAM for ureteroscopy scene understanding
Martin Villagrana ,†, Francisco Lopez-Tiro ,†, Clement Larose, Gilberto Ochoa-Ruiz, Christian Daul · 2025
Read Assessing the generalization performance of SAM for ureteroscopy scene understanding by Martin Villagrana ,†, Francisco Lopez-Tiro ,†, Clement Larose, Gilberto Ochoa-Ruiz, Christian Daul (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17210.txt.
Introduction
Martin Villagrana and co-authors test whether a general-purpose image segmentation model can outline kidney stones in scope images without special training, in a 2025 paper. Drawing stone borders by hand is slow and impractical across the large image sets that urology produces. The model under test, SAM, is known for segmenting almost anything it is pointed at.
The team raced SAM against three U-Net variants, the standard tools for medical outlining, on both familiar and unseen data. On familiar data SAM matched the best U-Net closely. On new data it pulled far ahead, beating every U-Net variant by margins of up to 23 percent. The authors read this as evidence that a general model can travel between hospitals and cameras better than narrow ones, a key step toward automatic stone typing.
Catalog record
| CREATOR | Martin Villagrana ,†, Francisco Lopez-Tiro ,†, Clement Larose, Gilberto Ochoa-Ruiz, Christian Daul |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17210
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Assessing the generalization performance of SAM for ureteroscopy scene understanding Martin Villagrana ,†, Francisco Lopez-Tiro ,†, Clement Larose, Gilberto Ochoa-Ruiz, Christian Daul Abstract The segmentation of kidney stones is regarded as a critical preliminary step to enable the identification of urinary stone types through machine- or deep-learning-based approaches. In urology, manual segmentation is considered tedious and impractical due to the typically large scale of image databases and the continuous generation of new data. In this study, the potential of the Segment Anything Model (SAM)—a state-of-the-art deep learning framework—is investigated for the automation of kidney stone segmentation. The performance of SAM is evaluated in comparison to traditional models, including U-Net, Residual U-Net, and Attention U-Net, which, despite their efficiency, frequently exhibit limitations in generalizing to unseen datasets. The findings highlight SAM’s superior adaptability and efficiency. While SAM achieves comparable performance to U-Net on in-distribution data (Accuracy: 97.68 ± 3.04; Dice: 97.78 ± 2.47; IoU: 95.76 ± 4.18), it demonstrates significantly enhanced generalization capabilities on out-of-distribution data, surpassing all U-Net variants by margins of up to 23%. 1 Introduction Kidney stone formation is recognized as a prevalent condition affecting millions of individuals globally. In the United States, lifetime prevalence rates have been reported to range from 7.2% to 10.1%, while higher rates—such as 21.1%—have been observed in specific populations [ 1 ] . Early detection and accurate diagnosis of kidney stone formation are essential to ensure the prescription of effective treatments and to mitigate complications, including renal damage, infections, and recurrent stone formation [ 2 ] . Through timely intervention, patient outcomes can be significantly improved, and healthcare costs may be reduced. Due to its high incidence and association with severe pain, the detection of kidney stones is considered a critical component of medical practice, particularly for patients with recurrent episodes or those at elevated risk [ 3 ] . The current diagnosis of kidney stones is primarily based on Morpho-Constitu-tional Analysis (MCA), a process that depends extensively on visual inspection and biochemical analysis performed by clinicians [ 4 ] . Consequently, numerous attempts have been made to support this diagnostic process through computer vision-based approaches. Recent advances in deep learning (DL) have demonstrated promising potential for improving kidney stone classification [ 5 ] , [ 6 ] , [ 7 ] . These methods could accelerate diagnosis by automating the traditionally operator-dependent MCA when conducted during endoscopic stone recognition (ESR) procedures [ 8 ] , a critical advancement for patients requiring immediate intervention. Accurate segmentation of detected kidney stones is regarded as an essential preliminary step prior to classification, as it allows the stone to be isolated from surrounding tissues, thereby enhancing diagnostic accuracy [ 9 ] . Deep learning techniques, particularly convolutional neural networks (CNNs) and transformer-based models, have exhibited notable efficacy in segmentation tasks, as relevant features within complex medical images can be automatically identified, improving the reliability of image-based diagnosis [ 10 ] , [ 11 ] . Traditional segmentation models, including U-Net and its variants (Attention U-Net and Residual U-Net), have demonstrated strong performance in image segmentation tasks. The U-Net architecture, while recognized for its limited data efficiency, has been shown to effectively preserve spatial information through its encoder-decoder structure, though challenges remain when processing complex or noisy images [ 12 ] . Residual U-Net, which incorporates residual connections, has been designed to mitigate issues such as vanishing gradients, thereby enhancing performance in deeper network architectures; however, its increased depth can result in longer training times [ 13 ] . Attention U-Net enhances traditional U-Net through the integration of attention mechanisms, enabling the model to concentrate on diagnostically relevant regions, though this improvement comes at the cost of higher computational requirements [ 14 ] . More recently, transformer-based models like the Segment Anything Model (SAM) [ 15 ] have been developed, offering notable adaptability across diverse image types without necessitating extensive task-specific modifications. Despite SAM’s promising generalization capabilities, its implementation remains constrained by computational complexity and significant resource demands. The segmentation of kidney stone images obtained from ureteroscopy videos has traditionally been conducted manually by clinical experts (Fig. 1(a) ). While this approach has proven effective for limited datasets, its application to large-scale databases has been recognized as impractical due to the time-consuming nature of the process and its inherent dependence on operator skill [ 8 ] . These limitations could potentially be addressed through advanced deep learning models like SAM, which have been shown to enable accurate and efficient automated segmentation of extensive video collections [ 15 ] . SAM’s demonstrated capability to generalize across heterogeneous datasets represents a substantial technological advancement, positioning it as a viable alternative to conventional manual segmentation methods for large-scale applications (Figure 1(b) ). (a) (b) Figure 1 : Comparison framework for kidney stone segmentation methods. In traditional approaches (top), frames are extracted from ureteroscopy videos to create Dataset D{{}_{i}} (distribution i), followed by manual expert labeling. The Segment Anything Model pipeline (bottom) is trained on i and its labels, then performs inference on a distinct unlabeled Dataset j (distribution j). Critical note: D{{}_{i}} and D{{}_{i}} represent different data distributions. The principal contributions of this work are summarized as follows: (i) Demonstrated Superior Generalization of SAM. The Segment Anything Model is shown to exhibit exceptional generalization capabilities for kidney stone segmentation tasks, systematically outperforming traditional architectures (U-Net, Residual U-Net, Attention U-Net). While conventional models exhibit significant performance degradation when applied to unseen data, SAM maintains robust accuracy across both known and unknown distributions when trained on identical datasets. This advancement highlights SAM’s potential as a reliable tool for renal image analysis under real-world clinical variability. (ii) Validated Multi-Class Adaptability Without Retraining. A novel three-class SAM variant ("kidney stone," "laser fiber," and "surrounding tissue") is validated, achieving high performance across both known and unknown distributions—a first in renal segmentation literature. Crucially, this adaptability is accomplished without requiring model retraining, as precision is preserved for both two-class and three-class tasks. This flexibility addresses a critical clinical challenge, where segmentation needs often vary across procedures and patient-specific anatomical presentations. This work is organized as follows: Section 2 details the experimental dataset and methodological framework, with particular emphasis on comparative analysis between conventional deep learning-based segmentation approaches and the SAM. Section 3 presents and analyzes the experimental findings, highlighting key performance comparisons. The study concludes with future research directions and final remarks in Section 4 . 2 Materials and Methods 2.1 Clinical image datasets Four distinct kidney stone datasets were utilized in our experimental framework (Fig. 2 ). The image data were acquired through two primary modalities: (1) conventional CCD camera systems and (2) endoscopic imaging captured during ureteroscopic procedures. The key characteristics of these datasets are detailed as follows: Figure 2 : The four datasets (distributions) of kidney stone images are displayed alongside their corresponding segmentation masks, which include the stone and laser fiber. From right to left: Dataset A (in vivo endoscopic), Dataset B (ex vivo endoscopic), Dataset C (in vivo endoscopic), and Dataset D (ex vivo CCD camera). All datasets enable two-class segmentation (kidney stone and tissue), while only Datasets A and C include a third class (laser). Database A ( D_{A} ): In-vivo Endoscopic Images. This dataset comprises 156 in-vivo endoscopic