USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images

Jiamu Wang, JinTae Kwak · 2025

Read USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images by Jiamu Wang, JinTae Kwak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16160.txt.

Introduction

Jiamu Wang and Jin Tae Kwak present USegMix, a 2025 method for making more training images for pathology AI without new labeling. Labeled tissue image sets are scarce, which starves learning models. Their method builds new images in two phases. First it harvests a pool of tissue segments automatically, using superpixels and a general segmentation model. Then it swaps a segment in a target image for a similar pooled segment and blends the seam with a diffusion model.

The synthetic images look diverse and realistic. On colorectal and prostate cancer data sets, training with the added images improved cancer classification. The authors present the method as an efficient way to stretch small pathology collections. Better data volume, they argue, is one of the cheapest paths to stronger medical image models.

Catalog record

CREATORJiamu Wang, JinTae Kwak
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16160

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images Jiamu Wang, JinTae Kwak Abstract In computational pathology, researchers often face challenges due to the scarcity of labeled pathology datasets. Data augmentation emerges as a crucial technique to mitigate this limitation. In this study, we introduce an efficient data augmentation method for pathology images, called USegMix. Given a set of pathology images, the proposed method generates a new, synthetic image in two phases. In the first phase, USegMix constructs a pool of tissue segments in an automated and unsupervised manner using superpixels and the Segment Anything Model (SAM). In the second phase, USegMix selects a candidate segment in a target image, replaces it with a similar segment from the segment pool, and blends them by using a pre-trained diffusion model. In this way, USegMix can generate diverse and realistic pathology images. We rigorously evaluate the effectiveness of USegMix on two pathology image datasets of colorectal and prostate cancers. The results demonstrate improvements in cancer classification performance, underscoring the substantial potential of USegMix for pathology image analysis. Figure 1: Comparison among USegMix(ours) and CutOut, MixUp, CutMix. 1 Introduction Computational pathology is a transformative approach in pathology research and practice, utilizing advanced computational techniques such as machine learning, deep learning, and digital imaging to improve disease diagnosis and management [ 8 ] . Despite its potential, it faces challenges such as the scarcity of annotated pathology datasets. Reliance on expert annotations and ethical considerations regarding patient consent limits the availability of diverse data, affecting the generalizability of computational models and hindering their application in real-world clinical settings. To address data limitations in pathology, data augmentation has become essential for enhancing dataset size and diversity. Early methods used simple manipulations like image flipping, rotation, and cropping to introduce variability. Later, more advanced mix-based techniques emerged; for instance, CutOut [ 9 ] simulates dropout by masking parts of the image, MixUp [ 10 ] blends two images in the alpha channel, and CutMix [ 11 ] replaces a region in one image with content from another to create a composite image. Style Transfer [ 12 ] merges the content of one image with the style of another, preserving content while adding stylistic variations. More recently, Attentive CutMix [ 13 ] replaces the most descriptive area according to the attention maps, TransMix [ 14 ] mixes labels based on the attention maps of Vision Transformers, ClassMix [ 15 ] , designed for semantic segmentation, mixes samples based on segmentation boundaries, CarveMix [ 16 ] creates new brain lesion images by copying and pasting lesions, GradMix [ 17 ] and InsMix [ 18 ] increase the diversity of nuclei by employing the copy-and-paste approach. These methods aim to provide richer, more varied training samples, overcoming traditional augmentation limitations. However, it is likely that such techniques introduce artifacts and thus create unrealistic images, which is a critical issue in computational pathology, as shown in Fig. 1 . In this study, we introduce USegMix, an efficient augmentation method designed to enhance the realism and diversity of pathology image datasets. USegMix creates a new, synthetic image in two phases. First, it decomposes pathology images into a set of segments in an unsupervised manner using superpixels and the Segment Anything Model (SAM) [ 1 ] , further constructing a tissue segment pool. Second, it selects a candidate tissue segment in a target image, replaces the candidate segment with a semantically similar segment, belonging to the same histologic category, from the tissue segment pool, and seamlessly blends the target image and the selected segment using a pre-trained diffusion model [ 2 , 3 , 4 , 5 , 6 ] . In this manner, USegMix can generate a variety of synthetic images that are realistic and semantically consistent without additional training on a target dataset. We evaluated USegMix using two public cancer datasets. The results demonstrate that USegMix increases the size and diversity of pathology datasets and improves the performance of downstream tasks, highlighting its potential to enhance computational pathology through advanced data augmentation. Figure 2: The two-phase procedure of USegMix. In phase 1, it first conducts SAM-based unsupervised segmentation to produce the segmentation map for each image, extracts features for each segment, and builds a tissue segment pool. In phase 2, it randomly selects a segment (Segment A) to replace, selects a similar segment (Segment B) from the segment pool, replaces Segment A with Segment B, and conducts inpainting using a pre-trained diffusion model to enhance the realism of the resultant image. 2 Methods USegMix conducts data augmentation in two phases. In the first phase, it performs unsupervised image segmentation to build a tissue segment pool using superpixels and Segment Anything Model (SAM) [ 1 ] . In the second phase, it mixes and blends segments from different images by adopting a diffusion model via image inpainting. Fig. 2 illustrates the procedure of USegMix. 2.1 USegMix Phase 1 - Segment Pool Preparation SAM-based Unsupervised Segmentation. SAM stands as a foundation model trained on over one billion data samples, which endows it with a robust zero-shot capability for segmentation tasks. SAM encodes the inputs of an image and a prompt such as masks, points, or boxes, and returns the decoded resultant masks. Since SAM is designed in a user-interactive manner, users need to specify regions of interest to conduct segmentation, limiting its usability. USegMix, however, combines superpixels and SAM to enable an automated generation of segmentation maps, thereby bypassing the need for manual prompts. The segmentation process begins with a SLIC-based superpixel method [ 20 ] that outlines preliminary segmentation maps S={s_{i},i=1,...,N_{s}} where s_{i} is the i th superpixel and N_{s} is the number superpixels ( N_{s} =30). For a superpixel s_{i} , we randomly sample a point within s_{i} and use it as a prompt to SAM to generate a segment mask. By repeating it K times with differing random points, we obtain K segment masks, producing \{m_{i,j}|j=1,...,K\} where m_{i,j} is the segment mask for the j th random point (K=15). For each m_{i,j} , we compute an identity vector (X(m_{i,j}),Y(m_{i,j}),\sqrt{CNT(m_{i,j})}) where X(.) and Y(.) denote the center coordinates and CNT(.) is the size of the segment mask. Using the identity vectors and L2 distance, we group the segment masks into a number of clusters, choose the largest cluster, and identify the most frequent segment mask as the anchor segment \alpha_{i} . After iterating this process across all superpixels, we remove the duplicate segments and obtain a refined segmentation map \cup_{i=1}\alpha_{i} in an automated manner. The segmentation procedure is illustrated in Fig. 3 . Figure 3: The procedure of SAM-based unsupervised segmentation in USegMix phase 1. Given an image, it generates superpixels, selects a target superpixel, samples a random point within the target superpixel, feeds SAM with the random point as a prompt for K times, and collects the resultant masks from SAM. Then, it clusters the resultant K masks based on their similarity and selects the most prevalent mask as the anchor mask to represent the superpixel. Feature Representation. For each image patch of class c , each anchor segment \alpha_{i} is cropped using its bounding box, resized into the size of 224x224, and fed into a feature

Read the complete text