Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology
Saghir Alfasly, Wataru Uegami, MD Enamul Hoq, Ghazal Alabtah, H.R. Tizhoosh · 2025
Read Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology by Saghir Alfasly, Wataru Uegami, MD Enamul Hoq, Ghazal Alabtah, H.R. Tizhoosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-17847.txt.
Introduction
Saghir Alfasly and 4 co-authors make fake tissue images to train medical image tools. Their 2025 paper deals with pathology slides, where tissue types mix in fine detail. Hand-tagged slides are scarce, and plain text prompts miss the true look of each tissue type.
Their diffusion model takes both a map of tissue zones and real crops from those zones. For untagged slide sets, it first groups slides into many tissue types to build maps on its own. Fake images from the plan scored close to real ones on later tasks. Scores for image likeness also rose a lot.
Catalog record
| CREATOR | Saghir Alfasly, Wataru Uegami, MD Enamul Hoq, Ghazal Alabtah, H.R. Tizhoosh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-17847
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology Saghir Alfasly, Wataru Uegami, MD Enamul Hoq, Ghazal Alabtah, H.R. Tizhoosh Abstract Synthetic data generation in histopathology faces unique challenges: preserving tissue heterogeneity, capturing subtle morphological features, and scaling to unannotated datasets. We present a latent diffusion model that generates realistic heterogeneous histopathology images through a novel dual-conditioning approach combining semantic segmentation maps with tissue-specific visual crops. Unlike existing methods that rely on text prompts or abstract visual embeddings, our approach preserves critical morphological details by directly incorporating raw tissue crops from corresponding semantic regions. For annotated datasets (i.e., Camelyon16, Panda), we extract patches ensuring 20 − 80 % 20-80\% tissue heterogeneity. For unannotated data (i.e., TCGA), we introduce a self-supervised extension that clusters whole-slide images into 100 tissue types using foundation model embeddings, automatically generating pseudo-semantic maps for training. Our method synthesizes high-fidelity images with precise region-wise annotations, achieving superior performance on downstream segmentation tasks. When evaluated on annotated datasets, models trained on our synthetic data show competitive performance to those trained on real data, demonstrating the utility of controlled heterogeneous tissue generation. In quantitative evaluation, prompt-guided synthesis reduces Fréchet Distance by up to 6 × 6\times on Camelyon16 (from 430.1 430.1 to 72.0 72.0 ) and yields 2 − 3 × 2-3× lower FD across Panda and TCGA. Downstream DeepLabv3+ models trained solely on synthetic data attain test IoU of 0.71 0.71 and 0.95 0.95 on Camelyon16 and Panda, within 1 − 2 % 1-2\% of real-data baselines ( 0.72 0.72 and 0.96 0.96 ). By scaling to 11,765 11,765 TCGA whole-slide images without manual annotations, our framework offers a practical solution for an urgent need for generating diverse, annotated histopathology data, addressing a critical bottleneck in computational pathology. 1 Introduction Histopathology image analysis forms the cornerstone of cancer diagnosis, yet remains constrained by data scarcity, laborious annotation processes, and privacy concerns campanella2019clinical ; komura2018machine ; louis2014digital ; abels2019computational ; kaissis2020secure . While generative AI has transformed natural image synthesis, its application to histopathological imaging in digital pathology faces unique challenges due to the complex, multi-scale architecture of biological tissues and the critical importance of preserving diagnostically relevant features wei2019generative ; gadermayr2019generative ; afshari2023single . Most of the current approaches have predominantly focused on generating homogeneous tissue types using text-based prompting systems, which introduce significant interobserver variability and limit clinical utility, particularly problematic in a domain where expert agreement is already inconsistent elmore2015diagnostic ; tizhoosh2021searching . The evolution from GAN-based methods to diffusion models has improved image quality and training stability dhariwal2021diffusion ; ho2020denoising , but existing frameworks fail to account for the heterogeneous nature of real-world histopathology samples. Levine et al. levine2020synthesis demonstrated that GANs could generate images indistinguishable from real histopathology samples, while Moghadam et al. moghadam2023morphology marked the transition to diffusion models with superior image quality and training stability. Despite these advances, clinical specimens typically contain multiple tissue types and pathological features within a single slide, requiring region-specific control over generation parameters. This limitation severely restricts the utility of synthetic data for developing robust diagnostic algorithms that must recognize complex patterns in diverse tissue regions. Figure 1: Comparison of reference and synthetic Camelyon16 patches using three conditioning schemes. From left to right: original histopathology patch; binary tumor–normal mask; generated image with a semantic segmentation map conditioning only; synthetic image by combining conditions of semantic maps and abstract embeddings; synthetic output of our model conditioned on the semantic map and tissue-specific visual crop prompts. The crop-guided generation recovers fine morphological details and staining heterogeneity more faithfully than embedding-based conditioning. Further complicating this landscape is the tension between the global architecture of the tissue and the local cellular details. Traditional generative approaches struggle to maintain consistency across multiple magnification levels while preserving the fine-grained morphological features critical for diagnosis quiros2021pathology ; tellez2019quantifying ; afshari2023single . Recent work, URCDM cechnicka2024urcdm , has addressed this through cascaded diffusion models generating images at multiple resolutions simultaneously, while DiffInfinite aversa2024diffinfinite enables arbitrary-size image synthesis with preserved long-range structural correlations. However, this multiscale challenge remains particularly acute when attempting to synthesize realistic transitions between different tissue types, a capability essential for training segmentation algorithms and supporting differential diagnosis. Our work addresses these challenges by introducing a visually prompted latent diffusion model designed specifically for heterogeneous tissue synthesis in histopathology. Unlike text-guided approaches, our framework leverages spatial masks and visual exemplars to provide fine-grained control over region-specific generation. Previous work, in the literature, NASDM shrivastava2023nasdm and subsequent extensions by Konz et al. konz2024segguided and Xu et al. xu2024topocellgen demonstrated the value of region-guided generation and semantic instance masks. Our method expands this concept to enable the synthesis of complex, multi-tissue samples with realistic transitions and clinically relevant features as shown in Figure 1 . Our approach significantly reduces the annotation burden while maintaining high fidelity to real-world pathological presentations 2 . The main contributions of our work include: • New Dual-Conditioning Architecture for Histopathology Synthesis. We developed a unique visual crop-guided diffusion model that combines semantic maps with raw tissue exemplars, preserving critical morphological features (nuclear texture, staining patterns, cellular structure) that are lost in text-based or embedding-based approaches. This enables precise region-specific control while maintaining authentic tissue appearance for heterogeneous sample generation. • Self-Supervised Framework for Unannotated Whole-Slide Images. We introduced a scalable solution for the TCGA dataset (11,765 WSIs) that automatically discovers and clusters 100 distinct tissue phenotypes without manual annotation. This democratizes access to diverse synthetic data across 33 cancer types while preserving patient privacy and addressing critical data scarcity in computational pathology. • Comprehensive Multi-Modal Validation Framework. We established a rigorous evaluation pipeline combining quantitative metrics (Fréchet distance across 8 foundation model encoders) with downstream task performance (segmentation IoU scores). Most notably, our blinded assessment by certified pathologists using a 5-point Likert scale across multiple quality criteria revealed that our synthetic images were indistinguishable from real samples, with one pathologist commenting: "The generated images tended to have equal or higher quality than the real images." Through rigorous evaluation involving both quantitative metrics (including the Fréchet Inception Distance borji2022pros ) and expert pathologist evaluation, we demonstrate that our approach generates histopathology images that are indistinguishable from real samples while providing unprecedented control over tissue composition . Synthetic datasets generated using our method effectively augment or replace real data in training diagnostic models, addressing the critical issue of data scarcity while preserving patient privacy. By enabling the generation of diverse, annotated histopathology datasets without requiring patient data sharing, our framework represents a significant step toward more equitable and robust AI development in computational pathology larson2021ethics . This capability is particularly valuable for rare cancer types and underrepresented populations, potentially democratizing access to high-quality training data between institutions, regardless of their size or resources. 2 Related Work Generative Models in Histopathology. Visual generative models have evolved from early GANs goodfellow2014generative to sophisticated diffusion models ho2020denoising , with recent advances like ControlNet zhang2023adding enabling fine-grained control. Adapting these to histopathology presents unique challenges due to the complexity of the tissue and the diagnostic significance of subtle morphological features. Although early GANs demonstrated feasibility levine2020synthesis ; quiros2021pathology ; afshari2023single , recent diffusion-based approaches show superior quality moghadam2023morphology . Domain-specific methods ktena2024generative