HistDiST: Histopathological Diffusion-based Stain Transfer

Erik Großkopf ∗, Valay Bundele ∗, Mehran Hosseinzadeh ∗, Hendrik P.A. Lensch · 2025

Read HistDiST: Histopathological Diffusion-based Stain Transfer by Erik Großkopf ∗, Valay Bundele ∗, Mehran Hosseinzadeh ∗, Hendrik P.A. Lensch (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06793.txt.

Introduction

Erik Großkopf and three co-authors turned diffusion models on the task of faking a cell stain, in a paper published in 2025. One stain rules tissue laboratories but shows no cell markers. The marker stain costs much and takes craft. Their model learns to paint the marker stain from the plain one, in a compressed space. Two guides steer it: shape codes from a pathology network, and the code of the plain stain itself.

Craft tricks fix light bias and keep the shape of the tissue through the painting. A new score grades whether the fake holds true cell markers, read by a large pathology network. On two sets the method beat earlier painters, with a gain of nearly a third on the new score for one marker. The authors aim at cheap, true synthetic stains.

Catalog record

CREATORErik Großkopf ∗, Valay Bundele ∗, Mehran Hosseinzadeh ∗, Hendrik P.A. Lensch
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06793

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HistDiST: Histopathological Diffusion-based Stain Transfer Erik Großkopf ∗, Valay Bundele ∗, Mehran Hosseinzadeh ∗, Hendrik P.A. Lensch Abstract Hematoxylin and Eosin (H&E) staining is the cornerstone of histopathology but lacks molecular specificity. While Immunohistochemistry (IHC) provides molecular insights, it is costly and complex, motivating H&E-to-IHC translation as a cost-effective alternative. Existing translation methods are mainly GAN-based, often struggling with training instability and limited structural fidelity, while diffusion-based approaches remain underexplored. We propose HistDiST, a Latent Diffusion Model (LDM) based framework for high-fidelity H&E-to-IHC translation. HistDiST introduces a dual-conditioning strategy, utilizing Phikon-extracted morphological embeddings alongside VAE-encoded H&E representations to ensure pathology-relevant context and structural consistency. To overcome brightness biases, we incorporate a rescaled noise schedule along with v v -prediction, enforcing a zero-SNR condition at the final timestep. During inference, DDIM inversion preserves the morphological structure, while an η \eta -cosine noise schedule introduces controlled stochasticity, balancing structural consistency and molecular fidelity. Moreover, we propose Molecular Retrieval Accuracy (MRA), a novel pathology-aware metric leveraging GigaPath embeddings to assess molecular relevance. Extensive evaluations on MIST and BCI datasets demonstrate that HistDiST significantly outperforms existing methods, achieving a 28% improvement in MRA on the H&E-to-Ki67 translation task, highlighting its effectiveness in capturing true IHC semantics. The code is available at https://github.com/ErikGro/HistDiST. * * footnotetext: These authors contributed equally to the work. 1 Introduction Histopathology is essential for disease diagnosis, revealing tissue structure and cellular morphology. While Hematoxylin and Eosin (H&E) staining is cost-effective and highlights structural details, it lacks molecular specificity. In contrast, Immunohistochemistry (IHC) provides molecular insights but is expensive and time-consuming. Automated H&E-to-IHC translation offers a scalable solution by inferring molecular details from tissue morphology, reducing costs, and enhancing diagnostic consistency. Deep learning methods, particularly GANs, GANs have shown promise in this task. Li et al. [ 11 ] use adaptive supervised PatchNCE loss to preserve structure, while MDCL [ 23 ] employs a conditional GAN with multi-domain contrastive learning to enhance cross-domain alignment. Pathology-specific approaches include patch-level feature extraction with multiple instance learning [ 12 ] and HER2 scoring enhancement via nuclei density estimation [ 17 ] . However, GANs still face mode collapse and require complex loss designs to maintain consistency in pathology screenings. Recently, diffusion models (DMs) [ 8 ] have emerged as a powerful alternative, offering superior stability and synthesis quality. Latent Diffusion Models (LDMs) [ 19 ] reduce the computational burden of DMs by operating in the latent space of pretrained autoencoders. LDM-based image editing methods, such as InstructPix2Pix [ 1 ] for instruction-based editing and ControlNet [ 26 ] for conditional generation, offer structured control. Inference-time optimization techniques using DDIM Inversion [ 22 ] and Delta Denoising Score [ 7 ] further enhance structural fidelity. Although there has been quite some work on applying LDMs for natural image editing, the application of LDMs for H&E-to-IHC stain translation remains largely unexplored. Existing diffusion-based methods [ 4 , 25 ] mainly use text prompts or unpaired signal vector conditioning, but often fail to ensure structural and semantic consistency, critical for accurate pathology translation. To address these limitations, we propose HistDiST, an LDM-based framework for high-fidelity H&E-to-IHC stain translation. Although LDM offers strong generative capabilities, its direct application to stain translation faces three key challenges: (i) lack of explicit morphological conditioning, (ii) brightness bias from conventional diffusion noise schedules, and (iii) insufficient structural preservation at inference. These limitations hinder the model’s ability to capture complex morphology–molecular relationships and lead to inaccurate brightness distributions that obscure diagnostically relevant molecular features. HistDiST overcomes these challenges through three key contributions. First, we employ a dual-conditioning strategy that integrates pathology-specific priors into denoising process. Specifically, Phikon-based embeddings [ 5 ] are injected via cross-attention within the U-Net, while VAE-encoded H&E representations are concatenated with latent noise input. This ensures that pathology-relevant context guides denoising for precise molecular reconstruction. Second, to address brightness inconsistencies, we incorporate a rescaled noise schedule and trailing timesteps [ 13 ] along with v -prediction [ 10 ] , enforcing zero terminal SNR. Third, we perform joint training for unconditional H&E generation and H&E-to-IHC translation, a critical step that enables the use of DDIM inversion [ 21 ] for structure preservation at inference. However, while inversion effectively anchors structure, the subsequent DDIM denoising—being fully deterministic—restricts generative flexibility. As a result, the model fails to capture subtle but diagnostically relevant molecular variations in the translated IHC image. To overcome this limitation, we introduce an \eta -cosine noise schedule that progressively increases stochasticity during DDIM denoising. This modification enables controlled exploration of molecular solution space while preserving morphological consistency. The resulting outputs show improvements in synthesis diversity and fidelity, as reflected by lower FID scores. Moreover, we propose Molecular Retrieval Accuracy (MRA), a novel feature-based metric that quantifies molecular fidelity using cosine similarity. Experiments on MIST and BCI datasets demonstrate that HistDiST outperforms state-of-the-art methods across both qualitative and quantitative benchmarks, achieving superior visual realism, structural preservation, and molecular accuracy. Notably, HistDiST yields 28% improvement in MRA on H&E-to-Ki67 translation, demonstrating highly accurate IHC synthesis. 2 Related Work 2.1 Image Generation using Diffusion Models Diffusion models (DMs) have emerged as a powerful alternative to GANs for image generation, offering greater diversity and training stability by learning the data distribution through iterative denoising processes [ 8 ] . These models operate either directly in the data space or in a compressed latent space, as in Latent Diffusion Models (LDMs) [ 19 ] , enabling more efficient computation. Building on their success in generative tasks, conditional diffusion models have been developed to allow guided image synthesis for applications such as image-to-image translation [ 16 ] and text-based image editing [ 1 ] . Conditioning is typically achieved by injecting external information into the denoising network. For instance, Stable Diffusion [ 19 ] and InstructPix2Pix [ 1 ] leverage text prompts mapped to a shared embedding space with images, while ControlNet [ 27 ] enhances this framework by incorporating spatial guidance, enabling fine-grained structural control. In addition, inference-time optimization techniques—such as DDIM inversion [ 22 ] and Delta Denoising Score (DDS) [ 7 ] —have been introduced to improve structural fidelity and control in a training-free manner. 2.2 Stain Transfer in Computational Pathology The generation of virtually stained images given a source staining type image is an active field of research, aiming to alleviate the lack of useful IHC counterparts to H&E images and meanwhile preventing the challenges caused by re-staining or re-sampling tissues. Generative Adversarial Networks (GANs) have been widely employed for H&E-to-IHC translation, with many approaches focusing on preserving structural and pathological integrity. For example, Li et al. [ 11 ] introduce an adaptive supervised PatchNCE loss to maintain structure, while Chen et al. [ 3 ] improve translation fidelity via a multi-branch discriminator with prototype consistency. Other methods incorporate domain-specific enhancements: Li et al. [ 12 ] propose a patch-level pathological feature extractor with multiple-instance learning, and Peng et al. [ 17 ] refine HER2 scoring using a nuclei density estimator and auxiliary alignment branch. Wang et al. [ 23 ] incorporate Multi-Domain Contrastive Learning (MDCL) within a conditional GAN to enhance cross-domain alignment, and Qu et al. [ 18 ] utilize a multi-magnification processing strategy with an attention module

Read the complete text