Towards comprehensive cellular characterisation of H&E slides

Benjamin Adjadj ∗,† , Pierre-Antoine Bannier ∗,† , Guillaume Horent ∗ , Sebastien Mandela, Aurore Lyon, Kathryn Schutte, Ulysse Marteau, Valentin Gaury, Laura Dumont, Thomas Mathieu, MOSAIC consortium, Reda Belbahri, Benoît Schmauch, Eric Durand, Katharina Von Loga, Lucie Gillet ∗ · 2025

Read Towards comprehensive cellular characterisation of H&E slides by Benjamin Adjadj ∗,† , Pierre-Antoine Bannier ∗,† , Guillaume Horent ∗ , Sebastien Mandela, Aurore Lyon, Kathryn Schutte, Ulysse Marteau, Valentin Gaury, Laura Dumont, Thomas Mathieu, MOSAIC consortium, Reda Belbahri, Benoît Schmauch, Eric Durand, Katharina Von Loga, Lucie Gillet ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09926.txt.

Introduction

Benjamin Adjadj, Pierre-Antoine Bannier, and a large consortium team work toward complete cell analysis of standard pathology slides in a 2025 medical imaging study. Hematoxylin and eosin slides are the daily bread of cancer diagnosis, and their tumor microenvironment, which cells sit where, carries prognostic information. Automated detection, segmentation, and classification of every cell would unlock that information at scale, but current methods perform poorly on unseen data.

The consortium effort targets robust performance across laboratories and scanners. Comprehensive cellular maps could make tumor grading and treatment selection more consistent between hospitals.

Catalog record

CREATORBenjamin Adjadj ∗,† , Pierre-Antoine Bannier ∗,† , Guillaume Horent ∗ , Sebastien Mandela, Aurore Lyon, Kathryn Schutte, Ulysse Marteau, Valentin Gaury, Laura Dumont, Thomas Mathieu, MOSAIC consortium, Reda Belbahri, Benoît Schmauch, Eric Durand, Katharina Von Loga, Lucie Gillet ∗
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-09926

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards comprehensive cellular characterisation of H&E slides Benjamin Adjadj ∗,† , Pierre-Antoine Bannier ∗,† , Guillaume Horent ∗ , Sebastien Mandela, Aurore Lyon, Kathryn Schutte, Ulysse Marteau, Valentin Gaury, Laura Dumont, Thomas Mathieu, MOSAIC consortium, Reda Belbahri, Benoît Schmauch, Eric Durand, Katharina Von Loga, Lucie Gillet ∗ Abstract 1 1 footnotetext: Core team. 2 2 footnotetext: Equal contributions. Abstract Cell detection, segmentation and classification are essential for analyzing tumor microenvironments (TME) on hematoxylin and eosin (H&E) slides. Existing methods suffer from poor performance on understudied cell types (rare or not present in public datasets) and limited cross-domain generalization. To address these shortcomings, we introduce HistoPLUS, a state-of-the-art model for cell analysis, trained on a novel curated pan-cancer dataset of 108,722 nuclei covering 13 cell types. In external validation across 4 independent cohorts, HistoPLUS outperforms current state-of-the-art models in detection quality by 5.2% and overall F1 classification score by 23.7%, while using 5x fewer parameters. Notably, HistoPLUS unlocks the study of 7 understudied cell types and brings significant improvements on 8 of 13 cell types. Moreover, we show that HistoPLUS robustly transfers to 2 oncology indications unseen during training. To support broader TME biomarker research, we release the model weights and inference code (see Code Availability). 1 Introduction Cell detection, segmentation and classification in histological images form the foundation of modern computational pathology, enabling quantitative analysis of the tumor microenvironment (TME) for applications in patient stratification [ 1 , 2 ] and personalized medicine [ 3 , 4 , 5 ] . Beyond cell identification, these techniques reveal spatial relationships between tumor cells, immune cells, and stromal compartments that correlate with disease outcomes [ 6 , 7 ] and treatment responses [ 8 , 9 ] . The spatial organization and density of specific cell populations – such as tumor-associated neutrophils (linked to immunosuppression [ 10 ] ) or cancer-associated fibroblasts (implicated in therapy resistance [ 11 , 12 , 13 ] ) – have become increasingly important in clinical research and trials [ 14 , 15 ] . While immunohistochemical methods provide targeted molecular information, H&E staining remains predominant in clinical practice due to its accessibility, cost-effectiveness, and rich morphological detail [ 16 ] . However, extracting comprehensive cellular information from H&E images presents substantial computational challenges. Cells in tissue sections exhibit marked heterogeneity in morphology, size and staining characteristics, while often being densely packed with indistinct boundaries. The identification of clinically relevant immune cells (e.g., lymphocytes, neutrophils, eosinophils, plasmocytes) and stromal cell subsets (e.g., fibroblasts, smooth muscle cells) requires extensive annotated training data from expert pathologists. Yet, to the best of our knowledge, current public annotated datasets have critical shortcomings: many datasets provide segmentation masks without classification labels [ 17 , 18 , 19 ] , limiting utility for TME studies; others offer only bounding boxes [ 20 ] , precluding morphometric analysis. Even datasets with cell-type annotations have limited class granularity [ 21 , 22 , 23 ] , lack rare cell representation [ 21 ] or focus on a limited set of cancer types [ 24 ] . These constraints, coupled with insufficient sample sizes [ 23 , 25 ] , have hindered the development of robust, generalizable models across diverse tissues and diseases. Recent pathology foundation models (PFMs) pre-trained via self-supervision [ 26 , 27 , 28 ] promise to mitigate these limitations by leveraging unlabeled histology data to learn transferable representations, reducing reliance on scarce expert annotations. Models like UNI [ 29 ] , Phikon [ 30 , 31 ] , and H-Optimus-0 [ 32 ] have advanced patch classification and molecular prediction [ 33 ] , but their applicability to cell detection, segmentation and classification – particularly for understudied subtypes, i.e. non-lymphoid immune cells (eosinophils, neutrophils and macrophages), plasmocytes, stromal cells (smooth muscle, endothelial) and rare events (mitotic figures, apoptotic bodies) – has not been thoroughly investigated. Despite evaluations of Hibou-Large [ 34 ] , UNI, Virchow and GigaPath [ 35 ] on cell-level tasks, no benchmark compares PFMs performance rigorously and exhaustively for simultaneous cell detection, segmentation and classification tasks. This gap is critical, as without systematic benchmarking of PFMs, researchers cannot identify which models maximize performance for these tasks – leaving potential performance gains unrealized despite their importance for quantifying the TME. To address these gaps, we present: • A pan-tumor dataset with granular labels for 13 cell types across 6 cancer indications, obtained using an active-learning based annotation pipeline that maximizes label quality and diversity, • A PFM-integrated model for pan-cancer cell detection, segmentation and classification in H&E, leveraging self-supervised distilled features [ 31 , 32 , 36 ] within a CellViT architecture that achieves state-of-the-art performances on all nuclei identification, including understudied types. Our model demonstrates superior performance both in cross-validation and on external validation. It paves the way for new analyses of spatial tumor-immune interactions. To facilitate community adoption and further progress in computational pathology, we release our pre-trained CellViT model. This contribution provides the research community with a state-of-the-art cell detection, segmentation and classification model on H&E slides. It aims to accelerate biomarker discovery, ultimately enhancing our ability to translate histological insights into clinically relevant applications. 2 Results 2.1 Pan-cancer nuclei dataset enriched in understudied types enables granular study of TME We introduce HistoTRAIN, a pan-cancer nuclei dataset designed to address critical gaps in cellular diversity, annotation quality, and clinical applicability. The dataset comprises 108,722 nuclei segmentations from 739 H&E whole-slide images (WSI), across six cancer types (bladder urothelial carcinoma, colon adenocarcinoma, lung adenocarcinoma, lung squamous cell carcinoma, mesothelioma and pancreatic adenocarcinoma) and covering 13 distinct cell types (Figure 1a). This dataset collection was done in 2 steps. First, point annotations of nuclei centroids together with their respective classes were entirely made by expert pathologists. Second, we leveraged NuClick [ 37 ] , a deep learning model specifically trained for inferring nuclei segmentations from point annotations in nuclei images, to obtain precise segmentations for each annotated nucleus. Leveraging NuClick for nuclei contours drastically reduces the variability in manual boundary delineation and accelerates the annotation process. To systematically prioritize rare and relevant cell types, we developed an active learning pipeline (Figure 1b) that iteratively identifies tissue regions with low model confidence and high probability of understudied cell types presence (see Methods). This approach increased the representation of understudied cell types compared to existing public datasets (see Table 1). The dataset is enriched with nuclei segmentations of non-lymphoid immune cells (eosinophils, neutrophils and macrophages), plasmocytes, stromal cells (smooth muscle, endothelial) and rare events (mitotic figures, apoptotic bodies). Epithelial cells (non-cancerous) and red blood cells were also included to improve morphological context. A detailed distribution of each cell type per cancer indication is presented in Figure 1a. In total, our dataset comprises 1,415 images of size 448x448 pixels (at a 40x magnification; Figure 1c). Cell types NuCLS CoNSeP Lizard PanNuke HistoTRAIN (Ours) Neutrophils 0.09% - 0.97% - 5.88% Eosinophils 0.01% - 0.73% - 1.86% Macrophages 2.69% - - - 5.29% Mitotic Figures - - - - 0.21% Apoptotic Bodies - - - 1.53% 5.33% Endothelial Cells - - - - 2.02% Smooth Muscle Cells - - - - 1.90% Table 1 : Comparison of the distribution of understudied cell populations in

Read the complete text