Adaptive Multi-Scale Integration Unlocks Robust Cell Annotation in Histopathology Images
Yinuo Xu, Yan Cui, Mingyao Li, Zhi Huang · 2025
Read Adaptive Multi-Scale Integration Unlocks Robust Cell Annotation in Histopathology Images by Yinuo Xu, Yan Cui, Mingyao Li, Zhi Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-13586.txt.
Introduction
Yinuo Xu and 3 co-authors present NuClass, a system that names cell types in pathology images by joining nuclear detail with tissue context. Tile-based models read nuclei well but miss the neighborhood that often decides a cell's identity. Human labels are also coarse and uneven, which blocks fine subtype work.
The team first built a marker-guided dataset from spatial transcriptomics with over two million labeled cells across eight organs. NuClass then fuses a local view of each nucleus with a wide view of its surroundings through a learned gate, steering the wide view toward spots where the local view is unsure. On three held-out cohorts it beat strong baselines and gave calibrated confidence with visual explanations.
Catalog record
| CREATOR | Yinuo Xu, Yan Cui, Mingyao Li, Zhi Huang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-13586
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adaptive Multi-Scale Integration Unlocks Robust Cell Annotation in Histopathology Images Yinuo Xu, Yan Cui, Mingyao Li, Zhi Huang Abstract Identifying cell types and subtypes in routine histopathology is fundamental for understanding disease. Existing tile-based models captures nuclear detail but miss the broader tissue context that influences cell identity. Current human annotations are coarse-grained and uneven across studies, making fine-grained, subtype-level classificsation difficult. In this study, we build a marker-guided dataset from Xenium spatial transcriptomics with single-cell–resolution labels for more than two million cells across eight organs and 16 classes to overcome the lack of high-quality annotations. Leveraging this data resource, we introduce NuClass , a pathologist workflow–inspired framework for cell-wise multi-scale integration of nuclear morphology and microenvironmental context. It combines Path local , which focuses on nuclear morphology from 224 2 224^{2} -pixel crops, and Path global , which models the surrounding 1024 2 1024^{2} -pixel neighborhood, through a learnable gating module to balance local and global details. An uncertainty-guided objective directs the global path to prioritize regions where the local path is uncertain, and we provide calibrated confidence estimates and Grad-CAM maps for interpretability. Evaluated on three fully held-out cohorts, NuClass achieves up to 96% F1 for its best-performing class, outperforming strong baselines. Our results demonstrate that multi-scale, uncertainty-aware fusion can bridge the gap between slide-level pathological foundation models and reliable, cell-level phenotype prediction. Figure 1 : NuClass: cell-wise multi-scale classification with gating. (Top) From whole-slide images (WSIs), we extract paired fields centered on each cell: a nucleus-scale crop ( 224^{2} at 0.25 mpp) and a co-registered contextual field-of-view (FOV; default 1024^{2} at 0.25 mpp). Spatial assays (Xenium-like) provide gene-marker profiles and cell centroids, enabling scalable, subcellular annotations. (Stage 1, Path local) A UNI2-h morphology backbone processes the 224^{2} crop. A tissue-conditioned FiLM adaptor modulates features before a linear head outputs per-cell probabilities. (Stage 2, Path global) A DINOv3 ViT-L/16 encoder ingests the 1024^{2} contextual FOV; its representation is concatenated with a Path local morphology vector to predict complementary probabilities. (Stage 3, Fusion Gate) A lightweight cell-wise gate receives statistics from both distributions and compact feature projections, then fuses experts in probability space \mathbf{p}_{\text{mix}}=(1-g)\mathbf{p}_{local}+g\mathbf{p}_{global} . The gate learns which expert to trust per cell , not to average logits. 1 Introduction Understanding cell types and their spatial interplay within tissues is central to studying human disease. While emerging technologies such as spatial transcriptomics enable cell type identification through marker genes, hematoxylin and eosin (H&E)-stained whole-slide images remain the most practical and widely adopted modality in routine clinical workflows, serving as the cornerstone for differential diagnosis and quantitative evaluation and guiding assessment of cancer stage, transplant outcomes, and treatment strategies. In H&E images, nuclei segmentation 42 is well studied, but assigning detected nuclei to robust, generalizable cell identities—i.e., cell classification—remains challenging. This largely stems from (1) the lack of large-scale, cross-slice, high-quality, unified training datasets and (2) the difficulty of learning cell type–specific features under heterogeneous morphology across patients, organs, and scanners. Together, these issues reflect two core limitations: data scarcity and model design . On the data side, recent studies show that a lack of high-quality, cell-level annotated training data , combined with stain/tissue heterogeneity and domain shifts, strongly constrains deep models for cell classification. Human annotations are often coarse and inconsistent and require substantial expert effort, limiting subcellular precision, scalability, and the diversity of labeled cell types. On the modeling side, three challenges remain. (i) Current cell embedding models are largely unaware of a cell’s neighborhood, even though distinguishing, for example, cancer-associated from normal epithelial cells or macrophages from T cells often requires context beyond the local nuclear image. Multi-scale pipelines for gigapixel slides improve slide-level reasoning 8 , but aggregation typically occurs at the bag or slide level, leaving the instance-wise balance between morphology and context underexplored. (ii) Models struggle with varying annotation granularities and taxonomies across tissues and datasets (e.g., the same cell may be labeled as “inflammatory cell,” “immune cell,” or “lymphocyte”). (iii) There is no large-scale pretrained foundation model across diverse diseases and organs to provide a strong encoder for downstream fine-tuning. These spatial and semantic nuances across data, annotations, and models remain underexplored, creating an urgent need for a foundation model trained on millions of cells that can harmonize diverse annotations and enable deeper understanding of cell types and tissue organization. To address data scarcity and model design jointly, we present a workflow from data curation to model development to advance understanding of cellular and nuclear features in H&E-stained images. On the data side, we build a scalable, marker-guided annotation pipeline on Xenium spatial assays that assigns fine-grained cell types with pixel-level coordinates to millions of cells, complementing and refining human supervision 1 . The resulting labels are objective and reproducible across tissues and institutions and provide precise anchors for cross-scale modeling and evaluation (Fig. 1 , top). On the modeling side, we develop NuClass , an adaptive framework for cell-wise multi-scale integrated learning. NuClass uses two fixed fields of view: Path local focuses on nucleus-centered morphology using small crops, whereas Path global processes the co-registered larger neighborhood to capture tissue architecture and micro-environmental cues. A learnable gate decides, for each cell, how much to rely on local detail versus global context, and training encourages the two paths to specialize and complement rather than duplicate one another. At scale, NuClass generalizes across organs and datasets for reliable cell-level phenotyping. Trained on more than 2M marker-labeled cells spanning 16 types across 8 organs and evaluated on three fully held-out cohorts (about 0.28M cells), it achieves up to 96% for its best-performing cell class and consistently matches or surpasses strong baselines while producing well-calibrated, interpretable predictions. 2 Related Work Multi-organ multi-class H&E cell datasets. Recent dataset efforts have shifted from single-organ toward multi-organ collections, such as MoNuSAC 40 (4 tissues) and PanNuke 13 (19 organs/tissues). However, due to the scarcity of large, well-annotated cell datasets, most cell-classification models remain tailored to specific organs or narrow use-cases, as in HoVer-Net 17 and CellViT 20 . Foundation models for cell classification. Foundation models are reshaping computational pathology by learning general-purpose visual or vision–language representations from large, heterogeneous datasets 7 ; 44 ; 41 ; 27 ; 28 ; 11 ; 43 . These approaches transfer well on patch- and slide-level benchmarks, and vision–language systems extend to retrieval, question answering, and outcome prediction. Yet most progress still targets slide-level endpoints. In routine practice, a cell’s identity depends on both its nuclear morphology and the tissue neighborhood in which it resides, but existing systems seldom integrate these two sources of evidence at per-cell granularity. The recent VOLTA algorithm 30 leverages over 800k cells across six cancer types to train a self-supervised cell representation model via contrastive learning, but it still relies on cell patches and spatial neighborhoods as unlabeled training signals and does not provide label-consistent, cross-organ cell types. Per-cell representation learning at scale. The transition from patch-level to cell-level understanding marks a fundamental step toward morphological precision. Traditional architectures like HoVer-Net 17 and CellViT 19 jointly segment and classify cells, yet remain trained on a limited set of annotated organs. Graph- or structure-aware models 26 extend this by modeling cell neighborhoods and propagating context among adjacent cells to