Hierarchical Classification for Improved Histopathology Image Analysis
Keunho Byeon, Jinsol Song, Seong Min Hong, Yosep Chong, Jin Tae Kwak · 2026
Read Hierarchical Classification for Improved Histopathology Image Analysis by Keunho Byeon, Jinsol Song, Seong Min Hong, Yosep Chong, Jin Tae Kwak (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00504.txt.
Introduction
Byeon and four colleagues replace flat classification in whole slide pathology with HiClass, a hierarchy aware framework that learns coarse and fine labels together. Bidirectional feature sharing lets coarse and fine views inform each other, while tailored losses enforce hierarchy consistency, class separation, and group wise accuracy. On a gastric biopsy set with four coarse and fourteen fine classes, HiClass beats flat baselines at both levels. Capturing label hierarchy thus sharpens both broad and detailed diagnosis from slide images.
Catalog record
| CREATOR | Keunho Byeon, Jinsol Song, Seong Min Hong, Yosep Chong, Jin Tae Kwak |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25188 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00504
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hierarchical Classification for Improved Histopathology Image Analysis Keunho Byeon, Jinsol Song, Seong Min Hong, Yosep Chong, Jin Tae Kwak Abstract Whole-slide image analysis is essential for diagnostic tasks in pathology, yet existing deep learning methods primarily rely on flat classification, ignoring hierarchical relationships among class labels. In this study, we propose HiClass, a hierarchical classification framework for improved histopathology image analysis, that enhances both coarse-grained and fine-grained WSI classification. Built based upon a multiple instance learning approach, HiClass extends it by introducing bidirectional feature integration that facilitates information exchange between coarse-grained and fine-grained feature representations, effectively learning hierarchical features. Moreover, we introduce tailored loss functions, including hierarchical consistency loss, intra- and inter-class distance loss, and group-wise cross-entropy loss, to further optimize hierarchical learning. We assess the performance of HiClass on a gastric biopsy dataset with 4 coarse-grained and 14 fine-grained classes, achieving superior classification performance for both coarse-grained classification and fine-grained classification. These results demonstrate the effectiveness of HiClass in improving WSI classification by capturing coarse-grained and fine-grained histopathological characteristics. Figure 1 : Overview of HiClass. N_{p} , N_{c} , and N_{f} represent the number of patches, coarse-grained classes, and fine-grained classes, respectively. 1 Introduction Whole-slide image (WSI) analysis plays a critical role in the diagnosis of various diseases in pathology [ 14 , 12 ] . The advent of deep learning-based methods has significantly improved the performance of automated analysis at the WSI level [ 2 , 1 , 7 ] . Due to the large size of WSIs, they are typically processed using multiple instance learning (MIL), which enables efficient feature extraction and aggregation for classification. This approach is mostly formulated as either a binary classification or multi-class classification problem, which is referred to as a flat classification problem [ 11 ] . However, disease diagnosis naturally follows a hierarchical structure, where class labels are organized into a class hierarchy [ 15 ] . For example, a WSI may first be classified as benign or tumor. If classified as tumor, it can be further sub-categorized into well-differentiated, moderately-differentiated, and poorly-differentiated tumors. Despite this natural hierarchical structure, to the best of our knowledge, existing methods for WSI analysis primarily focus on flat classification, largely ignoring the inherent class hierarchy of WSIs. Hierarchical classification categorizes WSIs at multiple levels, such as coarse-grained and fine-grained levels. Coarse-grained classification may include broader categories, e.g., benign and tumor, while fine-grained classification may involve detailed sub-categories, such as tumor differentiation grades. Although coarse-grained classification often achieves high accuracy, fine-grained classification remains challenging due to the higher inter-class similarity among fine-grained categories than coarse-grained categories and lack of sufficient samples for fine-grained classes [ 8 ] . Integrating hierarchical classification into WSI analysis can offer several advantages, as it provides a more structured and clinically relevant approach that closely aligns with the diagnostic process used by pathologists. A few studies have explored hierarchical classification for pathology image analysis. For instance, Jin et al. [ 7 ] proposed hierarchical multi-instance learning (HMIL) that conducts both coarse-grained and fine-grained classification. HMIL was evaluated on three cancer datasets, such as cervical cytology cancer, breast cancer, and prostate cancer datasets. Similarly, Cai et al. [ 3 ] developed a Transformer-based model for cervical cytology classification across three hierarchical levels. These studies suggest the potential of hierarchical classification for improved pathology image analysis. In this study, we propose HiClass, a \mathbf{H} ierarchical \mathbf{Class} ification framework for improved histopathology image analysis. By incorporating class hierarchy into the classification process, HiClass improves WSI classification performance for both coarse-grained classification and fine-grained classification. To achieve this, HiClass integrates a multiple instance learning (MIL) framework with bidirectional feature integration, facilitating the information exchange between coarse-grained and fine-grained feature representations. To further optimize hierarchical learning, we introduce tailored, hierarchy-aware loss functions, including hierarchical consistency loss, intra- and inter-class distance loss, and group-wise cross-entropy loss. We evaluate HiClass on a dataset of gastric endoscopic biopsy slides that includes 4 coarse-grained classes and 14 fine-grained classes. The results demonstrate that HiClass achieves high hierarchical classification performance, effectively capturing both coarse-grained and fine-grained histopathological characteristics. Figure 2 : Illustration of proposed hierarchy-aware loss functions. (a) Standard cross-entropy loss is applied independently to coarse- and fine-level predictions. (b) Hierarchical consistency loss ( L_{Con} ) aligns the most confident coarse- and fine-level features using Jensen-Shannon divergence. (c) Intra- and inter-class distance loss ( L_{Int} ) encourages coarse-level grouping in the fine-grained feature space through margin-based KL divergence. (d) Group-wise cross-entropy loss ( L_{GCE} ) limits the fine-level prediction space to classes within the predicted coarse category, improving intra-group discrimination. Table 1 : Details of a gastric endoscopic biopsy dataset. Coarse-grained Fine-grained Num slides Train Val Test Total Benign Fundic gland polyp 725 90 90 905 Hyperplastic polyp 288 36 36 360 Inflammatory fibroid polyp 259 32 32 323 Xanthoma 10 2 2 14 Cancer Malignant lymphoma 18 2 2 22 Neuroendocrine tumor 9 2 2 13 Tubular adenocarcinoma 549 68 68 685 Poorly cohesive carcinoma 118 14 14 146 Dysplasia Tubular adenoma (low-grade) 671 83 83 837 Tubular adenoma (high-grade) 96 11 11 118 Gastritis Chronic active gastritis 126 15 15 156 Chronic gastritis 618 77 77 772 Erosion 224 27 27 278 Ulceration 36 4 4 44 2 Method 2.1 Dataset We employ a set of gastric endoscopic biopsy slides collected from The Catholic University of Korea Uijeongbu St. Mary’s Hospital between 2014 and 2023. This dataset comprises 4,673 biopsy slides, each annotated with structured class labels, including both coarse-grained and fine-grained categories for detailed analysis. The details of this dataset and its hierarchical class labels are available in Table 1 . 2.2 Model Architecture The proposed method contains three key components: 1) Patch-level encoder \mathcal{E} ; 2) Feature aggregator \mathcal{A} ; 3) Hierarchical classifier \mathcal{H} . Each component is detailed below. 2.2.1 Patch-level Encoder The patch-level encoder \mathcal{E} receives a biopsy WSI, divides it into 512x512 image patches, and produces patch-level feature vectors. Specifically, for each image patch, we adopt UNI [ 5 ] , a pre-trained general-purpose model trained on over 100 million image patches via self-supervised learning, to produce a feature vector \in\mathbb{R}^{1024} . 2.2.2 Feature Aggregator Given a set of patch-level feature vectors, the feature aggregator \mathcal{A} produces a single representative feature vector for the entire slide. Following CLAM [ 9 ] , we utilize attention-based pooling to effectively aggregate these patch-level features into a single feature vector \in\mathbb{R}^{512} . 2.2.3 Hierarchical Classifier The hierarchical classifier \mathcal{H} simultaneously performs both coarse-grained classification and fine-grained classification through a structured pipeline with bidirectional feature integration, feature projection heads, and classification heads. Bidirectional feature integration receives an input feature vector from \mathcal{A} and split it into two feature vectors: one designated as the coarse-grained feature vector \mathbf{v}^{c}\in\mathbb{R}^{256} and the other as the fine-grained feature vector \mathbf{v}^{f}\in\mathbb{R}^{256} . Inspired by Chen et al. [ 4 ] , both feature vectors are then augmented with information from the other. To enrich the coarse-grained feature vector, we concatenate it with the fine-grained feature vector, while blocking the gradients from the fine-grained feature vector to prevent coarse-grained classification from being biased towards fine-grained representations. This procedure is given by \mathbf{v}^{c^{\prime}}=\mathbf{v}^{c}\circ G(\mathbf{v}^{f})\in\mathbb{R}^{512} where G is a gradient controller. Similarly, we augment the fine-grained