ASMIL: Attention-Stabilized Multiple Instance Learning for Whole Slide Imaging
Linfeng Ye, Shayan Mohajer Hamidi, Zhixiang Chi, Guang Li, Mert Pilanci, Takahiro Ogawa, Miki Haseyama, Konstantinos N. Plataniotis · 2026
Read ASMIL: Attention-Stabilized Multiple Instance Learning for Whole Slide Imaging by Linfeng Ye, Shayan Mohajer Hamidi, Zhixiang Chi, Guang Li, Mert Pilanci, Takahiro Ogawa, Miki Haseyama, Konstantinos N. Plataniotis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06658.txt.
Introduction
Ye and team find a new failure mode in attention-based multiple instance learning for whole slide image reading. It swings across epochs instead of settling, hurting performance, besides overfitting and over-concentration. They bring focus-stabilized multi-case learning (ASMIL), using an anchor model, a normalized sigmoid for softmax against narrow focus, and token random dropping against rote learning. ASMIL improves F1 by up to 6.49% over state-of-the-art methods, and its anchor plus tuned sigmoid always boost existing methods. Code and data are open.
Catalog record
| CREATOR | Linfeng Ye, Shayan Mohajer Hamidi, Zhixiang Chi, Guang Li, Mert Pilanci, Takahiro Ogawa, Miki Haseyama, Konstantinos N. Plataniotis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100408 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06658
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ASMIL: Attention-Stabilized Multiple Instance Learning for Whole Slide Imaging Linfeng Ye, Shayan Mohajer Hamidi, Zhixiang Chi, Guang Li, Mert Pilanci, Takahiro Ogawa, Miki Haseyama, Konstantinos N. Plataniotis Abstract Attention-based multiple instance learning (MIL) has emerged as a powerful framework for whole slide image (WSI) diagnosis, leveraging attention to aggregate instance-level features into bag-level predictions. Despite this success, we find that such methods exhibit a new failure mode: unstable attention dynamics. Across four representative attention-based MIL methods and two public WSI datasets, we observe that attention distributions oscillate across epochs rather than converging to a consistent pattern, degrading performance. This instability adds to two previously reported challenges: overfitting and over-concentrated attention distribution. To simultaneously overcome these three limitations, we introduce attention-stabilized multiple instance learning (ASMIL), a novel unified framework. ASMIL uses an anchor model to stabilize attention, replaces softmax with a normalized sigmoid function in the anchor to prevent over-concentration, and applies token random dropping to mitigate overfitting. Extensive experiments demonstrate that ASMIL achieves up to a 6.49% F1 score improvement over state-of-the-art methods. Moreover, integrating the anchor model and normalized sigmoid into existing attention-based MIL methods consistently boosts their performance, with F1 score gains up to 10.73%. All code and data are publicly available at https://github.com/Linfeng-Ye/ASMIL . 1 Introduction Computational pathology, at the intersection of digital imaging, machine learning, and clinical diagnostics, has transformed modern workflows ( 61 ) . Advances in whole slide imaging (WSI) now allow glass slides to be digitized into gigapixel images ( 2 ) , which are central to cancer diagnosis and treatment planning. WSIs preserve rich spatial context and enable large-scale sharing, but their extreme size and sparsity create major challenges: diagnostically relevant regions often occupy only a tiny fraction of the slide, and exhaustive pixel- or tile-level annotations are infeasible in practice. As a result, most datasets provide only weak slide-level labels, making it critical to design methods that learn effectively under weak supervision. This weakly supervised setting naturally motivates multiple instance learning (MIL) ( 32 ; 18 ; 43 ) . In MIL, a bag of instances is mapped to a single bag-level label. For WSIs, the image is divided into tiles, each treated as an instance, while only the slide-level label is required. This dramatically reduces annotation costs and makes large-scale WSI datasets more practical for research and clinical use. Early approaches to MIL-based WSI analysis focused on simple aggregation strategies, such as clustering instance features ( 66 ) or applying global pooling layers ( 34 ) . A major breakthrough came with the introduction of attention-based MIL (ABMIL) ( 30 ) , which provided theoretical guidance for neural network-based MIL algorithms and introduced a permutation-invariant attention mechanism to aggregate instance information into bag-level representations. ABMIL established a strong baseline for WSI analysis ( 49 ) and, importantly, enhanced interpretability through visualized attention scores, which is an essential property for clinical adoption. Building on this foundation, subsequent works have refined ABMIL to further improve performance, scalability, and robustness ( 65 ; 50 ; 73 ; 55 ; 76 ) . In particular, TransMIL replaces independent instance weighting with a transformer encoder that explicitly models inter-instance relations within a bag ( 50 ) . As a result, attention-based MIL has become the de facto choice for WSI subtyping not only because it aggregates instance features but also because its attention maps are used as clinical evidence of model interpretability. Despite its success, attention-based MIL still suffers from three major problems, which we denote as (PI) , (PII) , and (PIII) , and elaborate on in the sequel. A critical yet underexplored aspect of MIL-based WSI analysis is the convergence behavior of attention mechanisms during training. The gigapixel scale of WSIs, coupled with weak supervision, high variability, and sparsity, makes it difficult for models to consistently identify informative tiles among thousands of candidates. Our investigation reveals that existing MIL algorithms often fail to converge stably on WSI datasets. To the best of our knowledge, we are the first to identify and systematically analyze (PI) unstable attention dynamics , where attention distributions for individual WSIs oscillate substantially across epochs instead of converging into consistent patterns. To quantify this phenomenon, we measure the Jensen-Shannon divergence ( 15 ) between consecutive attention distributions of the same WSI, as illustrated for TransMIL ( 50 ) in Figure 1 . Additional experiments across methods and datasets are provided in Appendix P . This persistent oscillation results in unstable training and degraded performance, reflected in higher cross-entropy values compared to our proposed method. Beyond this new limitation identified in our study, prior work has highlighted two additional challenges. One is (PII) over-concentrated attention distribution ( 76 ; 40 ) , where models allocate excessive importance to only a few tiles, thereby harming generalization and interpretability. The other is (PIII) overfitting ( 73 ; 38 ) , a common issue in histopathology WSI classification caused by the limited number of available training samples. In this paper, we aim to simultaneously address the challenges (PI) – (PIII) . To stabilize attention distribution and the training process, we introduce an anchor model , which has the same architecture as the online model’s attention module and receives the same input, but is updated via an exponential moving average (EMA) instead of by backpropagation. Acting as a stable reference, the anchor provides smoother and more consistent attention distributions. To transfer this stability, we encourage the online model to mimic the anchor by minimizing the Kullback–Leibler (KL) divergence between their attention distributions. To mitigate over-concentration, which we attribute to the exponential sensitivity of the softmax function, we replace softmax in the anchor branch with a normalized sigmoid function (NSF), as defined in Equation 5 . Finally, we propose a simple yet effective token dropout strategy that regularizes the model and reduces overfitting. Together with the anchor model, these components form a unified framework called attention-stabilized multiple instance learning (ASMIL), which improves both the stability and generalization of MIL-based WSI analysis. In summary, this paper’s contributions are as follows: \bullet We are the first to identify and systematically analyze the problem of unstable attention dynamics in attention-based MIL for WSI analysis. This overlooked issue not only limits predictive performance but also undermines interpretability, since fluctuating attention distributions prevent consistent identification of the tissue regions that drive the model’s decisions. \bullet To overcome this instability, we introduce an anchor model that stabilizes attention distribution throughout training. The anchor model is updated using an exponential moving average of the online model, which ensures stable training dynamics and improves both performance and interpretability. \bullet We show mathematically that replacing softmax with an NSF alleviates attention over-concentration. Since applying the NSF to the online model causes vanishing gradients, we apply it to the anchor model instead, ensuring stable and well-distributed attention. \bullet To mitigate overfitting, we introduce token dropout, which randomly discards a portion of feature tokens during training while retaining all tokens during inference. \bullet By integrating these innovations, we present attention-stabilized MIL (ASMIL), a novel MIL-based WSI analysis algorithm. Through comprehensive experiments on multiple public WSI datasets, we demonstrate that ASMIL achieves state-of-the-art performance in subtyping and localization tasks. Paper Organization. The remainder of this paper is structured