Spatial Blindness in Whole-Slide Multiple Instance Learning

Xiangyu Li, Ran Su · 2026

Read Spatial Blindness in Whole-Slide Multiple Instance Learning by Xiangyu Li, Ran Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17449.txt.

Introduction

Li and Su expose spatial blindness in whole-slide multiple-instance learning. Strong pathology models keep nearly the same slide-level scores after patch coordinates are shuffled. That shows predictions rest on appearance composition rather than tissue architecture. They trace this to early learning of dense look statistics under slide-level supervision, starving spatial relations of gradient. Their ResTopoMIL fits a shuffle-proof prototype histogram first. It then trains a light graph branch on the residual, restoring coordinate sensitivity across nine benchmarks.

Catalog record

CREATORXiangyu Li, Ran Su
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80200
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17449

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spatial Blindness in Whole-Slide Multiple Instance Learning Xiangyu Li, Ran Su Abstract Whole-slide MIL models are often called context-aware once graphs, Transformers, or state-space modules are placed above patch embeddings. We show that this label can be deceptive. On pathology tasks where tissue architecture is part of the diagnostic signal, several strong MIL baselines retain nearly unchanged slide-level AUC after patch coordinates are permuted. Their predictions are accurate, but largely compositional. We refer to this failure mode as spatial blindness . Our explanation is optimization-based: dense appearance statistics are learned early under slide-level supervision, leaving weak gradients for sparse spatial relations. ResTopoMIL addresses the issue by first fitting a permutation-invariant prototype histogram and then freezing it while a lightweight graph branch learns the residual under a coordinate-shuffling constraint. The architecture is simple by design; the intervention is in how the spatial branch is trained. Across 9 public WSI benchmarks, ResTopoMIL improves classification and survival prediction with 1.15M parameters, restores sensitivity to coordinate perturbation, and gives stronger localization evidence on CAMELYON-16. 1 Introduction Computational pathology increasingly learns diagnostic and prognostic models directly from digitized whole-slide images (WSIs), where cellular morphology, tissue architecture, and long-range context are preserved at gigapixel scale ( 35 ; 52 ; 47 ) . The weak supervision problem is severe: a slide may contain tens of thousands of relevant and irrelevant regions, but most clinical datasets provide only a slide-level label. Multiple instance learning (MIL) is therefore the standard formulation for WSI analysis ( 15 ; 31 ; 4 ; 30 ) : patches are treated as instances, the slide as a bag, and the model predicts the bag label without patch-level annotation. MIL works remarkably well in pathology, but its success can hide a limitation. Many labels are not determined by local appearance alone. Gleason grading depends on gland formation; breast cancer subtyping distinguishes ductal growth from single-file strands; prognosis often reflects invasive fronts, immune aggregates, papillary cores, solid growth, or tumor-stroma organization ( 40 ; 11 ; 54 ) . These are spatial statements. They depend not just on which tissue components appear, but on how they are arranged. Figure 1 : The ResTopoMIL Concept. (a) A standard MIL model may give similar predictions before and after spatial permutation, indicating that it mainly uses composition. (b) ResTopoMIL separates the problem into a statistical stream and a topological stream. (c) The statistical stream provides a base prediction, while the topological stream learns a residual correction from spatial organization. At first glance, recent context-aware MIL methods should address this issue. Graph networks, Transformers, hierarchical models, and state-space models all process a slide as more than an unordered bag ( 10 ; 36 ; 1 ; 51 ; 44 ; 8 ; 20 ; 60 ; 65 ) . Architecture alone, however, does not tell us what the trained predictor uses. We use a simple stress test: keep every patch embedding fixed, and randomly permute the coordinates used to build spatial context. On tasks where architecture is label-relevant, a topology-using model should suffer. Several strong context-aware baselines barely move. They have spatial machinery, but their learned decision rules behave much like bag-of-visual-words classifiers. The question is then not whether a model contains a spatial operator, but why such an operator can remain unused. Our explanation is optimization-based. Tissue composition provides a dense, early signal: many patches contribute to the same slide label. Topological evidence is sparser and harder to align with slide-level supervision. Under joint training, the network can reduce the loss by fitting composition first; once that happens, little useful gradient remains for the spatial branch. We call this behavior optimization laziness , in the descriptive sense that the easiest explanatory signal dominates training while the harder structural signal is left undertrained. The phenomenon is related to simplicity bias, texture bias, and gradient starvation ( 41 ; 18 ; 32 ; 38 ) . ResTopoMIL follows this diagnosis. Rather than asking one network to discover composition and topology at the same time, it learns the compositional explanation explicitly. A permutation-invariant statistical stream is trained first and frozen. A lightweight graph stream is then trained on the residual, with a shuffle-based loss that asks it to distinguish real tissue topology from coordinate-permuted topology. The graph module is intentionally small. The point is not to add a heavier context block, but to change the training problem faced by the spatial branch. This shifts the evaluation away from architecture labels. “Context-aware” should mean that a model’s decision changes when clinically relevant spatial organization is removed. We therefore test topology-destroying coordinate perturbations, separate pure composition from pure topology in a controlled benchmark, and ask whether residual training changes both accuracy and spatial behavior. Topology-preserving transformations are discussed only as expected graph invariances, not as an additional measured benchmark. The paper makes four contributions: • Spatial blindness is defined and tested as insensitivity to coordinate perturbations on structure-dependent MIL tasks. • A controlled composition–topology diagnostic benchmark shows that strong MIL models can solve compositional tasks while failing on pure topology. • ResTopoMIL learns composition first and topology as a residual correction, with design analysis deferred to the appendix. • Experiments on 9 public pathology benchmarks show gains in prediction, spatial sensitivity, and localization quality with a compact 1.15M-parameter model. 2 Related Work MIL has a long history as a weak-supervision framework for learning from bag labels without instance labels ( 15 ; 31 ) . In pathology, this formulation is natural because WSIs are too large for end-to-end pixel-level training and dense annotation is expensive. Early WSI approaches used patch classifiers, clustering, global pooling, or multi-view CNN aggregation to move from local tiles to slide-level labels ( 59 ; 26 ; 14 ; 13 ; 53 ; 6 ) . Attention-based MIL then became a central baseline because it learns instance weights while remaining permutation-invariant and interpretable ( 23 ; 30 ) . Later methods select critical instances, split bags, mine hard examples, regularize attention, or improve instance-level classifiers ( 27 ; 63 ; 48 ; 28 ; 66 ; 67 ; 39 ; 43 ; 70 ; 69 ) . A second line goes beyond unordered aggregation by modeling context among patches. Graph models treat WSIs as point clouds or tissue graphs ( 10 ; 1 ; 36 ; 34 ) ; Transformer and low-rank attention models capture long-range bag dependencies ( 51 ; 44 ; 58 ; 8 ; 57 ) ; hierarchical and context-aware variants exploit WSI pyramids ( 64 ; 21 ; 3 ; 50 ; 17 ; 7 ) ; and state-space models provide efficient long-sequence modeling ( 20 ; 60 ; 65 ) . These methods matter because pathology is not i.i.d.; structured MIL already recognizes that bags with the same instances can have different labels when dependencies differ ( 68 ; 62 ) . We ask a complementary question: after adding such modules, does the learned predictor rely on topology, or does it still mainly count visual words? Foundation models have greatly improved pathology patch embeddings ( 9 ; 24 ; 33 ; 5 ; 29 ; 25 ) , while vision-language and prompt-based WSI methods broaden slide supervision ( 45 ; 22 ; 56

Read the complete text