Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images

Ardhendu Sekhar , Vasu Soni , Keshav Aske , Shivam Madnoorkar , Pranav Jeevan , Amit Sethi · 2025

Read Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images by Ardhendu Sekhar , Vasu Soni , Keshav Aske , Shivam Madnoorkar , Pranav Jeevan , Amit Sethi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06266.txt.

Introduction

Ardhendu Sekhar and co-authors predict cancer patient survival directly from whole-slide microscope images of tissue. Such images are enormous, vary strongly from place to place, and produce survival patterns that resist simple curves. Their framework attacks all three problems with four cooperating parts.

One part selects the slide regions most tied to prognosis, another groups patches by position and appearance, and a third weighs local tissue detail against whole-slide context. A mixture of log-logistic experts then models complicated survival distributions flexibly. Across large cancer cohorts covering lung, kidney, and breast cancers, the method set new best scores and stayed well calibrated.

Catalog record

CREATORArdhendu Sekhar , Vasu Soni , Keshav Aske , Shivam Madnoorkar , Pranav Jeevan , Amit Sethi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06266

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images Ardhendu Sekhar , Vasu Soni , Keshav Aske , Shivam Madnoorkar , Pranav Jeevan , Amit Sethi Abstract Accurate survival prediction from histopathology whole-slide images (WSIs) remains challenging due to gigapixel resolutions, spatial heterogeneity, and complex survival distributions. We introduce a comprehensive computational pathology framework that addresses these limitations through four synergistic innovations: (1) Quantile-Gated Patch Selection to dynamically identify prognostically relevant regions; (2) Graph-Guided Clustering that groups patches by spatial-morphological similarity to capture phenotypic diversity; (3) Hierarchical Context Attention to model both local tissue interactions and global slide-level context; and (4) an Expert-Driven Mixture of Log-Logistics module that flexibly models complex survival distributions. On large-scale TCGA cohorts, our method achieves state-of-the-art performance, with time-dependent concordance indices of 0.644 ± 0.059 0.644{\pm}0.059 on LUAD, 0.751 ± 0.037 0.751{\pm}0.037 on KIRC, and 0.752 ± 0.011 0.752{\pm}0.011 on BRCA—significantly outperforming both histology-only and multimodal benchmarks. The framework provides improved calibration and interpretability, advancing the potential of WSIs for personalized cancer prognosis. † † address: Indian Institute of Technology Bombay 1 Introduction Accurate survival prediction from histopathology images is critical for personalized cancer care, guiding treatment selection and risk stratification. While clinical and genomic markers are routinely used, histopathology images contain rich, underutilized morphological information that could significantly enhance prognostic accuracy. Whole-slide images (WSIs) capture comprehensive tissue-level morphology—including tumor architecture, stromal composition, and microenvironmental context—but their gigapixel resolutions and lack of localized annotations make direct modeling computationally prohibitive. Patch-based multiple instance learning (MIL) has thus emerged as the dominant paradigm for weakly supervised analysis of WSIs. However, survival prediction introduces unique challenges beyond conventional classification tasks. It requires modeling long-range spatial dependencies and subtle morphological patterns across distributed tissue regions, where the arrangement and interaction of different tissue phenotypes may carry critical prognostic significance. Conventional statistical approaches, such as Cox proportional hazards, struggle to capture the complex, high-dimensional, non-linear relationships in histopathology data, while deep learning methods often lack interpretability and fail to maintain spatial coherence across the slide. Current MIL approaches for survival analysis face several limitations: they often treat patches as independent instances, ignoring spatial relationships; they lack mechanisms to disentangle heterogeneous tissue phenotypes; and they typically rely on simplistic survival modeling that cannot capture complex, multi-modal hazard distributions. These limitations restrict their ability to fully leverage the rich morphological information embedded in WSIs. To overcome these challenges, we introduce a unified computational pathology framework that integrates four key innovations: (1) Quantile-Gated Patch Selection to dynamically identify prognostically relevant regions while filtering noise; (2) Graph-Guided Clustering to group patches by spatial-morphological similarity, capturing phenotypic diversity; (3) Hierarchical Context Attention to model both fine-grained local interactions and global slide-level context; and (4) an Expert-Driven Mixture of Log-Logistics that flexibly models complex survival distributions through specialized components. Together, these elements enable interpretable and accurate survival prediction by jointly leveraging spatial structure, phenotypic abstraction, and probabilistic outcome modeling, providing a more comprehensive approach to WSI-based prognosis. 2 Related Work Weakly supervised learning via multiple instance learning (MIL) has been central to WSI-based survival modeling, addressing the challenge of slide-level labels with patch-level features. Early attention-based methods, such as ABMIL [ 5 ] , CLAM [ 9 ] , and DSMIL [ 8 ] , identified prognostic patches but overlooked spatial context and inter-patch relationships. Subsequent approaches incorporated structural information through graph-based methods (PatchGCN [ 2 ] ) and hierarchical transformers (HGT [ 10 ] , TransMIL [ 12 ] , HIPT [ 1 ] ), capturing spatial and multi-scale relations at higher computational cost. Recent efforts have explored multimodal integration of histopathology with genomic (PathoGen-X [ 7 ] ), methylation (CoC [ 16 ] ), and language data (HiLa [ 3 ] ). In parallel, probabilistic methods, such as SCMIL [ 15 ] employ Gaussian distributions for survival estimation but lack flexibility for complex hazard shapes. However, these approaches typically lack explicit mechanisms for phenotype disentanglement and struggle to model heterogeneous compositions in cancer microenvironments. Our method addresses these limitations through integrated phenotype-aware clustering and expert-driven survival modeling. We combine graph-guided clustering for spatial-morphological coherence, hierarchical attention for multi-scale context, and a mixture of log-logistic experts that adapt to diverse survival patterns. This unified approach enables interpretable phenotype disentanglement while flexibly modeling complex survival distributions beyond conventional parametric assumptions. Figure 1 : (a) Proposed survival framework. (b) Expert-Driven Mixture of Log-logistics block. 3 Methodology Figure 1 illustrates the overall methodology. Each Whole Slide Image (WSI) is divided into non-overlapping 224\times 224 patches at 40 \times magnification. Background regions are removed, and remaining patches are encoded using a histopathology foundation model [ 6 ] , yielding a patch feature matrix \mathbf{P}_{\text{feat}}\in\mathrm{R}^{n\times d} where n is the number of patches and d the feature dimensionality. 3.1 Quantile-Gated Patch Selection (QGPS) A quantile-based MLP selects task-relevant patches using importance scores from model logits, filtering out uninformative regions to reduce computational burden and noise. For each WSI, threshold \tau_{q} is the q -quantile of logits, retaining the top (1-q)\times 100\% of patches: \mathcal{P}_{\text{sel}}=\{\mathbf{P}_{i}\mid\text{logit}_{i}>\tau_{q}\},\quad\mathcal{P}_{\text{rem}}=\{\mathbf{P}_{i}\mid\text{logit}_{i}\leq\tau_{q}\}. (1) 3.2 Graph-Guided Clustering (GGC) Selected patches \mathcal{P}_{\text{sel}}\in\mathrm{R}^{m\times d} are clustered into morphologically and spatially coherent groups to capture phenotypic diversity. Morphological similarity uses cosine similarity: S_{\text{morph}}(i,j)=\frac{\langle\mathbf{P}_{i},\mathbf{P}_{j}\rangle}{\|\mathbf{P}_{i}\|\,\|\mathbf{P}_{j}\|}, (2) while spatial similarity uses an exponential kernel over Euclidean distances: S_{\text{spatial}}(i,j)=\exp\!\left(-\frac{D_{ij}}{\sigma_{D}}\right),\quad\sigma_{D}=\text{std}(D)+\varepsilon. (3) The composite similarity S=\omega_{\text{morph}}S_{\text{morph}}+\omega_{\text{spatial}}S_{\text{spatial}} guides a k -NN graph construction, with patches clustered into G groups {L_{1},\dots,L_{G}} using GPU-accelerated K-Means. 3.3 Hierarchical Context Attention (HCA) Each cluster L_{i} undergoes multi-head self-attention (MHSA) [ 14 ] for local dependencies, capturing fine-grained relationships within phenotypically similar regions: L^{\prime}_{i}=\text{LayerNorm}\!\big(L_{i}+\text{MHSA}(L_{i})\big). (4) Refined clusters are summarized into embeddings R_{i}=\frac{1}{|L^{\prime}_{i}|}\sum_{\mathbf{x}\in L^{\prime}_{i}}\mathbf{x} , then processed by another MHSA layer for global relationships across different tissue regions: R^{\prime}=\text{LayerNorm}\!\big(R+\text{MHSA}(R)\big). (5) Intra-cluster representations are concatenated as \widetilde{P}=\mathrm{Concat}(L^{\prime}_{g})_{g=1}^{G} . A broadcast descriptor R^{\prime}_{\mathrm{exp}} , derived by averaging R^{\prime} across clusters, is added residually to form \widehat{P} , which is concatenated with remaining patches to yield \mathcal{P}_{\text{final}} . Slide-level representation is derived via attention pooling: \mathbf{z}_{\text{WSI}}=\sum_{i}\alpha_{i}\,\mathcal{P}_{\text{final},i},\quad\alpha_{i}=\text{softmax}\!\big(W_{a}\,\tanh(W_{h}\,\mathcal{P}_{\text{final},i}^{\top})\big), (6) where \mathbf{z}_{\text{WSI}} represents the integrated prognostic feature for the entire slide. 3.4 Expert-Driven Mixture of Log-logistics (EDMLL) The module models survival probabilities using multiple experts specializing in distinct WSI features \mathbf{z}_{\text{WSI}} , enabling capture of diverse risk patterns. Each expert models the conditional distribution using log-logistic mixtures: p(t\mid\mathbf{z}_{\text{WSI}},e)=\sum_{k=1}^{K}\lambda^{(e)}_{k}(\mathbf{z}_{\text{WSI}})\,\mathrm{LLD}\!\big(t\mid\alpha^{(e)}_{k},\beta^{(e)}_{k}\big), (7) where the log-logistic density is: \mathrm{LLD}(t\mid\alpha,\beta)=\frac{(\beta/\alpha)(t/\alpha)^{\beta-1}}{\big[1+(t/\alpha)^{\beta}\big]^{2}},\quad t>0. (8) Learnable global anchors (P_{\alpha},P_{\beta})\in\mathrm{R}^{K} are projected as: \alpha^{(e)}=\text{softplus}\!\big(W^{(e)}_{\alpha}P_{\alpha}\big),\quad\beta^{(e)}=\text{softplus}\!\big(W^{(e)}_{\beta}P_{\beta}\big). (9) A gating network G(\mathbf{z}_{\text{WSI}}) assigns softmax weights to experts, producing the time probability density(TPDF) and survival probability functions(SPF): \text{TPDF}(t\mid\mathbf{z}_{\text{WSI}})=\sum_{e,k}G_{e}(\mathbf{z}_{\text{WSI}})\lambda^{(e)}_{k}\,\mathrm{LLD}\!\big(t\mid\alpha^{(e)}_{k},\beta^{(e)}_{k}\big), (10) \text{SPF}(t\mid\mathbf{z}_{\text{WSI}})=1-\!\sum_{e,k}G_{e}(\mathbf{z}_{\text{WSI}})\lambda^{(e)}_{k}F_{\mathrm{LLD}}\!\big(t\mid\alpha^{(e)}_{k},\beta^{(e)}_{k}\big). (11) where F_{\mathrm{LLD}} is obtained by integrating the LLD. The model is trained using negative log-likelihood loss handling both censored ( c=0 ) and uncensored ( c=1 ) data: \displaystyle\mathcal{L}_{\text{NLL}}= \displaystyle-c\log\!\big(\text{TPDF}(t_{d}\mid\mathbf{z}_{\text{WSI}})\big) (12) \displaystyle-(1-c)\log\!\big(\text{SPF}(t_{d}\mid\mathbf{z}_{\text{WSI}})\big), combined with gating-entropy loss: \mathcal{L}_{\text{ent}}=-\sum_{e}G_{e}(\mathbf{z}_{\text{WSI}})\log(G_{e}(\mathbf{z}_{\text{WSI}})) (13) to encourage diverse expert usage. The total loss is \mathcal{L}_{\text{total}}=\mathcal{L}_{\text{NLL}}+\lambda_{\text{ent}}\mathcal{L}_{\text{ent}} . Figure 2 : Kaplan–Meier survival curves showing significant high- vs. low-risk separation across TCGA cohorts. Table 1 : Left: TCGA performance (TDC, higher is better). Right: Ablations on q in QGPS, GGC, HCA and e in EDMLL. Best TDC results in bold . (a) TCGA results. Modality:I=image,G=genomic,M=methylation,L=language prompt. Method Modality LUAD KIRC BRCA MEAN AMIL [ 5 ] I 0.632\pm 0.021 0.691\pm 0.022 0.735\pm 0.013 0.686 CLAM [ 9 ] I 0.615\pm

Read the complete text