Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images
1 st Bingjun Li, 2 nd Mostafa Karami, 3 nd Masum Shah Junayed, 4 rd Sheida Nabavi · 2024
Read Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images by 1 st Bingjun Li, 2 nd Mostafa Karami, 3 nd Masum Shah Junayed, 4 rd Sheida Nabavi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02534.txt.
Description
Abstract Understanding the intricate cellular environment within biological tissues is crucial for uncovering insights into complex biological functions. While single-cell RNA sequencing has significantly enhanced our understanding of cellular states, it lacks the spatial context necessary to fully comprehend the cellular environment. Spatial transcriptomics (ST) addresses this limitation by enabling transcriptome-wide gene expression profiling while preserving spatial context.
Catalog record
| CREATOR | 1 st Bingjun Li, 2 nd Mostafa Karami, 3 nd Masum Shah Junayed, 4 rd Sheida Nabavi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02534
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images Thanks: This study is funded by National Science Foundataion (NSF) 1 st Bingjun Li, 2 nd Mostafa Karami, 3 nd Masum Shah Junayed, 4 rd Sheida Nabavi Abstract Understanding the intricate cellular environment within biological tissues is crucial for uncovering insights into complex biological functions. While single-cell RNA sequencing has significantly enhanced our understanding of cellular states, it lacks the spatial context necessary to fully comprehend the cellular environment. Spatial transcriptomics (ST) addresses this limitation by enabling transcriptome-wide gene expression profiling while preserving spatial context. One of the principal challenges in ST data analysis is spatial clustering, which reveals spatial domains based on the spots within a tissue. Modern ST sequencing procedures typically include a high-resolution histology image, which has been shown in previous studies to be closely connected to gene expression profiles. However, current spatial clustering methods often fail to fully integrate high-resolution histology image features with gene expression data, limiting their ability to capture critical spatial and cellular interactions. In this study, we propose the spatial transcriptomics multi-modal clustering (stMMC) model, a novel contrastive learning-based deep learning approach that integrates gene expression data with histology image features through a multi-modal parallel graph autoencoder. We tested stMMC against four state-of-the-art baseline models: Leiden, GraphST, SpaGCN, and stLearn on two public ST datasets with 13 sample slices in total. The experiments demonstrated that stMMC outperforms all the baseline models in terms of ARI and NMI. An ablation study further validated the contributions of contrastive learning and the incorporation of histology image features. I Introduction Biological tissue samples contain highly complex cellular processes, which are shaped by cell distribution patterns, cell types, cell states, composition, and cell-cell interactions [ 1 ] . Such information is crucial for understanding tissue development, repair, and responses to external signals [ 2 , 1 ] . Single-cell RNA sequencing technology has evolved dramatically in recent years to be more efficient, accessible, and accurate, which enables researchers to obtain deep insights into cellular states and led to the discovery of new cell types [ 3 , 4 ] . However, while single-cell sequencing provides valuable insights, the lack of crucial contextual information limits the understanding of how cells cohabit, interact, and communicate within their environments [ 5 , 6 ] . Spatial transcriptomics (ST) addresses this gap by enabling transcriptome-wide gene expression profiling while preserving spatial context [ 7 , 8 ] , which enables researchers to move beyond the scope of cell clustering to find higher-order tissue structures. Spatial clustering on ST data has become a standard first step for any downstream analysis, such as tissue anatomy visualization, finding domain-dependent biomarkers, and identifying domain-specific molecular regulatory networks [ 9 , 10 , 11 , 12 ] . As the amount of ST data rapidly expands, with technologies such as Visium [ 13 ] , seqFISH+ [ 14 ] , and MERFISH [ 15 ] becoming more accessible, there is a growing need for advanced spatial clustering methods that can handle this kind of complex data. Current ST procedures also contain a high-resolution histology image. While it has been proven in prior studies that histology image features and gene expression are closely linked [ 16 , 17 ] , most current analytical methods do not fully integrate spatial information and histology image with gene expression data. This limitation obstructs the clustering models from extracting critical information from the histology images, such as cell-cell interactions and spatial changes in cell states. Given these challenges, finding patterns in gene expression profiles with spatial and image context remains one of the significant challenges in spatial transcriptomics analysis. To address this, we propose a novel contrastive learning-based deep learning model named the spatial transcriptomics multi-modal clustering (stMMC) model. Our model integrates both gene expression and histology image features through a parallel graph autoencoder, leveraging contrastive learning to regulate feature extraction for each modality. This approach aims to provide a more accurate and comprehensive analysis of spatial transcriptomics data. Fig. 1 : The overall structure of the proposed model, stMMC is plotted here, where trapezoids represent the GCN layer, and rectangles represent extracted features. Dashed lines with double arrowheads represent that both GCNs share the same weight. stMMC takes two data modalities and passes them through the multi-modal parallel graph autoencoder (MPGA), where each modality is regulated by a contrastive learning mechanism. The detailed process of contrastive learning is shown in Figure 2 . The MPGA reconstructs a refined gene expression data, which is then used for spatial clustering. Our contributions of this study are: i) we propose stMMC, a novel multi-modal contrastive learning-based clustering method using high-resolution histology images for spatial clustering in spatial transcriptomics data; ii) we demonstrate that combining gene expression data and learned histology image features improves the spatial clustering performance and paves a new way for future research; iii) we conduct experiments to show the proposed method achieves superior performance on benchmark datasets. Fig. 2 : The detailed process of contrastive learning mechanism for a random spot on any modality is plotted here, where the top row is the corrupted graph, the bottom row is the original graph, and there is three steps of the contrastive learning mechanism: 1) obtaining the learned spot feature by GCN; 2) computing the original local community representation and the corrupted one; 3) assigning positive pairs to the learned features and the community representation from the same graph, and negative pair to the learned feature and the community representation from different graphs. The positive pair is shown in blue and negative pair is shown in red. II Related Works As ST sequencing has emerged as a powerful tool for mapping gene expression profiles with spatial context, researchers have proposed several methods to tackle the spatial clustering problem [ 18 ] . Neural network has emerged as a powerful feature extractor in recent years [ 19 ] . These methods can be divided into three major categories based on how they extract information from the gene expression: 1) conventional-based methods; 2) neural-network-based methods using only location information; and 3) neural-network-based methods using both location information and image features. Zhao et al. proposed BayesSpace, which is a Bayesian statistical model that utilizes spatial neighborhood information to refine the expression data and clustering on a low-dimension representation of the data [ 20 ] . Li et al. proposed BASS (Bayesian Analytics for Spatial Segmentation), which is able to do multi-sample and multi-scale analysis and utilizes a Bayesian hierarchical modeling framework for spatial clustering [ 21 ] . Leiden is a widely used clustering model on graph-structured data that is not designed for ST data [ 22 ] . It is an improved version of the Louvain model and works by iteratively optimizing the partitions to identify densely connected communities within a network. These conventional-based methods are very efficient for training and inferencing, but they do not deal well with the high dimensionality of the ST data. Neural-network-based deep learning methods have shown great results of extracting complex features in many field [ 23 ] . Most neural-network-based methods represent the location information in the form of graphs and utilize some kind of graph neural