Integrating scRNA-seq and scATAC-seq with inter-type attention heterogeneous graph neural networks Authors: Lingsheng Cai, Xiuli Ma, Jianzhu Ma Published: 2025 ABSTRACT AbstractSingle-cell multi-omics techniques, which enable the simultaneous measurement of multiple modalities such as RNA gene expression and Assay for Transposase-Accessible Chromatin (ATAC) within individual cells, have become a powerful tool for deciphering the intricate complexity of cellular systems. Most current methods rely on motif databases to establish cross-modality relationships between genes from RNA-seq data and peaks from ATAC-seq data. However, these approaches are constrained by incomplete database coverage, particularly for novel or poorly characterized relationships. To address these limitations, we introduce single-cell Multi-omics Integration (scMI), a heterogeneous graph embedding method that encodes both cells and modality features from single-cell RNA-seq and ATAC-seq data into a shared latent space by learning cross-modality relationships. By modeling cells and modality features as distinct node types, we design an inter-type attention mechanism to effectively capture long-range cross-modality interactions between genes and peaks. Benchmark results demonstrate that embeddings learned by scMI preserve more biological information and achieve comparable or superior performance in downstream tasks including modality prediction, cell clustering, and gene regulatory network inference compared to methods that rely on databases. Furthermore, scMI significantly improves the alignment and integration of unmatched multi-omics data, enabling more accurate embedding and improved outcomes in downstream tasks. Introduction Single-cell sequencing techniques, such as single-cell RNA sequencing (scRNA-seq) and single-cell Assay for Transposase-Accessible Chromatin sequencing (scATAC-seq), have revolutionized the exploration of cellular heterogeneity. These techniques allow scientists to analyze genomics, transcriptomics, epigenomics, and other molecular characteristics of cells in unprecedented detail, revealing crucial insights in diverse fields such as cancer biology, immuno-oncology, and neuroscience [1]. Traditional single-cell studies typically concentrate on a single modality, such as scRNA-seq or scATAC-seq data, which can lead to an incomplete view of cellular information. Those approaches may overlook crucial interactions and information, limiting the holistic understanding of cellular systems [2]. Advancements in sequencing technology have enabled single-cell multi-omics techniques to simultaneously measure multiple modalities within individual cells [3]. These techniques facilitate a more comprehensive analysis of cellular processes by capturing diverse modalities from the same cell. In particular, the integration of data from scRNA-seq and scATAC-seq technologies has become a critical topic of research [4]. This integration necessitates sophisticated representation learning strategies to effectively interpret the vast and complex multi-omics datasets. The representations enable simultaneous profiling of transcriptomic and epigenomic features within individual cells, providing a comprehensive and integrated perspective on cell phenotypes and regulatory mechanisms. For instance, integrated representations of scRNA-seq and scATAC-seq data can be used to identify different cell types, gaining deeper insights into cellular heterogeneity. Additionally, gene regulatory networks (GRNs) inferred from integrated representations help understand the regulatory mechanisms driving gene expression and cellular functions [5]. Several approaches have been developed to integrate scRNA-seq and scATAC-seq data. These methods typically involve aligning or correlating different modalities at the single-cell level to create a unified representation that captures the full spectrum of cell phenotypes. Typical solutions can be classified into three categories: correlation-based methods, such as canonical correlation analysis [6] and matrix factorization [7], which identify shared components across modalities for integration; joint dimensionality reduction methods, like multi-omics factor analysis (MOFA) [8], which reduce the dimensionality of multiple omics datasets while preserving inter-modality relationships; and neighbor-based approaches, such as Seurat v4’s weighted nearest neighbor algorithm [9], which identifies mutual nearest neighbors across modalities to facilitate cohesive analysis. While these approaches show promise, they often rely on predefined relationships between modalities, limiting their ability to uncover complex interactions. This highlights the need for more robust and flexible integration techniques. Recently, heterogeneous graphs have emerged as a powerful tool in multimodal learning. In the context of single-cell multi-omics analysis, researchers encode cells and modality features from different omics, such as genes and chromatin accessibility peaks, as distinct types of nodes and model their relationships within a heterogeneous graph. By capturing the complex relationships and interactions between different types of nodes, heterogeneous graphs facilitate the discovery of intricate biological insights that are otherwise challenging to uncover using traditional methods. Heterogeneous graph attention network models show their effectiveness in node classification tasks, particularly in biological contexts [10]. Furthermore, DeepMAPS employs a heterogeneous graph transformer to analyze cells and genes from single-cell multi-omics data, aiming to infer cell-type-specific gene regulatory networks [11]. However, DeepMAPS integrates multi-omics data primarily based on prior knowledge instead of utilizing heterogeneous graphs to capture the complex cross-modality relationships. Some researchers have begun incorporating modality features, such as genes and peaks, into heterogeneous graphs using prior knowledge. For example, SIMBA links genes and peaks through motifs and utilizes heterogeneous graph learning to generate general embeddings for downstream analysis [12]. Unlike traditional methods, heterogeneous graphs provide greater flexibility by avoiding the need for direct projection of ATAC-seq features onto RNA-seq genes. Instead, they construct heterogeneous graphs based on cross-modality relationships from databases, allowing the integration of multi-omics information within a heterogeneous graph framework. However, the graph construction in these methods heavily relies on prior knowledge, specifically by identifying active transcription factor (TF) binding sites using known motifs. While these approaches have merit, their effectiveness is compromised when motif information in existing databases is incomplete. Additionally, the inherent cellular heterogeneity at the single-cell level can lead to distinct motif expression patterns across different cell subpopulations, complicating motif enrichment analysis and potentially resulting in an incomplete understanding of TFs activity. In those cases, the nodes representing genes and peaks in the heterogeneous graph are not directly connected, forcing information to be propagated solely through intermediary cell nodes. This limitation hampers the ability of these methods to effectively capture long-range dependencies and uncover hidden cross-modality relationships. As a result, the integration of data from scRNA-seq and scATAC-seq remains challenging, with current methods often failing to discover novel cross-modality interactions beyond those already encoded in the motif databases. Consequently, there is a need for more advanced approaches that can autonomously detect and model these complex relationships to improve the integration of multi-omics data. In this paper, to tackle the above challenges and limitations, we propose an inter-type relationship deep learning method on heterogeneous graphs for single-cell Multi-omics Integration (scMI). scMI models cells, genes and peaks from multi-omics datasets within a unified heterogeneous graph. By leveraging a frequency-based multiple-restart Random Walk (RW) strategy, scMI ensures the consistent capture of both structural patterns and data features inherent in the heterogeneous graph. Furthermore, we design an inter-type attention mechanism to capture long-range topological structures across different node types within heterogeneous graphs, effectively enhancing the discovery of accurate cross-modality relationships between peaks and genes. Compared to existing heterogeneous graph methods, scMI excels by autonomously learning and optimizing cross-modality relationships without relying on extensive prior knowledge, making it more adaptable to complex, novel datasets. In our framework, we combine scMI with existing multi-omics integration methods, particularly for unmatched multi-omics datasets, and leverage the inter-type relationship deep learning method to improve cell alignment in these datasets, thus enhancing the accuracy and robustness of multi-omics data integration. To validate scMI, we conduct extensive experiments on real-world datasets involving modality prediction, cell clustering, and GRN inference. Results demonstrate that scMI achieves comparable or superior performance compared to previous methods. Materials and methods Overview Figure 1(a) depicts the overview of our proposed scMI framework. scMI starts by preprocessing scRNA-seq and scATAC-seq data, filtering out genes and peaks that are not expressed in a substantial portion of the dataset. The preprocessed data are then used to construct a heterogeneous graph, modeling the intricate relationships between cells and modality features. To handle the data’s complexity, a frequency-based subgraph sampling method generates smaller, more manageable subgraphs for analysis. scMI then jointly learns low-dimensional embeddings for cells and features, employing an inter-type attention mechanism to highlight key relationships. Finally, scMI incorporates a collaborative learning strategy that simultaneously optimizes the embeddings with downstream tasks, such as cell clustering and GRN inference, for further biological analysis. The architecture of scMI. (a) The overview of scMI. (b) A frequency-based RW algorithm with restart to sample subgraphs. The algorithm samples \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} subgraphs starting from the same node, and the final subgraph is obtained by filtering based on frequency. (c) Representation learning with inter-type attention heterogeneous graph neural networks. Graph convolutions preserve the topological structure information of the subgraph, while the inter-type attention mechanism aims to capture the implicit cross-modality relationships within the multi-omics data. Datasets Peripheral blood mononuclear cells Peripheral blood mononuclear cells (PBMCs) consist of a heterogeneous mix of immune cells, including lymphocytes (such as T cells, B cells, and NK cells), monocytes, and dendritic cells, each playing a critical role in the immune system. These cells are isolated from peripheral blood and are widely used in research. Three PBMC datasets, PBMC 3k, PBMC 7k, and PBMC 10k datasets, derived from healthy donors, are selected for single-cell multi-omics integration. The PBMC 3k (https://www.10xgenomics.com/datasets/pbmc-from-a-healthy-donor-granulocytes-removed-through-cell-sorting-3-k-1-standard-2-0-0) and 10k (https://www.10xgenomics.com/datasets/pbmc-from-a-healthy-donor-granulocytes-removed-through-cell-sorting-10-k-1-standard-2-0-0) datasets are available at 10X Genomics official website. The two datasets are processed using the 10x Genomics Chromium platform, and annotated with the Cell Ranger pipeline. The PBMC 7k datasets can be obtained from NCBI GEO under accession GSE156478. All datasets provide valuable insights into immune cell diversity and serve as essential resources for advancing the field of single-cell multi-omics analysis. Bone marrow mononuclear cells Bone marrow mononuclear cells (BMMCs) are a diverse population of cells found in the bone marrow, consisting primarily of lymphocytes, monocytes, and hematopoietic stem and progenitor cells. These cells are crucial for studying hematopoiesis, immune function, and various hematological disorders. BMMCs serve as an important model for understanding the complex processes of blood cell formation and the regulation of immune responses. The BMMC dataset, sourced from the NeurIPS 2021 competition and available from NCBI GEO under accession GSE194122, served as a benchmark for evaluating model performance in modality prediction and data integration tasks [13]. The GSE194122 dataset includes data from 4 sites and 10 donors. In our study, to ensure the generalization ability of the model, we selected three datasets—s1d2, s2d1, and s3d6—for our experiments, where s stands for sequencing site and d stands for donor number. Human brain tissue The flash-frozen Human healthy Brain Tissue dataset was obtained from 10X Genomics official website(https://www.10xgenomics.com/datasets/frozen-human-healthy-brain-tissue-3-k-1-standard-2-0-0). Nuclei were isolated from a 2605-mg section of the brain and prepared following established protocols for single-cell multiome ATAC and gene expression sequencing. Paired ATAC and gene expression libraries were generated and sequenced to provide key insights into chromatin accessibility and gene expression in human cerebellar cells and is a valuable resource for single-cell multi-omics analysis. Mouse embryonic and adult brain cortex The Mouse Embryonic Brain Cortex (MEBC) dataset, available under the accession number GSE126074, was generated by Chen et al. using SNARE-seq, a pioneering single-cell sequencing technique that simultaneously captures scRNA-seq and scATAC-seq data from the same cell [14]. We select joint profiles of 5081 cells from neonatal mouse brain cortices for analysis. Additionally, we include a dataset of cells from the adult Mouse Brain Cortex, which can be accessed in BioStudies under the ID E-MTAB-11264. This dataset provides further insights into the cellular landscape of the mature mouse brain cortex. Mouse skin tissue and retina cells The Mouse Skin Tissue (MST) dataset, available under the accession number GSE140203, was obtained by Ma et al., offering valuable insights into the regulatory mechanisms that govern skin cell differentiation and function [15]. This dataset provides a detailed view of the transcriptional programs and epigenetic states within mouse skin tissue by integrating scRNA-seq and scATAC-seq data. The combined analysis of RNA-seq and chromatin accessibility data facilitates a deeper understanding of how chromatin potentially influences gene expression during skin development and maintenance. The adult Mouse Retina Cells dataset, available under the accession number GSE201402, was generated using the 10x Genomics Multiome ATAC+RNA kit to capture both scRNA-seq and scATAC-seq data from an adult mouse retina sample. This dataset offers a comprehensive view of the gene expression and chromatin accessibility profiles, enabling the study of various neuronal and non-neuronal cell types in the retina. The detailed statistical information for all datasets above is provided in Supplementary Table 1. Building single-cell multi-omics heterogeneous graphs using scRNA-seq and scATAC-seq data The construction of a single-cell multi-omics heterogeneous graph network involves integrating four distinct types of nodes: cells from scRNA-seq data, cells from scATAC-seq data, genes, and chromatin accessibility peaks, as shown in Fig. 1(a). In the graph, edges are established based on specific biological relationships. First, corresponding cells from scRNA-seq and scATAC-seq data are connected by edges, representing the same underlying biological cell across different modalities. Additionally, edges are formed between scRNA-seq cells and genes that are highly expressed in those cells, reflecting the transcriptional activity captured in the scRNA-seq data. To identify highly expressed genes, we first select highly variable genes within the dataset. For each highly variable gene, we determine its expression threshold by calculating the 95th percentile of its expression values across all cells. A gene is considered highly expressed in a cell if its expression in that cell exceeds this threshold, thereby establishing an edge between the cell and the corresponding gene to represent transcriptional activity. Similarly, scATAC-seq cells are connected by edges to chromatin regions (peaks) that are accessible within those cells, indicating potential regulatory regions. This graph structure allows for the simultaneous representation and integration of multiple omics layers, enabling the exploration of complex cellular relationships and regulatory mechanisms within a unified framework. For multi-omics datasets lacking cell matching information, however, the edges between cells are unknown. To facilitate the integration of unmatched datasets, we employ established algorithms, Seurat v4, GLUE [16], and bindSC [17], to perform initial pairing among cells. All three methods ultimately map cells from different modalities into a shared latent space. In this space, biologically similar cells originating from different modalities are positioned near each other. To establish connections in our heterogeneous graph, we identify the nearest scATAC-seq cell for each scRNA-seq cell in this shared space, based on the cosine similarity of their embeddings. An edge is then established between these paired cells in the heterogeneous graph, facilitating the integration of multi-omics data in the absence of direct cell-to-cell correspondence. Subgraph sampling via Frequency-based Random Walk with Restart In heterogeneous graph representation learning, sampling subgraphs plays a crucial role due to the complexity and size of real-world graphs. Traditional RW methods randomly select nodes and their neighbors, reducing computational complexity while preserving the structural features of the original graph. However, in biological heterogeneous graphs, the nodes are more massive and the graphs are sparser. Traditional subgraph sampling methods exhibit significant uncertainty, which can introduce noise and instability to subsequent learning processes. In addition, disparities in the quantities of different edge types lead to limitations like uneven coverage across node types, biases towards more prevalent edge types, and inefficiencies. To address this, as illustrated in Fig. 1(b), we introduce a Frequency-based Random Walks with Restart (FRWR) algorithm to sample subgraphs. Starting from a node, the algorithm performs multiple restart RWs based on edge weights to generate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} distinct subgraphs, promoting path diversification. For each type of node, the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} most frequently sampled nodes, along with their associated edges, form the final subgraph, which is subsequently used for representation learning. This algorithm reliably captures the most salient aspects of the graph’s topology and feature distribution in a stable manner. Inter-type attention heterogeneous graph neural networks To extract biological information, it is essential to extract the topological and data features of nodes within subgraphs. Heterogeneous graph convolutional networks (HGCNs) extend GCNs to operate on heterogeneous graphs that contain multiple types of nodes and edges. These networks leverage the graph’s complex structure to aggregate feature information from different types of nodes and their interactions, thereby learning rich and comprehensive node representations. The fundamental operation of an HGCN layer can be expressed as follows: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \tilde{A}_{r} $\end{document} is the adjacency matrix for edge type \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ r $\end{document} with added self-loops (where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ A_{r} $\end{document} is the original adjacency matrix for edge type \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ r $\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ I $\end{document} is the identity matrix), \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \tilde{D}_{r} $\end{document} is the degree matrix corresponding to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \tilde{A}_{r} $\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ W_{r}^{(l)} $\end{document} is the trainable weight matrix for edge type \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ r $\end{document} at the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ l $\end{document}th layer and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{R}$\end{document} is a set of edge types in the heterogeneous graph. In current multi-omics integration methods, such as SIMBA, traditional heterogeneous graph learning techniques are employed to integrate information from genes and peaks connected through prior knowledge including motif databases. To uncover potential cross-modality relationships that may be missing from the database, scMI needs to identify and focus on inter-type nodes that are not connected by edges but exhibit co-occurrence or similarity in data expression. To capture these long-range inter-type relationships within subgraphs, we propose an inter-type attention mechanism. Subgraphs based on frequency sampling reflect the relationships between nodes, where any two nodes within a subgraph exhibit high correlation. Based on this, we extend the original subgraph into a fully connected graph by connecting all pairs of nodes in the subgraph, and then compute the attention weights between nodes. For each node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{i}$\end{document}, attention coefficients are computed between node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{i}$\end{document} and its neighboring nodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{j}$\end{document}, considering of the type of edges and nodes: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $h_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $h_{j}$\end{document} are the feature vectors of nodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{j}$\end{document}, respectively. Notably, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{r_{ij}}$\end{document} is a learnable inter-type attention matrix according to the edge type between nodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{j}$\end{document}. The inter-type attention focuses on the relationship between different types of nodes, especially the attention parameter matrix between peaks and genes, which is the key to multimodal integration. Furthermore, these coefficients are normalized across all nodes using the softmax function: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $e_{ik}$\end{document} denotes to all edges that have the same type as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $e_{ij}$\end{document} incident to node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{i}$\end{document}. Node embeddings are obtained by aggregating the features of neighboring nodes, weighted by the normalized attention coefficients: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{A}_{r}$\end{document} is the matrix of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\alpha _{r,ij}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ W_{r}^{(l)} $\end{document} is the trainable weight matrix for edge type \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ r $\end{document} at the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ l $\end{document}th layer. In order to jointly consider the subgraph’s topological structure and long-range inter-type interaction information, we combine equations (3) and (6) to obtain the final aggregation: This approach effectively captures the complex interplay between different node types and their relationships within the heterogeneous graph. By capturing rich contextual information, The aggregated embeddings form a fundamental basis for downstream analysis. Since most scRNA-seq and scATAC-seq data lack labels for supervised learning, scMI employs a contrastive learning approach to refine node embeddings by ensuring that similar nodes within a subgraph are closer in the embedding space, while dissimilar nodes, from outside the subgraph, are farther apart. For each node, positive samples are drawn from other nodes within the same subgraph, leveraging the inherent similarities in their properties or relationships. Negative samples are randomly chosen nodes from other subgraphs. The training employs the Noise Contrastive Estimation (NCE) loss to differentiate between these positive and negative samples. The NCE loss for a node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v$\end{document} and the graph \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $G$\end{document} are given by where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ h_{v^{+}} $\end{document} is the embedding of the positive sample, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ h_{v^{-}} $\end{document} are embeddings of negative samples, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\text{cos}(.)$\end{document} denotes the cosine similarity, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \tau $\end{document} is a temperature parameter. Contrastive learning enables embeddings to capture the multi-omics information inherent in the heterogeneous graph. To enhance the embeddings’ ability to represent data patterns and improve generalization, scMI incorporates modality reconstruction as a supervisory signal during training. Node features and associated edges for 50% of the cells from the scATAC-seq data in training sets are masked in a manner that varies across epochs, ensuring that different nodes are masked each time and promoting robustness in learning. This strategy enhances the model’s ability to generalize by training it to infer missing information and adapt to variability. The reconstruction of masked modality feature values is assessed using the Root Mean Square Error (RMSE). The loss is formulated as follows: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathbf{y}_{i}$\end{document} represents the modality feature values of the masked cells, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \hat{\mathbf{y}}_{i} $\end{document} is the predicted values from the embedding, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ n $\end{document} is the total number of masked cells. By minimizing the RMSE loss, scMI refines the embeddings to accurately capture interconnections among features across modalities. Downstream tasks and evaluation Modality prediction Modality prediction is a key task for assessing whether multi-omics data has been successfully integrated. The objective is to predict one modality of multi-omics data from another, such as inferring scATAC-seq data from scRNA-seq data, or vice versa. This process is similar to modality reconstruction but is carried out on the test set to ensure that the data being predicted is completely new to the model. This task is essential for understanding the relationships between different omics and for imputing missing data in multi-omics datasets. The effectiveness of models performing modality prediction is evaluated using multiple metrics, including RMSE, Pearson correlation coefficient (PCC), and for RNA-seq to ATAC-seq predictions, the Area Under the Receiver Operating Characteristic Curve (AUROC) is additionally calculated. A lower RMSE indicates a smaller discrepancy between the predicted and actual data, reflecting the model’s accuracy in inferring missing modalities. Higher PCC values demonstrate stronger linear correlations between predicted and true values, while higher AUROC scores assess the model’s classification performance in the restoration of chromatin accessibility. Cell clustering Clustering is crucial in single-cell analysis as it allows for the identification of distinct cell populations and states. We employ the k-means algorithm with the embeddings derived from our heterogeneous graph model. The algorithm aims to partition cells into \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} clusters, with each cell assigned to the nearest mean, which acts as a cluster prototype. The optimization employs the within-cluster sum of squares as the loss function: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ N $\end{document} is the number of cells, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ k $\end{document} is the number of clusters, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ h_{i} $\end{document} is the embedding of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ i $\end{document}th cell, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \mu _{j} $\end{document} is the centroid of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ j $\end{document}th cluster, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ z_{ij} $\end{document} is a binary indicator variable that equals 1 if the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ i $\end{document}th cell is assigned to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ j $\end{document}th cluster and 0 otherwise. To comprehensively evaluate the clustering quality, we use metrics such as the Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), and Average Silhouette Width (ASW). ARI measures the similarity between the clustering labels assigned by the algorithm and true labels, while NMI quantifies the mutual dependence between the predicted and true cluster assignments. ASW assesses how well each cell is clustered by calculating the average silhouette value, where higher values indicate that cells are appropriately clustered with their closest counterparts. GRN inference GRN inference is a essential downstream task that aims to unravel the complex network of interactions that control gene expression within a cell. This process involves identifying the regulatory relationships between TFs and their target genes. Transcription factor genes encode proteins that bind to specific DNA sequences and modulate the transcription of other genes. By accurately determining which genes are regulated by specific TFs, researchers can reconstruct the biological networks that dictate cellular functions and behaviors. For this task, we select representative TFs and their corresponding target genes for each cell type. GRN inference can be framed as a link prediction task. We apply a pairwise scoring function that calculates the likelihood of a regulatory link between a TF gene and a target gene. This scoring function takes the embeddings of the TF gene and the potential target gene as input, evaluating their compatibility for forming a regulatory link. The loss function for link prediction is formulated as a binary cross-entropy loss: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ y_{g_{\text{TF}_{i}},g_{j}} $\end{document} represents the ground truth label indicating the presence or absence of a regulatory link between the TFs gene \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ g_{\text{TF}_{i}} $\end{document} and the target gene \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ g_{j} $\end{document}, while \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ \hat{y}_{g_{\text{TF}_{i}},g_{j}} $\end{document} is the predicted probability of such a link as determined by the model. In the context of GRN inference, we assess the performance of model using AUROC for prediction of regulons associated with specific TFs. By focusing on regulons, which are groups of genes regulated by the same TF, we can precisely measure the model’s ability to capture the underlying regulatory relationships governed by these TFs. Higher AUROC values demonstrate that the model accurately identifies the regulatory targets of the TFs, reflecting its effectiveness in reconstructing the gene regulatory network. The ground-truth of GRNs is obtained from three public functional databases, including Reactome [18], DoRothEA [19], and TRRUST v2 [20]. These three databases provide extensive information on gene regulation and biological pathways and have been used for validating GRN enrichment analysis by several single-cell multi-omics GRN inference methods, such as DeepMAPS. Additionally, these databases provide specific annotations for TFs and their target genes for both human and mouse data. To ensure species consistency, we carefully separated the regulatory data into human and mouse categories when selecting the ground-truth, aligning the datasets used for GRN inference with species-specific regulatory information. Among three databases, we gathered 1,186 TFs and 43,178 associated regulons from human data, and 1,072 TFs with 38,665 associated regulons from mouse data. For each dataset, we selected TFs and their associated regulons based on biological matching and filtered them using highly variable genes to serve as the ground-truth for that dataset. This selection process further refines the choice of ground-truth, ensuring that the regulons are representative within the dataset. Results and discussion Overall performance evaluation on matched datasets The results of modality prediction on datasets of four types are shown in Fig. 2(a–c). LS. Lab and Cajal are the winning methods for RNA-seq to ATAC-seq and ATAC-seq to RNA-seq predictions in the NeurIPS 2021 competition, respectively [21]. scJoint [22] and scMoGNN [23] are deep learning-based prediction methods. In the ATAC-seq to RNA-seq prediction, scMI achieves the lowest average RMSE and the highest PCC among all compared methods. Specifically, scMI demonstrates the best RMSE performance in seven out of eight datasets (Supplementary Table 4) and achieves the highest PCC in five datasets (Supplementary Table 5). For the RNA-seq to ATAC-seq task, scMI still achieves the best RMSE performance and ranks second in terms of PCC. Additionally, it obtains the highest average AUROC in five out of eight datasets (Supplementary Table 8), with the overall average being the highest among the compared methods. Overall performance evaluation on matched datasets. (a) Average RMSE and PCC values of scMI and baselines predicting scRNA-seq data from scATAC-seq data across eight matched datasets. The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $x$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $y$\end{document} axes are average PCC and RMSE values, respectively. Error bars indicate the SD of eight datasets. (b) Same as (a), but the algorithms predict scATAC-seq data from scRNA-seq data. (c) Bar plots illustrate the AUROC values of scMI and baselines predicting scATAC-seq data from scRNA-seq data across eight matched datasets. (d) Same as (a), but the results are evaluated by the average ARI and NMI values of cell clustering across seven datasets. (e) Bar plots illustrate the ASW values of cell clustering across seven datasets. (f) Same as (e), but the results are evaluated by the average AUROC values of GRN Inference. All the source data can be found in Supplementary Tables 4–12. For cell clustering, we compare scMI with Seurat v1 [24], MOFA+ [25], DeepMAPS , MultiVI [26], SIMBA, and scEMC [27]. As shown in Fig. 2(d–e), scMI outperforms other methods in ARI, NMI, and ASW. In particular, scMI achieves the highest ARI and NMI in five out of the seven datasets (Supplementary Tables 9–10), and it achieves the highest ASW score across six datasets(Supplementary Table 11). Compared to the single-omic method Seurat v1, scMI significantly improves across all seven datasets, with an average increase of 73.5% ARI. By integrating data from multiple omics, the model captures the complex biological relationships between cells more effectively, leading to more accurate and distinct clustering. Compared to baseline methods, scMI better captures and preserves biological differences between cell types, resulting in clearer and more biologically meaningful clusters. The UMAP visualization of the embeddings learned by SIMBA and scMI is shown in Fig. 3, colored by clusters. Compared to SIMBA, scMI exhibits more distinct clustering boundaries, with less overlap among the embeddings of different clusters. This result is attributed to scMI’s ability to simultaneously capture long-range inter-type relationships and local graph structure, thereby enhancing the distinction between different cell types and avoiding over-smoothing. On the other hand, SIMBA focuses solely on representation learning for biological networks without differentiating downstream tasks. In contrast, our dynamic collaborative learning approach optimizes the loss function to simultaneously refine cell embeddings and minimize the loss for downstream clustering tasks, enabling fine-tuned embeddings tailored for cell clustering and resulting in superior performance. The UMAP visualization of clustering results for SIMBA (ARI=0.836) and scMI(ARI=0.867) on the PBMC 10k dataset, with each color representing a cluster. Even without relying on motif databases, scMI’s clustering results still exhibit clearly boundaries. To validate the GRN inference capability of scMI, we select GRNBoost2 [28], scMTNI [29], DeepMAPS, and SIMBA as baseline methods. Considering real-world scenarios in single-cell multi-omics, we calculate the AUROC for inference of regulons (Fig. 2f). GRNBoost2 infers GRNs solely using scRNA-seq data, while scMTNI, DeepMAPS, and SIMBA integrate multi-omics information to comprehensively infer regulatory relationships. scMI exhibits superior performance across all datasets (Supplementary Table 12), excelling in identifying regulons associated with specific TFs. The enhancement of GRNs inference through multi-omics integration lies in supplementing missing gene information from single-omics data with scATAC-seq data, uncovering more regulatory relationships. Our inter-type attention mechanism specifically targets gene-peak relationships, enabling the discovery of relationships that prior-knowledge-based models may have overlooked. It is observed that scMI does not surpass existing methods in modality prediction and cell clustering on certain datasets. Unlike baseline methods, scMI does not rely on cross-modality relationships derived from motif databases; instead, it aims to discover these relationships directly from the data. When dealing with smaller datasets, the inherent information may be insufficient to fully capture the cross-modality relationships specific to certain cell types, resulting in performance that falls short of existing methods. Nevertheless, scMI maintains robustness across various tasks and achieves the best average performance across multiple metrics in different task scenarios. Performance of omics alignment on unmatched datasets The integration of unmatched multi-omics data, such as scRNA-seq and scATAC-seq, poses considerable challenges due to the inherent difficulty of establishing direct correspondence between cells across different omics. This lack of direct alignment complicates efforts to accurately combine the datasets, as traditional integration methods often rely on matched or paired data. For unmatched multi-omics datasets, scMI leverages existing alignment methods for initial pairing and further integrates the data with the inter-type attention mechanism. Our results demonstrate that scMI not only enhances the alignment between these datasets but also significantly improves the overall integration process, leading to more accurate and biologically meaningful insights from the combined multi-omics data. We assess the alignment quality by visualizing the embeddings of cells from PMBC scRNA-seq and scATAC-seq data using UMAP. The embeddings of both node types are combined, and the degree of mixing is analyzed. As shown in Fig. 4, we visualize the embeddings obtained through Seurat v4, GLUE, and bindSC methods, as well as the concatenated embeddings (Concat) from these three methods. The plots reveal that the enhanced embeddings achieved by scMI based on the integration of these three methods exhibit a higher degree of mixing between cells from the two omics sources, indicating a more seamless integration facilitated by scMI. The UMAP of cell embeddings in unmatched PBMC 10k datasets. The six columns represent embeddings generated by Seurat v4, GLUE, bindSC, their concatenation (Concat), SIMBA, and the proposed scMI. (SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA.) Cells are colored by data source (scRNA-seq or scATAC-seq) in the first row, and by type in the second row. The metric match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{10}$\end{document}, used to assess alignment performance, indicates the ratio of cases where the corresponding scATAC-seq cell is among the 10 nearest neighbors of scRNA-seq cell in the latent space. The corresponding t-SNE visualization is shown in Supplementary Fig. 1. To quantitatively analyze the alignment and integration performance of the model on unmatched datasets, we define a metric match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{k}$\end{document} to evaluate the effectiveness of cell pairing between scRNA-seq and scATAC-seq data in the latent space. For each scRNA-seq cell (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}^{\text{RNA}}$\end{document}), we identify the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} nearest scATAC-seq cells (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{j}^{\text{ATAC}}$\end{document}) based on their embeddings in the latent space. A pairing is considered successful if the true corresponding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}^{\text{ATAC}}$\end{document} is within the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} nearest neighbors of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}^{\text{RNA}}$\end{document}. The match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{k}$\end{document} is defined as follows: where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\text{KNN}_{k}(c_{i}^{\text{RNA}})$\end{document} represents the set of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} nearest neighbors of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}^{\text{RNA}}$\end{document} in the latent space. This metric provides a quantitative measure of the quality of the alignment of the scRNA-seq and scATAC-seq data, with higher match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{k}$\end{document} values indicating better alignment and integration between the two modalities. As shown in Fig. 4, scMI achieves a match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{k}$\end{document} value of 0.2218, significantly outperforming the baseline methods. This substantial improvement indicates that scMI is more effective in accurately aligning scRNA-seq and scATAC-seq cells in the latent space, leading to more successful pairings between the two modalities. To validate the biological functionality of embeddings obtained through unmatched multi-omics methods and scMI, we further visualize the embeddings from the perspective of cell types. As shown in Fig. 4, the embeddings enhanced by scMI produce more distinct boundaries between different cell types. This improvement highlights the ability of scMI to maintain and enhance the biological distinctions among various cell populations. To further evaluate the alignment and integration performance of our model in cell subtype heterogeneity, we perform UMAP visualization on three types of B cells, as shown in Fig. 5. Naive B cells are activated to become transitional B cells, with a subset eventually differentiating into memory B cells. The alignment effectiveness of the four baseline methods has certain limitations, resulting in embeddings that do not adequately distinguish between the three types of B cells after integration. Due to these limitations, the boundaries between the different types of B cells become blurred, making it difficult to achieve accurate clustering and downstream analyzes. In contrast, the embeddings enhanced by scMI exhibit significantly improved alignment, which allows for clearer separation of the three B-cell types. The scMI-enhanced embeddings reveal distinct boundaries between the different cell populations, demonstrating the method’s ability to preserve and highlight biological differences in the data. The UMAP of cell embeddings among three B cell subtypes in unmatched PBMC 10k datasets. The interpretation of each image is consistent with that of Fig. 4. Similarly, match\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $_{3}$\end{document} indicates the ratio of cases where the corresponding scATAC-seq cell is among the three nearest neighbors of scRNA-seq cell in the latent space. The corresponding t-SNE visualization is shown in Supplementary Fig. 2. Additionally, to evaluate the omics alignment and degree of mixing of the scRNA-seq and scATAC-seq data on unmatched datasets, we introduce the integrated Local Inverse Simpson’s Index (iLISI) [30] as an evaluation metric. iLISI assesses the batch mixing quality by measuring the diversity of batch labels within the neighborhood of each cell in the integrated space. For this analysis, we treat the scRNA-seq and scATAC-seq data as two distinct batches and compute the iLISI score for each cell type. The final iLISI score is obtained by averaging across all cell types, providing a comprehensive metric that indicates how well the integration method aligns and mixes cells from different omics sources. Higher iLISI scores suggest better integration, reflecting effective omics alignment. We compare scMI with existing methods for unmatched datasets, including the baeline methods Seurat v4, GLUE, bindSC as well as Harmony [30], scJoint, SIMBA and scBridge [31]. As shown in Table 1, scMI consistently achieves competitive iLISI scores, demonstrating superior integration capability across most datasets. It also shows the highest average iLISI score of 0.810, indicating that it maintains strong alignment performance across various cell types and datasets. scMI’s overall results highlight its robust and effective integration of scRNA-seq and scATAC-seq data, particularly in cases where datasets are unmatched. Table 1Performance comparison on unmatched datasets of iLISI for omics alignment. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold iLISI\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} PBMC 3kPBMC 10kBMMC 1BMMC 2BMMC 3MEBCMSTAverageSeurat v40.5930.5170.6760.5790.7530.4540.4140.570GLUE0.8270.6870.6710.6370.7400.6970.7150.710bindSC0.7210.7150.7390.583 0.764 0.6860.6240.690Harmony0.8530.7560.7590.5100.6940.6490.7460.710scJoint0.8280.7490.6120.5570.6660.5840.6990.671SIMBA*0.8990.7750.6610.7050.709 0.750 0.7430.749scBridge0.859 0.835 0.7140.7210.7340.6530.8210.762scMI(unmatched) 0.890 0.815 0.856 0.757 0.7570.738 0.857 0.810 Performance comparison on unmatched datasets of iLISI for omics alignment. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold Evaluation of downstream task performance on unmatched datasets Enhanced alignment improves the accuracy of data integration and embedding learning, leading to more precise representations of the underlying biological relationships. This, in turn, enables the model to perform better in downstream tasks, such as cell clustering and gene regulatory network inference, by providing a more coherent and biologically relevant foundation for these analyses. To validate the effectiveness of scMI on unmatched datasets, we compare scMI with three methods designed for unmatched datasets. The results are shown in Table 2, with the last row representing the results for the fully matched dataset as a reference. The results demonstrate that scMI not only surpasses all existing state-of-the-art methods for unmatched datasets but also approaches the performance levels achieved with fully matched datasets. This indicates scMI’s superior capability in integrating multi-omics data and improving the accuracy and reliability of downstream analyses, even in the absence of direct cell-to-cell correspondence between different omics. Table 2Performance comparison on unmatched datasets of cell clustering. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold ARI\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} PBMC 3kPBMC 10kBMMC 1BMMC2BMMC 3MEBCMSTAverageSeurat v40.8830.6290.6520.3600.5260.3820.3640.542GLUE0.9130.7310.6570.4500.5530.5640.5780.635bindSC0.9050.7520.7170.5860.5380.5410.6030.663Harmony0.8920.6930.7250.4910.5330.4720.5940.629scJoint0.8680.7500.6560.5370.4910.5490.5830.633SIMBA*0.9090.8220.6750.6940.6270.5980.7330.723scBridge0.9270.8190.7980.7730.5280.5700.7690.741scMI(unmatched) 0.953 0.854 0.860 0.823 0.813 0.742 0.849 0.842 scMI(matched)0.9680.8670.9240.8550.8400.8410.8810.882 Performance comparison on unmatched datasets of cell clustering. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold Furthermore, the GRN inference task also shows excellent performance on unmatched datasets in Table 3, reinforcing scMI’s effectiveness in deciphering complex regulatory interactions and providing robust insights into cellular mechanisms. Table 3AUROC on unmatched datasets of GRN inference comparing regulons from specific TFs. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold AUROC\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} PBMC 3kPBMC 10kBMMC 1BMMC 2BMMC 3MEBCMSTAverageGLUE0.2900.3790.3560.5510.5940.4050.3920.424bindSC0.4190.3760.3900.4230.5030.3420.4490.415SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document}0.5270.6200.4940.4080.6530.3620.4700.505scMI(unmatched) 0.769 0.737 0.654 0.628 0.838 0.667 0.724 0.717 scMI(matched)0.7960.7980.7190.6950.8870.6780.8230.771 AUROC on unmatched datasets of GRN inference comparing regulons from specific TFs. SIMBA\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $^{*}$\end{document} refers to the multi-omics integration analysis module in SIMBA, which takes unmatched multi-omics data as input. The best-performing results for unmatched datasets are highlighted in bold While scMI demonstrates superior performance on both matched and unmatched datasets, computational efficiency remains a critical consideration, especially for large-scale datasets. To further evaluate the efficiency of scMI, we compare its computation time for cell clustering with the baseline methods mentioned above, providing insights into the computational scalability of scMI. We record the computational time of scMI and the baselines across three datasets: PBMC 3k (2,711 cells), MST (5,692 cells), and PBMC 10k (11,898 cells). All experiments were conducted under the same hardware configuration. As shown in Fig. 6, traditional R-based methods (e.g. Seurat and MOFA+) consistently exhibit lower computational complexity due to their reliance on pre-computed databases or comparatively simple model architectures. However, these methods trade computational efficiency for limited performance and flexibility when dealing with complex or unmatched datasets. Computational time comparison between scMI and baseline methods for cell clustering across three datasets (PBMC 3k, MST, and PBMC 10k). The source data and hardware configuration can be found in Supplementary Table 13. Among deep learning methods, scMI demonstrates comparable computation time to other graph-based methods, including SIMBA and DeepMAPS. On the largest dataset, scMI achieves a shorter running time than SIMBA and DeepMAPS. Notably, as the number of cells increases, scMI becomes more efficient compared to SIMBA and DeepMAPS. This improvement can be attributed to the FRWR algorithm and the inter-type attention mechanism, which optimize cross-modality relationship learning and enable scMI to handle larger datasets efficiently. Ablation study Frequency-based subgraph sampling Since the learning of embeddings in scMI relies on subgraph sampling, we evaluate the efficacy of the FRWR algorithm on the PBMC datasets, as shown in Table 4. We compare our method with standard RW and Random Walk with Restart (RWR) approaches. Our results demonstrate that FRWR achieves better performance and stability, ensuring that the sampled subgraphs retain biologically meaningful relationships. Table 4Ablation study on subgraph sampling algorithms. The best-performing results are highlighted in bold ARI\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} m k PBMC 3kPBMC 10kRW––0.7220.683RWR––0.7210.754FRWR1050.9310.815305 0.968 0.8402050.963 0.867 2020.9420.8552080.9500.852 Ablation study on subgraph sampling algorithms. The best-performing results are highlighted in bold The performance of FRWR depends on several key hyperparameters, including the number of subgraphs \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} and parameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} regulating the size of subgraphs. To ensure optimal sampling and representation learning, we systematically optimize these hyperparameters through extensive experiments. We conduct grid search to optimize \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} within predefined ranges: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m \in \{10, 20, 30\}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k \in \{5, 8, 10\}$\end{document}. The results are shown in Table 4. The number of subgraph samples \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} significantly influences the model’s performance by affecting the diversity and stability of captured subgraph structures. As \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} increases, the generated subgraphs better represent the graph’s overall structure, leading to enhanced performance in GRN inference. Specifically, we observe that setting \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m=20$\end{document} strikes a balance between stability and computational efficiency. While further increasing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} (e.g. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m=30$\end{document}) continues to improve subgraph stability, it results in diminishing diversity and increased computational costs. When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} exceeds 30, the diversity of subgraphs sampled from the same starting node decreases significantly during training, which can lead to overfitting. The parameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document}, representing the number of nodes of each type in a subgraph, plays a crucial role in balancing structural detail and information aggregation. If \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} is too small, the subgraph lacks sufficient context to capture complex graph structures, limiting the learning process. Conversely, as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document} continues to increase, additional noise can be introduced into the subgraph. For example, when learning the embedding of an scRNA-seq cell, the inclusion of multiple unrelated scATAC-seq cell nodes can lead to blurred clustering boundaries, adversely affecting the model’s performance in clustering tasks. This noise complicates the representation and can impair the model’s ability to accurately distinguish cell types. In our study, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m=20,k=5$\end{document} provides the best trade-off, enabling scMI to maintain effective representation learning without over-smoothing or excessive noise. To enhance the performance of scMI, we conduct grid search for the other hyperparameters of scMI across different downstream tasks. The tuning experiments for model hyperparameters are presented in Supplementary Figs 3–5, and the hyperparameters for all downstream tasks are summarized in Supplementary Table 2. Inter-type attention mechanism Inter-type attention mechanism automatically captures cross-modality relationships, enhancing scMI’s capability to integrate multi-omics data. In this ablation study, we evaluate the role of the inter-type attention mechanism within scMI by conducting experiments with three different setups: using only HGCN, using only the Inter-type Attention mechanism (I.A.), and using both HGCN and inter-type attention together. These experiments are carried out on the PBMC 3k and PBMC 7k datasets to assess their performance on modality prediction tasks. As shown in Table 5, the combination of HGCN and the inter-type attention mechanism achieved the best performance across all three experimental setups. Table 5Ablation study on inter-type attention mechanism RMSE\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\downarrow $\end{document} ATAC-seq to RNA-seqRNA-seq to ATAC-seq PBMC 3kPBMC 7kPBMC 3kPBMC 7kHGCN0.19700.53170.32761.0310I.A.0.17760.28240.26890.8375HGCN+I.A.0.12140.20650.22380.7796 Ablation study on inter-type attention mechanism HGCN primarily focuses on aggregating local neighborhood information and preserving topological structure. This means that while it can capture structural and connectivity patterns in the graph, it lacks the capability to highlight interactions between distinct node types, such as those between genes and peaks. As a result, the model’s performance is constrained by its inability to fully represent the complex, long-range dependencies that are crucial for cross-modality learning. The inter-type attention mechanism addresses some of these issues by allowing the model to focus on significant cross-modality relationships between different node types, helping to capture these long-range dependencies. By learning cross-modality relationships and combining with information from one modality, the model can more accurately predict another modality. Notably, the improvement brought by the inter-type attention mechanism is more pronounced on larger-scale datasets, indicating that larger datasets enable the model to learn more comprehensive cross-modality relationships. Multi-omics integration in GRN inference To quantify the enhancement of GRN inference by multi-omics information and our model, we employ the single-omic GRN inference method GRNBoost2 for comparison. Three setups include inferring GRN using scRNA-seq data with GRNBoost2, using embeddings generated by scMI as input for GRNBoost2, and inferring GRN with scMI. F-scores for the top \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $K$\end{document} edges with the highest confidence from each setup are presented in Table 6. Comparing the first two experimental setups, using integrated embeddings as input for GRNBoost2 provides more information than scRNA-seq alone, resulting in higher accuracy in GRN generation. The outcomes of the latter two experiments show that our framework and collaborative learning approach, based on omics integration, better adapt to GRN inference. Table 6Ablation study on multi-omics integration. The best-performing results are highlighted in bold AUROC\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} BMMC 1BMMC 2BMMC 3GRNBoost20.4400.5920.524scMI(embedding)0.6910.6010.848scMI 0.719 0.695 0.887 Ablation study on multi-omics integration. The best-performing results are highlighted in bold Conclusion In this paper, we introduce scMI, a novel deep learning method based on heterogeneous graphs for single-cell multi-omics integration. This approach effectively integrates scRNA-seq and scATAC-seq data, addressing the limitations of traditional methods that rely heavily on motif databases. By employing a frequency-based multiple Random Walk with Restart strategy for subgraph sampling, scMI effectively balances the representation of different node types while maintaining biological interpretability. A key innovation of scMI is the introduction of an inter-type attention mechanism, which captures long-range cross-modality relationships between different node types, such as genes and chromatin accessibility peaks. This mechanism enhances the model’s ability to uncover complex biological interactions that may be missed by other approaches. To evaluate the effectiveness of our model, we conduct extensive experiments to compare scMI with baselines. The results on various datasets consistently demonstrate the superiority and robustness of scMI in tasks including cell clustering and GRN inference. Notably, scMI enhances the alignment and integration of unmatched multi-omics data, leading to more accurate and biologically meaningful insights. The method’s ability to capture long-range cross-modality relationships without the need for motif databases further underscores its robustness and adaptability. Key Points We propose scMI, a novel heterogeneous graph embedding approach to integrate scRNA-seq and scATAC-seq data. Our framework models cells and modality features from multi-omics datasets onto heterogeneous graphs for representation learning, eliminating the reliance on motif databases. We propose scMI, a novel heterogeneous graph embedding approach to integrate scRNA-seq and scATAC-seq data. Our framework models cells and modality features from multi-omics datasets onto heterogeneous graphs for representation learning, eliminating the reliance on motif databases. We introduce an inter-type attention mechanism to capture long-range cross-modality relationships on heterogeneous graphs, enhancing the biological interpretability of gene-peak interactions. We introduce an inter-type attention mechanism to capture long-range cross-modality relationships on heterogeneous graphs, enhancing the biological interpretability of gene-peak interactions. For unmatched datasets, scMI builds upon existing alignment methods and employs the inter-type attention mechanism to adjust misaligned pairings, enhancing alignment accuracy. For unmatched datasets, scMI builds upon existing alignment methods and employs the inter-type attention mechanism to adjust misaligned pairings, enhancing alignment accuracy. Extensive evaluations are conducted to validate the effectiveness of scMI. Visualization elucidates the biological interpretability and insights gained from our study. Extensive evaluations are conducted to validate the effectiveness of scMI. Visualization elucidates the biological interpretability and insights gained from our study. Supplementary Material