miCDER: a context-aware transformer model for joint miRNA-disease entity and multi-level regulatory relation extraction Authors: Jiangcheng Shi, Lijun Wang, Lixue Liu, Jinhao Su Published: 2025 ABSTRACT AbstractBackgroundDysregulation of microRNAs (miRNAs) is closely linked to the progression of diverse human diseases. However, the lack of a standardized, fine-grained dataset of miRNA–disease regulatory interactions and the limited ability of existing methods to capture multi-level regulatory relations hinder full understanding of these mechanisms.ResultsWe constructed a fine-grained, multi-level annotated biomedical dataset covering ten entity types and thirteen relation categories, tailored for modeling microRNA–disease interactions. We then propose miCDER, a transformer-based model that jointly extracts entities and relations through contextual encoding and inter-span transformer attention. miCDER achieves state-of-the-art performance in both named entity recognition (NER F1 = 87.34%) and relation extraction (RE F1 = 77.28%), significantly outperforming all methods, including SpERT, STER, BiLSTM-CRF, and CNN. Compared to the strongest baseline SpERT, our model achieved improvements of 8.84% in NER F1 and 12.35% in RE F1, respectively. To validate generalizability, we evaluated miCDER on two public benchmark datasets and compared it with state-of-the-art methods, achieving competitive performance on CoNLL04 (NER F1 = 88.83%, RE F1 = 72.31%) and superior results on the biomedical ADE dataset (NER F1 = 90.42%, RE F1 = 82.81%). Applied to 27,051 curated miRNA-related text segments from PubMed, miCDER automatically extracted 93,221 high-confidence regulatory triplets to construct the miRNA–disease knowledge graph, MAAD-HCD-KG. Among these, 1,735 targeting relations not found in the microRNA target database (miRTarBase) demonstrate miCDER’s ability to capture literature-supported regulatory associations that are not yet included in manually curated databases.ConclusionOur proposed miCDER model can serve as an effective tool for extracting fine-grained regulatory information from biomedical texts. The resultant knowledge graph constitutes a significant resource for the advancement of research on diseases related to microRNA.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12864-025-12342-x.Keywords: miRNA–disease regulatory interactions, Multi-level regulatory relations, Transformer, Knowledge graph Background MicroRNAs (miRNAs) represent a class of small non-coding RNAs that regulate gene expression at the post-transcriptional level [1]. A growing body of evidence indicates that miRNAs are crucial regulators of a wide array of biological processes, including cell differentiation, proliferation, and apoptosis [2–4]. Aberrant miRNA expression is implicated in a wide range of pathological conditions such as cancer [5, 6], Alzheimer’s disease [7, 8], and cardiovascular disorders [9, 10]. Importantly, the regulatory functions of miRNAs exhibit significant directionality and are highly dependent on the specific context [11]. First, directionality is reflected in the bidirectional causal relationship between miRNAs and diseases: pathological conditions can alter miRNA expression, and miRNAs, in turn, can influence disease progression. For example, miR-125b is downregulated in breast cancer, resulting in increased ETS1 expression, which illustrates the regulatory influence from disease to miRNA [12]. Conversely, miR-125b facilitates glioblastoma progression by inhibiting apoptotic pathways, exemplifying the regulatory influence from miRNA to disease [13]. Second, the functional effects of miRNAs are highly contingent upon their specific biological context, indicating that the same miRNA can have divergent effects under varying biological conditions. For example, miR-21 promotes tumor development through the PTEN/PI3K-AKT signaling pathway in various cancers [14, 15], yet it provides cardioprotective effects by targeting PDCD4 and activating AP-1 in cardiac conditions [16]. These findings underscore the need for fine-grained annotations of miRNA–disease associations that capture both their directionality and mechanistic specificity. In recent years, research on miRNA–disease association prediction has advanced considerably, evolving through distinct methodological phases. Early approaches primarily utilized single data sources such as miRNA functional similarity or disease phenotypic similarity, employing statistical methods and network propagation algorithms [17]. Jiang et al. [18] pioneered the field with a hypergeometric distribution–based algorithm, establishing the foundational “guilt-by-association” principle. This was subsequently advanced by score-based methods such as WBSMDA [19], which integrated multiple similarity measures to predict associations without requiring negative samples. The field then progressed to more sophisticated network-based approaches. RWRMDA [20] applied random walk with restart on functional similarity networks, while semi-supervised methods like RLSMDA [21] addressed the absence of negative samples using regularized least squares. RBMMMDA [22] further expanded prediction capability by leveraging restricted Boltzmann machines to capture specific association types. A significant paradigm shift emerged toward multi-source integration. Path-based methods such as PBMDA [23] exploited heterogeneous network structures to enhance miRNA–disease association prediction, while systematic reviews by Huang et al. [24] highlighted the evolution of fusion strategies and their significance in advancing the field. Notable integrated models include MDHGI [25], which combines matrix factorization with heterogeneous graph inference, and GAEMDA [26], which adopts graph autoencoders for unsupervised feature learning. More recently, graph neural networks have become central for deep representation learning. NIMCGCN [27] employs multi-hop neighborhood aggregation in multilayer GCNs, MMGCN [28] integrates heterogeneous data via multi-view attention with adaptive feature weighting, and HGANMDA [29] constructs miRNA–disease–lncRNA heterogeneous graphs applying hierarchical attention at node and semantic levels, followed by bilinear decoding for association reconstruction. As systematically reviewed by Huang et al. [30, 31], these advances have emphasized cross-modal fusion strategies and proposed standardized evaluation frameworks, both of which improve reproducibility and facilitate systematic assessment across the field. Parallel to the rapid development of computational methods, significant progress has been made in building miRNA-disease association data resources. Initially, databases like miR2Disease [32] compiled miRNA-disease associations through manual curation. Nevertheless, this database is relatively limited in size and lacks comprehensive coverage. In contrast, the human microRNA disease database (HMDD v4.0) [33] now includes over 53,000 experimentally validated entries, and miRTarBase [34] incorporates data from luciferase reporter assays and cross-linking immunoprecipitation sequencing experiments to construct miRNA-target gene interaction networks associated with disease phenotypes. Additionally, the database of Differentially Expressed miRNAs in human Cancers (dbDEMC) [35] is dedicated to the differential expression of miRNAs across various cancer types. Facing the explosive growth of biomedical literature, miRNA-related articles in PubMed had already exceeded 180,000 by 2024, making traditional manual curation impractical and highlighting the urgent need for automated information extraction methods. Early computational approaches to named entity recognition (NER) and relation extraction (RE) primarily relied on rule-based methods, which often fail to capture the complex semantics inherent in biomedical texts. The advent of deep learning improved performance significantly: BiLSTM-CRF [36] combined the sequence modeling capability of bidirectional long short-term memory networks with the global optimization of conditional random fields, enhancing entity boundary detection accuracy, while CNNs [37] excelled at extracting local features in RE, effectively identifying semantic relationships between entity pairs. However, in practical information extraction tasks, pipeline architectures are often adopted, performing NER first, followed by RE. This approach suffers from cascading errors and prevents joint optimization, limiting overall performance [38]. To overcome these limitations, joint extraction has emerged as a research focus, with the Transformer architecture providing a powerful foundation for modeling long-distance dependencies via self-attention mechanisms. Building on this foundation, transformer-based models pretrained on domain-specific corpora, such as BioBERT [39], BioMegatron [40], and SciBERT [41], have significantly improved the performance of biomedical information extraction tasks. In the general domain, span-based models like SpERT [42] have further improved joint entity and relation extraction by employing span enumeration and joint optimization strategies. Recently proposed STER [43] introduces a contrastive student-teacher learning framework for joint entity and relation extraction, where teacher networks leverage privileged features—information available only during training that supports entity and relation detection. The student network learns to integrate knowledge from the teacher networks via contrastive learning, enabling improved modeling of interactions between entities and relations and achieving state-of-the-art performance on general-domain benchmarks. However, existing research faces the following key challenges: First, most miRNA-disease association research primarily focuses on binary classification problems of whether associations exist between miRNAs and diseases. This simple binary relationship modeling cannot capture the complex regulatory mechanisms of miRNA-disease interactions, particularly lacking fine-grained characterization of regulatory directionality, multi-level regulatory networks, and mechanistic specificity. Second, although related databases are expanding, annotation granularity and quality remain limited, mostly modeling simple binary associations without detailed directional or pathway-level information. Third, while current information extraction techniques perform well in general domains, they still cannot explicitly model directionality or multi-hop regulatory cascades, which are characteristic of miRNA–disease interactions. Currently, no specialized joint extraction framework exists for capturing the complex, multi-entity regulatory relationships in this domain. To address these challenges, we present the following key contributions: We propose miCDER, a context-aware transformer model for the joint extraction of miRNA–disease entities and their multi-level regulatory relations. By integrating context-aware span encoding with inter-span contextual transformer attention, miCDER effectively captures complex interactions between entities and relations, achieving state-of-the-art performance. This approach addresses the limitations of traditional pipeline methods and demonstrates robust generalization across both biomedical and general-domain datasets. We propose miCDER, a context-aware transformer model for the joint extraction of miRNA–disease entities and their multi-level regulatory relations. By integrating context-aware span encoding with inter-span contextual transformer attention, miCDER effectively captures complex interactions between entities and relations, achieving state-of-the-art performance. This approach addresses the limitations of traditional pipeline methods and demonstrates robust generalization across both biomedical and general-domain datasets. We construct the first fine-grained biomedical dataset specifically tailored to miRNA–disease regulatory scenarios, encompassing 10 entity types and 13 relation types. This dataset uniquely captures multi-level regulatory mechanisms and directional relationships, filling a critical gap in existing resources and providing a standardized benchmark for future research on complex miRNA–disease interactions. We construct the first fine-grained biomedical dataset specifically tailored to miRNA–disease regulatory scenarios, encompassing 10 entity types and 13 relation types. This dataset uniquely captures multi-level regulatory mechanisms and directional relationships, filling a critical gap in existing resources and providing a standardized benchmark for future research on complex miRNA–disease interactions. We develop the miRNA Association Annotation and Disease Knowledge Graph (MAAD-HCD-KG), leveraging miCDER to process 27,051 curated miRNA-related text segments from PubMed. This process yields 93,221 high-confidence regulatory triplets, including 1,735 previously unrecorded in miRTarBase 2025, demonstrating miCDER’s capability to identify novel, literature-supported regulatory associations. The resulting knowledge graph serves as a rich, structured resource for downstream biomedical research, enabling systematic exploration of miRNA regulatory functions across diverse disease phenotypes. We develop the miRNA Association Annotation and Disease Knowledge Graph (MAAD-HCD-KG), leveraging miCDER to process 27,051 curated miRNA-related text segments from PubMed. This process yields 93,221 high-confidence regulatory triplets, including 1,735 previously unrecorded in miRTarBase 2025, demonstrating miCDER’s capability to identify novel, literature-supported regulatory associations. The resulting knowledge graph serves as a rich, structured resource for downstream biomedical research, enabling systematic exploration of miRNA regulatory functions across diverse disease phenotypes. Methods Initially, we outline the methodology employed in constructing a dataset that encapsulates intricate regulatory associations between miRNAs and diseases. Following this, we provide a comprehensive description of the model’s architecture, alongside the procedures for its training, evaluation, and application within the modeling framework. The entire workflow is depicted in Fig. 1. Complete workflow of miCDER. The model is first fine-tuned on the PubMed Full Text Corpus. After data normalization (shown on the left), text instances are fed into the model for inference to identify multiple entities and their corresponding ternary relations. The inference results are stored in the Neo4j database as the miRNA–disease regulatory knowledge graph, MAAD-HCD-KG, which supports structured representation and flexible query-based exploration of fine-grained biological relations Dataset construction The biomedical relation extraction dataset developed in this study is based on the authoritative HMDD v4.0, which manually curates miRNA–disease association data from PubMed literature sentences reporting such associations, serving as a foundation for systematically investigating their regulatory relationships and underlying molecular mechanisms. Through manual curation, we identified and selected text segments from the literature that encompass comprehensive regulatory pathways, as well as multiple nested clauses and subclauses. This approach was employed to ensure that the data samples demonstrate multi-layered regulatory mechanisms and complex syntactic structures. The objective of this strategy is to augment the model’s ability to discern intricate semantic relationships among biomedical entities that extend across sentence boundaries. Consequently, we successfully extracted 641 high-quality biomedical relational instances. Each data instance comprises semantically complete text segments, which may consist of either single sentences or multi-sentence composites. This design maintains sentence-level contextual information, thereby optimizing multi-granularity semantic understanding and enabling the model to effectively capture cross-sentence dependencies. The annotation schema rigorously delineates 10 entity types and 13 relation types, with comprehensive definitions and statistical summaries detailed in Tables S1 and S2 in the Supplementary material. The entity types encompass fundamental biomedical concepts, such as miRNAs, diseases, and genes, among others, with annotations preserving the original nomenclature from the source literature to ensure semantic accuracy. The relation types focus on nuanced regulatory roles, adhering to the following annotation principles: Relation type classification: The regulatory nature of interactions is determined based on explicit modulatory verbs—verbs such as induce and activate are annotated as promote, whereas suppress and inhibit are annotated as negatively regulate. Relation type classification: The regulatory nature of interactions is determined based on explicit modulatory verbs—verbs such as induce and activate are annotated as promote, whereas suppress and inhibit are annotated as negatively regulate. Directionality determination: The direction of regulatory action is identified by resolving the agent (regulator) and the patient (regulated entity) based on syntactic and semantic context. Directionality determination: The direction of regulatory action is identified by resolving the agent (regulator) and the patient (regulated entity) based on syntactic and semantic context. Pathway integrity: Multi-hop annotation enables the capture of complete regulatory pathways. For example, a sentence might describe a scenario in which a miRNA regulates a gene, which subsequently modulates a signaling pathway, ultimately affecting disease progression. In these instances, intermediate entities, such as genes, proteins, and signaling pathways, are not merely linguistic placeholders but serve as mechanistic mediators within molecular biology. By explicitly representing these intermediates and their interactions (e.g., miRNA ⟶ gene ⟶ pathway ⟶ disease), our approach maintains the continuity of regulatory chains, thereby accurately reflecting biological processes that traverse multiple molecular layers. This preservation of pathway integrity is essential for downstream applications, including causal inference and the construction of biomedical knowledge graphs. Pathway integrity: Multi-hop annotation enables the capture of complete regulatory pathways. For example, a sentence might describe a scenario in which a miRNA regulates a gene, which subsequently modulates a signaling pathway, ultimately affecting disease progression. In these instances, intermediate entities, such as genes, proteins, and signaling pathways, are not merely linguistic placeholders but serve as mechanistic mediators within molecular biology. By explicitly representing these intermediates and their interactions (e.g., miRNA ⟶ gene ⟶ pathway ⟶ disease), our approach maintains the continuity of regulatory chains, thereby accurately reflecting biological processes that traverse multiple molecular layers. This preservation of pathway integrity is essential for downstream applications, including causal inference and the construction of biomedical knowledge graphs. In the upper part of the Fig. 2, a sentence extracted from biomedical literature is displayed, where different biomedical entities are highlighted with distinct background colors. For example, “miR-139-5p” is recognized as a microRNA (light blue), “lncRNA NEAT1” as a non-coding RNA (blue), “liver fibrosis” as a disease (green), and “beta-catenin/SOX9/TGF-beta Pathway” as a signaling pathway (orange). Representative annotation example from the dataset. Different colors indicate distinct entity types, and directional arrows represent the types and directionality of regulatory relations In the lower part of the Fig. 2a structured graph is constructed to illustrate the semantic relations among these entities. Arrows denote the directionality and type of the regulatory relation. For instance: lncRNA NEAT1 is linked to miR-139-5p with a “sponging” relation, indicating a competitive endogenous RNA interaction. lncRNA NEAT1 is linked to miR-139-5p with a “sponging” relation, indicating a competitive endogenous RNA interaction. miR-139-5p is annotated to target the beta-catenin/SOX9/TGF-beta pathway, a key mediator in fibrotic processes. miR-139-5p is annotated to target the beta-catenin/SOX9/TGF-beta pathway, a key mediator in fibrotic processes. The relation miR-139-5p regulates liver fibrosis is inferred via its modulation of the signaling pathway, i.e., the regulation is not direct, but occurs through the miR-139-5p ⟶ pathway ⟶ disease, which is explicitly captured using the “via” relation in the dataset. The relation miR-139-5p regulates liver fibrosis is inferred via its modulation of the signaling pathway, i.e., the regulation is not direct, but occurs through the miR-139-5p ⟶ pathway ⟶ disease, which is explicitly captured using the “via” relation in the dataset. To assess the model’s generalization capabilities and mitigate data bias, we employed a text block-level stratified sampling strategy to divide the dataset into training, validation, and test sets. Specifically, we utilized 13 distinct relation types as stratification variables to ensure that the distribution of relations in the training, validation and test sets accurately reflects that of the original dataset. Each data item consists of either a single sentence or a multi-sentence composite text, which may include multiple semantically related sentences and instances of relations. Ultimately, the training set comprised 461 data items (71.9%), the validation set included 108 data items (16.8%), and the test set contained 72 data items (11.2%). Given that each data item may encompass multiple entities and relations, and that certain entity and relation categories, such as miRNA entities and upregulate relations, are more prevalent in our dataset, we sought to regulate the distribution of each entity and relation type across the training, validation, and test sets to approximate an 8:1:1 ratio. This strategy ensures that categories with high frequency are sufficiently represented in all subsets while preserving the overall diversity of less frequent types. As a result, although the exact ratios may vary slightly for each category, the dataset splits maintain a balanced and representative distribution, thereby facilitating robust model training. The distribution of entities and relations is as follows: the training dataset comprises 2,275 entity instances and 1,893 relation instances, the validation dataset includes 420 entity instances and 346 relation instances, and the test dataset contains 303 entity instances and 240 relation instances. Table 1 provides a detailed overview of the entity distribution within the training, validation, and test datasets. Table 1Distribution of entity types in the training, validation, and test datasetsEntity TypeTrainDevTestMicroRNA87012695Disease57814295Gene2375039Cell57128Pathway58115Protein3864RNA1231815Biological Process2464327Therapy1633Axis52912Total2275420303 Distribution of entity types in the training, validation, and test datasets Similarly, Table 2 provides a comprehensive overview of the distribution of relation types within the training, validation, and test datasets. Table 2Distribution of relation types in the training, validation, and test datasetsRelations TypeTrainDevTestUpregulate3938040Downregulate2975435Associate2844133Biomarker66158Dysregulate611412Negatively correlate62116Negatively regulate1382316Positively correlate3577Promote1181813Regulate1633628Sponging2443Target1142419Via1381920Total1893346240 Distribution of relation types in the training, validation, and test datasets miCDER The miCDER model is designed to perform joint extraction of named entities and their relations between miRNAs and complex human diseases. By integrating explicit entity boundaries into span representations and employing an inter-span contextual transformer attention mechanism to derive relational representations, the model demonstrates enhanced suitability for biomedical relation extraction tasks. Figure 3 illustrates the overall architecture of the miCDER model. The remainder of this section elaborates on four key components: span representation, span classification and filtering, relation classification and filtering, and model training. Architecture of the miCDER model. Biomedical text sentences are processed to generate context-sensitive token embedding vectors using PubMedBERT. Subsequently, all possible text spans are enumerated and filtered for entity classification. Finally, the identified entities are combined with their intervening contexts to support relation classification Span representation Existing research has demonstrated that pre-trained text encoders, utilizing the Transformer architecture, such as BERT [44], can substantially enhance performance through domain-specific fine-tuning. Current domain adaptation strategies are primarily categorized into three approaches: (1) continued pre-training on a domain-specific corpus (e.g., PubMed) using a generalized pre-trained model (e.g., BERT-Base); (2) multi-domain joint pre-training, which involves training a single model on a heterogeneous corpus; and (3) constructing a domain-specific vocabulary and pre-training a model from scratch, thereby excluding generalized domain data. However, Figure 3 Architecture of the miCDER model.Biomedical text sentences are processed to generate context-sensitive token embedding vectors using PubMedBERT. Subsequently, all possible text spans are enumerated and filtered for entity classification. Finally, the identified entities are combined with their intervening contexts to support relation classification. Gu et al. [45] introduced PubMedBERT, pretrained from scratch on PubMed abstracts and full-text articles. Through systematic comparative experiments, they demonstrated that this ab initio pretraining consistently outperforms continued pretraining and mixed-domain approaches across various biomedical Natural Language Processing (NLP) tasks. Consequently, this study utilizes PubMedBERT as a text encoder to enhance the modeling efficacy of the biomedical semantic space. In accordance with Eqs. (1–3), let us define a sentence containing words. The possible spans within a sentence are denoted as , where, and the contextual embedding derived from the PubMedBERT output is represented as : where is the global semantic embedding of [CLS] token, and is the contextual embedding of the word (d is the hidden layer dimension of the model, d = 768 in PubMedBERT-base). For any candidate span, satisfying , the span representation is generated as follows. Local saliency feature extraction Local saliency feature extraction The Max-Pooling operation of Eq. (4) is used to extract salient features from the embedding of all tokens in the span : b.Span width embedding Span width embedding In named entity recognition tasks, the span length plays a crucial role in determining the accuracy of entity identification. Specifically, spans that are excessively long are generally unlikely to correspond to entities, whereas spans that are excessively short may fail to provide sufficient information to fully characterize entities. To address this challenge, we propose the implementation of span width embedding, which integrates a priori knowledge of various span lengths into the model. For a span of width , we extract the corresponding width embedding from a specialized embedding matrix, . This embedding matrix is designed to predefine fixed-size embeddings for each potential span width (), and is optimized via backpropagation to enhance the model’s sensitivity to different span lengths. c.Boundary and global representation Boundary and global representation The boundary information of entities (the start and end position of the span ) is essential for improving the model’s accuracy in entity recognition. Concurrently, the global semantic information, represented as , offers a more comprehensive context for entity representations. Therefore, we utilize an integrated approach that combines boundary and global representations to improve the representation of spans. As demonstrated in Eq. (5), the boundary representation captures the specific details of a span’s start and end positions, incorporating pertinent contextual information. Conversely, the global representation, outlined in Eq. (6), extracts the comprehensive semantic information of the entire sentence via the embedding of the [CLS] token, denoted as : Ultimately, the complete span representation is derived by integrating the global representation , the local saliency feature , the span width embedding , and the boundary representation , as illustrated in Eq. (7): Span classification and filtering After obtaining the representations of candidate spans, we aim to enhance the model’s robustness and computational efficiency by classifying these spans into entity categories and filtering out non-entity spans. This process effectively reduces the search space in the relation extraction task. Specifically, we employ a feed-forward neural network (FFN) to conduct the classification and eliminate spans categorized as NoneEntity, thereby improving both the accuracy and computational efficiency of the model. Span classification To further decrease computational complexity, we limit the maximum span length to 15 tokens, meaning that only candidate spans with 15 or fewer tokens are considered. The architecture of the feed-forward neural network is detailed in Eqs. (8–9): where and are the trainable parameter matrices for entity categorization, and are the biases of the FFN. The transformed hidden representation is denoted as , and contains the predicted scores of the span across all entity categories. Subsequently, the Softmax function is employed to compute the probability distribution of the span category , as illustrated in Eq. (10): Span filtering During the span categorization phase, the model employs a maximum probability decision strategy to determine the most likely entity class to which the span belongs, as illustrated in Eq. (11): Let represent the predicted category of the span . If the span is classified as , it is excluded from further consideration. Conversely, if the span is assigned any other category, it is incorporated into the final set of entity predictions, denoted as , as illustrated in Eq. (12): Relation classification and filtering To construct the set of all possible entity pairs, denoted as , we initiate the process with the set of entities and form the Cartesian product of this set. For each candidate entity pair (,) extracted from , the representation of the relation is derived through the following procedure: Span concatenation Initially, the spanning representations of the two entities are concatenated to form the transformer query vector . To mitigate the risk of overfitting due to high-dimensional concatenation, the spanning representations and undergo a nonlinear mapping, as specified in Eq. (13), to reduce their dimensionality: where and are the spanning representations of entities and , respectively. Inter-span context Extracts the context embedding between entity pairs and utilizes it as both the key and value vectors in the transformer, as demonstrated in Eqs. (14–15): Inter-span contextual transformer attention The Transformer is a neural network architecture specifically designed for processing sequential data. It relies on attention mechanisms to capture dependencies between elements in the input sequence, regardless of their relative distance. By stacking multi-head attention mechanisms and feed-forward networks, the architecture builds deep hierarchical representations, enabling efficient and parallel modeling of complex long-range dependencies. Our model employs the transformer attention mechanism to dynamically capture the contextual dependencies between entity pairs, thereby generating a relation-aware representation , where denotes the dimension of the key vectors, as illustrated in Eq. (16): Inter-span context maxpooling representation Applies maximum pooling to the PubMedBERT context embedding , as indicated by the fluorescent green arrow in Fig. 3, in order to extract the salient semantic feature , as illustrated in Eq. (17): In the subsequent process, the inter-span contextual transformer attention output , the pooled features , and the dimensionality reduced query vector are concatenated to form the final relation representation . It is important to note that if two entities overlap or if there is no intervening text (e.g., in the case of nested entities), then is set to a zero vector, as illustrated in Eq. (18): Upon obtaining the relation representation , akin to span classification, the relation categories of the entity pairs (,) are predicted using a FFN with a Softmax classifier, as demonstrated in Eqs. (19–21): where and are trainable parameter matrices, , are biases, is activation function. Finally, we use the principle of maximum probability to determine the final relation category of entity pairs , as illustrated in Eq. (22): Model training Our experiments employ PubMedBERT as the text encoder, utilizing the AdamW optimizer for training, with an initial learning rate of 5 × 10⁻⁵ and a weight decay coefficient of 0.03 for parameter updates. The training process was conducted over 60 epochs with a batch size of 4, incorporating a dropout rate of 0.3 to enhance the model’s generalization capabilities. To address the issue of lengthy entities common in biomedical texts, the maximum entity span limit was extended to 15 words. During the training process, we implemented a negative sampling strategy to enhance the model’s robustness. This technique, widely adopted in relation extraction tasks, involves constructing negative samples from the dataset, such as pairs of entities that do not hold any target relation, and labeling them as “no relation” for training purposes. This approach improves the model’s ability to distinguish irrelevant entity pairs. The negative sampling strategy was configured with a sampling quantity of 150 for both entities and relations. The experimental setup was deployed on an NVIDIA RTX 3060 computing platform. Our training process uses supervised learning to optimize the model based on the labeled sentences (including entities and their category and relation information). To ensure precise classification of entities and their relations, we incorporate a joint loss function as depicted in Eq. (23): Where denotes the overall loss of the model, denotes the cross-entropy loss of the entity recognition task, and denotes the cross-entropy loss of the relation classification task. Given that the relation classification task is generally more complex than entity recognition, we assign a greater weight to to bolster the model’s capacity to learn relational information effectively. Evaluation metrics We conducted an evaluation of the model’s performance in both named entity recognition and relation extraction tasks. For the named entity recognition task, accurate entity recognition requires both precise alignment of text spans and semantic consistency of type labels. In assessing relation triples, it is necessary to ensure not only the accuracy of relation types but also that the boundaries and types of associated entities adhere to labeling criteria. We employed precision, recall, and macro-averaged F1 score (Macro-F1) as the main evaluation metrics, consistent with the gold standard for joint entity and relation extraction. The macro-averaged approach calculates F1-scores for each category independently and then averages them arithmetically. This ensures equal weighting of low-frequency categories (e.g., the “Sponging” relation with only 24 samples) and high-frequency categories (e.g., the “Upregulate” relation with 393 samples), thereby mitigating the bias caused by the dominance of large categories in micro-averaged metrics. In addition to these primary metrics, we report Accuracy, Matthews Correlation Coefficient (MCC), Area Under the ROC Curve (AUC), and Area Under the Precision-Recall Curve (AUPR) to provide complementary perspectives on model performance. For entity recognition, metrics are computed over all spans appearing in either the ground truth or model predictions, and for relation extraction over all entity pairs appearing in either. This union-based evaluation strategy ensures task-relevant evaluation while avoiding computational complexity and inflation from trivial negatives. Construction of the miRNA–disease regulatory knowledge graph To systematically organize and represent the intricate regulatory relationships identified by our model, we constructed a structured knowledge graph, termed MAAD-HCD-KG, using the Neo4j graph database. In this graph, biological entities are modeled as nodes, and their regulatory or associative interactions are represented as directed edges, enabling efficient storage, retrieval, and analysis of complex miRNA–disease associations. We applied our model miCDER to a corpus of 27,051 curated text segments derived from miRNA-related articles indexed in PubMed. This process yielded 93,221 high-confidence triplets, each comprising two biological entities connected by a directional relation. Provenance information, such as the source sentence and associated PubMed ID, was also retained for each triplet. These structured outputs provide a robust foundation for constructing a fine-grained miRNA regulatory knowledge graph. In Neo4j, the MAAD-HCD-KG comprises over 40,000 unique nodes and more than 90,000 edges, forming a structured and queryable resource for exploring miRNA regulation. It supports flexible querying via the Cypher language, enabling researchers to retrieve specific miRNA-disease associations, explore multi-hop regulatory pathways, and visualize key subnetworks. As illustrated in Fig. 4, a representative subgraph highlights how MAAD-HCD-KG supports efficient querying and exploration of miRNA regulatory interactions. Multi-level interactive exploration example from the miRNA–disease knowledge graph. A Graph retrieved via Cypher query for the disease node “renal cell carcinoma,” highlighting key nodes including the miRNA “hsa-mir-217,” gene “IGF1R,” and the disease itself. B Expanded subgraph obtained by clicking the miRNA node “hsa-mir-217” in (A), revealing its regulatory relationships with multiple diseases, genes, and proteins. C Expanded subgraph obtained by clicking the gene node “IGF1R” in (A), displaying its connections with miRNAs, diseases, and cells. D Detailed information window triggered by clicking the “downregulate” relationship edge in (A), presenting the regulatory type, supporting sentence, and PubMed ID as experimental evidence. The legend indicates node colors representing different entity types Results Performance on miRNA-disease dataset To thoroughly evaluate the performance of our proposed model, we conducted a series of multi-dimensional comparative experiments using miRNA–disease regulatory relationship datasets, with all models trained from scratch. The models used for comparison included SpERT, STER, BiLSTM-CRF, and CNN-based methodologies. The evaluation was conducted from two complementary perspectives: (1) Independent Task: For the NER task, we conducted an enhanced BiLSTM-CRF [46] model, which can handle nested entities using a dynamic hierarchical neural network. For the RE task, we adopted the CNN model proposed by Zeng et al. [47], which leveraged gold-standard entity annotations to minimize error propagation and provided an objective evaluation of the model’s capability in capturing multi-class relationships. (2) Joint Extraction Task: We implemented the popular span-based joint extraction model, SpERT, in addition to the STER model, a teacher-student learning framework, designed to enhance joint extraction performance by leveraging privileged features. Table 3 presents a comparative performance analysis of miCDER and baseline models on our miRNA–disease dataset. A more comprehensive evaluation, including metrics such as accuracy, Matthews correlation coefficient (MCC), area under the ROC curve (AUC), and area under the precision-recall curve (AUPR), is provided in the Supplementary Materials. Specifically, Table S6 shows the results on our miRNA–disease dataset, while Tables S7 and S8 present independent evaluations on the publicly available CoNLL04 and ADE datasets. Table 3Comparative performance analysis of miCDER and comparative modelsTaskMetric(Macro)SpERTSTERBiLSTM-CRF|CNNOur Model(vs. SpERT)(vs. STER)(vs. BiLSTM-CRF|CNN)NERPrecision81.56 ± 1.82%75.86 ± 1.74%69.78 ± 1.53%89.45 ± 1.45%+ 7.89%***+ 13.59%***+ 19.67%***Recall76.94 ± 1.33%71.48 ± 1.58%56.12 ± 1.28%86.63 ± 1.34%+ 9.69%***+ 15.15%***+ 30.51%***F178.64 ± 1.55%72.73 ± 1.62%71.65 ± 1.65%87.48 ± 0.80%+ 8.84%***+ 14.75%***+ 15.83%***REPrecision66.61 ± 2.52%60.84 ± 1.89%59.64 ± 7.13%78.86 ± 3.15%+ 12.25%***+ 18.02%***+ 19.22%***Recall62.63 ± 1.60%56.84 ± 1.68%33.72 ± 2.42%75.12 ± 2.07%+ 12.49%***+ 18.28%***+ 41.40%***F163.91 ± 1.60%57.84 ± 1.59%39.19 ± 2.13%76.26 ± 1.36%+ 12.35%***+ 18.42%***+ 37.07%*** Statistically significant differences are marked with *** (p < 0.001), determined by a two-tailed t-test Comparative performance analysis of miCDER and comparative models + 7.89% *** + 13.59% *** + 19.67% *** + 9.69% *** + 15.15% *** + 30.51% *** + 8.84% *** + 14.75% *** + 15.83% *** Statistically significant differences are marked with *** (p < 0.001), determined by a two-tailed t-test All reported performance metrics represent the mean ± standard deviation across n = 5 independent runs with different random seeds. To rigorously assess statistical significance, two-tailed t-tests (α = 0.001) were performed comparing miCDER to each baseline model (SpERT, STER, BiLSTM-CRF, and CNN). As presented in Table 3, our model significantly outperformed all comparison approaches across both tasks (p < 0.001). In the NER task, our model achieved the highest macro-averaged F1-score of 87.48 ± 0.80%, substantially surpassing the SpERT baseline (78.64 ± 1.55%, Δ=+8.84%, p < 0.001), STER (72.73 ± 1.62%), and BiLSTM-CRF (71.65 ± 1.65%). The precision and recall improvements of + 7.89% and + 9.69% respectively, over SpERT demonstrate substantial enhancement in entity coverage while effectively managing false positives. In the RE task, our model achieved the best performance with 76.26 ± 1.36% F1-score, significantly outperforming the SpERT baseline (63.91 ± 1.60%, Δ=+12.35%, p < 0.001), STER (57.84 ± 1.59%), and CNN (39.19 ± 2.13%). The performance hierarchy across all models demonstrates the effectiveness of our approach in handling the complex challenges of miRNA-disease regulatory relationship extraction, with particularly large improvements in both precision and recall compared to existing joint extraction methods. To ensure experimental completeness, we provide comprehensive performance metrics in the Supplementary Materials, including detailed results for each entity type recognition and relation extraction performance in Tables S3 and S4. Specifically, Table S4 in the Supplementary Materials reports relation classification accuracy based on correctly recognized entities, and we additionally include Table S5 in the Supplementary Materials to evaluate relation classification independently of entity recognition. The relatively lower performance of STER compared to SpERT on our dataset can be explained by several factors: (1) its teacher–student framework was primarily optimized for general text and may lack sufficient adaptation to the domain-specific characteristics of miRNA–disease regulatory relationships; (2) our dataset includes extensive nested entities and multi-level regulatory relationships that likely exceed STER’s original design scope; and (3) its privileged feature mechanism shows limitations when processing the fine-grained relation types defined in our annotation schema. Furthermore, the CNN model’s performance (F1 = 39.19 ± 2.13%) on relation extraction, despite using gold-standard entities, highlights both the intrinsic complexity of miRNA–disease regulatory relationships and the inadequacy of simple feature-based approaches in this domain. These findings underscore the necessity of developing specialized models for miRNA–disease regulatory relationship extraction and confirm the effectiveness of our proposed approach in addressing the unique challenges of this task. Performance on public benchmark datasets To further validate the generalizability and robustness of our model beyond the miRNA-disease domain, we conducted experiments on two widely used benchmark datasets for joint entity and relation extraction. Covering different domains and complexities, these datasets demonstrate our method’s adaptability from general text to specialized biomedical applications, providing strong evidence of its broader applicability. Datasets We selected two representative datasets that offer complementary evaluation perspectives: CoNLL04 for general-domain text processing capabilities and ADE for biomedical text processing performance. CoNLL04: The CoNLL04 [48] dataset contains 1,441 sentences extracted from news articles, representing general-domain text processing challenges. It includes four entity types (People Location, Organization, Other) and five relation types (Work-For, Kill, Organization-Based-In, Live-In, Located-In). We utilize the dataset division established by Gupta et al. [49], comprising a training set of 1,153 sentences and a test set of 288 sentences. CoNLL04: The CoNLL04 [48] dataset contains 1,441 sentences extracted from news articles, representing general-domain text processing challenges. It includes four entity types (People Location, Organization, Other) and five relation types (Work-For, Kill, Organization-Based-In, Live-In, Located-In). We utilize the dataset division established by Gupta et al. [49], comprising a training set of 1,153 sentences and a test set of 288 sentences. ADE: The ADE [50] dataset comprises 4,272 sentences that have been meticulously annotated to identify mentions of drugs, adverse effects, dosages, and the interrelations among these elements. Within this dataset, there are 6,821 annotated relationships that specifically describe “Adverse-Effect” interactions between drug entities and adverse effect entities. For the evaluation of our model, we employ a 10-fold cross-validation, consistent with prior research methodologies. ADE: The ADE [50] dataset comprises 4,272 sentences that have been meticulously annotated to identify mentions of drugs, adverse effects, dosages, and the interrelations among these elements. Within this dataset, there are 6,821 annotated relationships that specifically describe “Adverse-Effect” interactions between drug entities and adverse effect entities. For the evaluation of our model, we employ a 10-fold cross-validation, consistent with prior research methodologies. Comparison with established state-of-the-art models Table 4 presents the evaluation results on the CoNLL04 and ADE test sets. The results for each dataset represent the average of five runs, except for ADE, for which the standard 10-fold cross-validation was applied. For fair comparison, we also reproduced the STER model under our experimental settings, and the corresponding results are listed as “STER (reproduced)” in Table 4. Table 4Results on the CoNLL04 and ADE test setsDatasetModelEneityRelationPRF1PRF1CoNLL04Multi-head83.7584.0683.9063.7560.4362.04Relation-Metric84.4684.6784.5767.9758.1862.68SpERT85.7085.7585.6069.9966.6068.06KdNre85.9285.2185.4971.6672.5172.08TableF-GCN89.2691.6690.4467.1773.7070.28STER89.4588.9989.2174.7074.2674.44reproduced87.7288.6988.2066.2970.3868.28miCDER88.5089.1688.8375.1869.7472.31ADEMulti-head84.7288.1686.4072.1077.2474.58Relation-Metric86.1688.0887.1177.3677.2577.29SpERT87.3290.7489.0076.5981.0878.77KdNre88.5388.0188.2579.5477.7078.59TableF-GCN87.7590.5789.1477.8682.3280.02STER91.3390.8291.0882.1380.3681.24reproduced89.2291.0690.1378.7081.5380.09miCDER89.5891.2990.4281.6684.0082.81 Results on the CoNLL04 and ADE test sets On the CoNLL04 dataset, compared with other baselines such as SpERT with an F1 score of 68.06% and KdNre [51] with 72.08%, miCDER maintains competitive relation extraction results while achieving the highest precision of 75.18%. Its relation extraction performance is also comparable to STER and significantly better than the reproduced STER implementation, as well as TableF-GCN [52]. Moreover, miCDER demonstrates distinct advantages over previous methodologies, including Multi-head [53] and Relation-Metric [54]. These results demonstrate miCDER’s robustness even in general-domain news data with relatively simple relational structures. Given that relation extraction is inherently more challenging than entity recognition, miCDER’s strong F1 score further highlights its effectiveness and capability in capturing complex relational patterns. On the ADE dataset, miCDER delivers comprehensively superior performance, leading all baselines in every key metric. It achieves the highest scores across all three metrics for both entity recognition (Precision, Recall, F1) and relation extraction (Precision, Recall, F1). Compared with other baselines, miCDER’s entity recognition F1 score reached 90.42%, and its relation extraction F1 score of 82.81% significantly outperformed SpERT and KdNre, as well as surpassing TableF-GCN and the reproduced STER implementation. These results demonstrate miCDER’s ability to effectively capture complex biomedical relations. In summary, the proposed miCDER model exhibits strong competitiveness against both TableF-GCN and STER across the CoNLL04 and ADE datasets. In terms of the comprehensive F1 metric, miCDER surpasses TableF-GCN in three key aspects: relation extraction F1 on CoNLL04, as well as entity recognition F1 and relation extraction F1 on ADE. Moreover, it outperforms the reproduced STER across all four F1 scores—entity and relation F1 on both datasets. The significant improvement in relation extraction F1, a core and challenging objective in joint extraction, is particularly noteworthy and underscores the efficacy of our approach. To provide deeper insights into model characteristics, we conduct a comprehensive comparison with STER across two benchmark datasets. On the CoNLL04 dataset, miCDER demonstrates competitive performance despite STER’s advantages, which underscores STER’s proficiency in processing general-domain text with straightforward relational patterns through its use of privileged feature mechanisms. Conversely, on the ADE dataset, the performance disparity is notably reduced: while STER achieves a slightly higher NER F1 score, miCDER significantly outperforms STER in relation extraction. This performance trend underscores fundamental architectural differences: STER’s contrastive learning with privileged features excels in entity recognition within general domains, whereas miCDER, with its inter-span contextual transformer attention and specialization in biomedical domains (utilizing PubMedBERT and span width embeddings), offers considerable advantages for modeling complex relationships. Notably, on our miRNA–disease dataset, which is characterized by nested entities and 13 fine-grained relation types, miCDER consistently surpasses STER. This outcome illustrates that miCDER’s specialized architecture is particularly effective for extracting complex, domain-specific regulatory relationships. This cross-dataset performance variation can be attributed to differences in dataset characteristics. miCDER’s advantages are more pronounced in scenarios featuring complex relationship structures, fine-grained relationship types, and nested entity representations. Such datasets demand fine-grained semantic distinctions and multi-hop contextual understanding, precisely the challenges that miCDER’s cross-span contextual transformer attention mechanism addresses effectively. Conversely, the CoNLL04 dataset exemplifies scenarios with relatively simple relational patterns: it contains only five relation types with clear semantic boundaries, entities are typically short spans such as person names or organizations, and most relations occur as direct pairwise associations (e.g., Person–Organization, Person–Location). These characteristics reduce the need for sophisticated contextual modeling or multi-hop reasoning. These baselines, including STER, SpERT, and TableF-GCN, generally excel in general-domain tasks due to their reliance on mechanisms optimized for direct entity–relation mapping and relatively simpler relational structures. In summary, miCDER delivers the greatest value in scenarios characterized by: High relation granularity and semantic complexity, requiring precise differentiation across diverse relation types. High relation granularity and semantic complexity, requiring precise differentiation across diverse relation types. Heterogeneous and long-span entity distributions, frequently observed in biomedical texts. Heterogeneous and long-span entity distributions, frequently observed in biomedical texts. Multi-hop regulatory cascades, where interactions span multiple molecular layers or biological processes. Multi-hop regulatory cascades, where interactions span multiple molecular layers or biological processes. These findings suggest that, although strong baselines share strengths in handling straightforward relational patterns, miCDER not only maintains competitive performance in such scenarios but also demonstrates distinct advantages in extracting complex, domain-specific regulatory relationships, particularly in the biomedical context. Ablation study To assess the efficacy of each component within the model, a comprehensive ablation study was performed using our dataset of miRNA-disease complex regulatory associations. As illustrated in Table 5, the complete model demonstrated optimal performance on the test set, achieving an F1-score of 77.28%, precision of 82.99%, and recall of 73.45%. The exclusion of the entity boundary and inter-span contextual transformer attention components resulted in reductions in F1-scores by 2.23% and 1.36%, respectively, thereby substantiating the importance of these components in modeling long-distance entity relations. Substituting PubMedBERT with generalized encoder BERT led to the most significant performance decline, with a 12.28% decrease in the F1-score and a substantial reduction in accuracy to 67.07%, underscoring the limitations of general-purpose text encoders when applied to biomedical relation extraction. Furthermore, the removal of inter-span contextual max-pooling led to a 1.53% decrease in the model’s F1-score. In conclusion, each component significantly and positively contributes to the overall performance of the model. Table 5Ablation study of the miCDER modelModel P R F1ΔF1miCDER (Ours)82.99%73.45%77.28%--Entity Boundary81.92%71.14%75.05%−2.23%-Inter-Span Contextual Attention73.13%79.56%75.92%−1.36%-PubMedBERT67.07%64.85%65.00%−12.28%-Inter-Span Contextual Max-Pooling76.95%75.47%75.75%−1.53% Ablation study of the miCDER model Negative sample sampling study To maintain the stability of the training process, we adhere to the experimental findings. of the SpERT model, which suggest that the training process demonstrates greater stability when the parameter for the number of negative samples, N, is set to a sufficiently high value, specifically N ≥ 100. Consequently, we systematically explore the performance differences over the parameter space N ∈ [100, 250] with increments of 10, as illustrated in Fig. 5.The experimental findings indicate that the model achieves optimal performance at N = 150, with Micro-F1 and Macro-F1 scores of 76.20% and 77.28%, respectively. These scores represent improvements of 4.16% and 0.73% compared to when N = 100. Impact of the number of negative samples (N) on Micro-F1 and Macro-F1 scores. The blue solid line and the orange dashed line denote the Micro-F1 and Macro-F1 scores, respectively. Both metrics reach their maximum values at N = 150, with Micro-F1 at 76.20% and Macro-F1 at 77.28%, before declining as N increases Biological consistency analysis The 93,221 fine-grained regulatory triplets extracted by miCDER cover 5,764 unique miRNAs and 6,437 distinct diseases. These triplets formed the basis for a comprehensive miRNA–disease regulatory knowledge graph, elucidating multi-level regulatory pathways across diverse disease phenotypes. Table 6 presents five representative examples of the model’s extraction outcomes. Sentences 1 and 5 exemplify typical conclusion statements characterized by clear and concise structures, which succinctly summarize the study’s key findings. These sentences are particularly suitable for assessing the model’s capability to extract explicit regulatory relationships. Sentence 2 employs a compound sentence structure (e.g., “while…”), incorporating contrasting regulatory directions, thereby enabling an evaluation of the model’s proficiency in managing complex semantic distinctions. Sentences 3 and 4 involve nested regulatory cascades (e.g., “sponging… thereby upregulating…”), which serve to assess the model’s ability to discern multi-level regulatory pathways, thus highlighting its potential in modeling intricate biological regulatory mechanisms. These extraction. Table 6miCDER relation extraction exampleSentenceTripletsPMIDCONCLUSION: Circulating miR-21-5p and miR-126 may be promising prognostic biomarkers for patients with AMI and IR-ATO.[“miR-21-5p”, “biomarker”, " AMI “][“miR-21-5p”, “biomarker”, " IR-ATO”][“miR-126”, “biomarker”, “AMI”][“miR-126”, “biomarker”, " IR-ATO”]36,330,017For example, an increase of the concentration of circulating miR-17 has been linked to lung cancer and metastatic breast cancer, while its decrease has been found in multiple sclerosis and gastric cancer.[“miR-17”, “associate”, “lung cancer”][“miR-17”, “downregulate”, “multiple sclerosis”][“miR-17”, “downregulate”, “gastric cancer”]32,798,803Furthermore, T042819 was significantly upregulated in SGPA, which may upregulate PLAG1 by sponging has-miR195-5p.[“T042819”, “upregulate”, “SGPA”][“T042819”, “upregulate”, “PLAG1”][“T042819”, “sponging”, “has-miR195-5p”][“PLAG1”, “via”, “has - miR195-5p”]34,775,268The findings of our study suggest that miR-383 may inhibit LCN2 and inactivate the JAK/STAT pathway, suppressing the progression of psoriasis in a rat model.[“miR-383”, “negatively regulate”, “LCN2”][“miR-383”, “associate”, “psoriasis”][“miR-383”, “negatively regulate”, “progression of psoriasis”][“miR-383”, “negatively regulate”, “JAK/STAT pathway”]33,819,732CONCLUSION: MSCs-derived exosomal miR-653-5p exerted inhibitory effects on LP progression through targeting BZW2, which provided a novel idea for the therapy of LP.[“miR-653–5p”, “negatively regulate”, “LP”][“miR-653–5p”, “target”, “BZW2”][“ LP “, “via”, " BZW2”]36,473,368 miCDER relation extraction example [“miR-21-5p”, “biomarker”, " AMI “] [“miR-21-5p”, “biomarker”, " IR-ATO”] [“miR-126”, “biomarker”, “AMI”] [“miR-126”, “biomarker”, " IR-ATO”] [“miR-17”, “associate”, “lung cancer”] [“miR-17”, “downregulate”, “multiple sclerosis”] [“miR-17”, “downregulate”, “gastric cancer”] [“T042819”, “upregulate”, “SGPA”] [“T042819”, “upregulate”, “PLAG1”] [“T042819”, “sponging”, “has-miR195-5p”] [“PLAG1”, “via”, “has - miR195-5p”] [“miR-383”, “negatively regulate”, “LCN2”] [“miR-383”, “associate”, “psoriasis”] [“miR-383”, “negatively regulate”, “progression of psoriasis”] [“miR-383”, “negatively regulate”, “JAK/STAT pathway”] [“miR-653–5p”, “negatively regulate”, “LP”] [“miR-653–5p”, “target”, “BZW2”] [“ LP “, “via”, " BZW2”] results significantly surpassed our initial expectations, demonstrating the model’s ability to effectively identify potential regulatory relationships across various types of complex texts. To further assess the biological reliability of the miRNA-target interactions extracted by our model, we conducted a systematic comparison with miRTarBase version 2025 [55], a comprehensive database of experimentally validated miRNA-target relations. It is important to note that miRNA precursors (e.g., miR-21) are frequently used in the. literature to describe regulatory interactions, leading to some miRNA-target associations in our model’s extractions being based on precursor or non-strand-specific nomenclature. Conversely, miRTarBase solely records experimentally validated interactions between mature somatic miRNAs, such as hsa-miR-21-5p, and their respective targets. Therefore, our analysis concentrated on the mature miRNA entries specifically designated as “targeted” within the extracted data. The findings from this comparative analysis are illustrated in Fig. 6. Statistics and comparison of mature miRNA entries regulated by “targeting”. The left pie chart shows that 59.86% (2,486) of the “target” entries extracted by miCDER involve mature miRNAs. Meanwhile, 40.14% (1,667) involve precursor miRNAs. The right Venn diagram compares the mature miRNA targets extracted by miCDER with entries in miRTarBase 2025. It reveals 751 overlapping entries and 1,735 unique entries identified by miCDER Through analysis of the extracted results, a total of 4,153 entries classified under target regulation were identified. After refining based on miRNA nomenclature, 2,486 entries (59.86%) were found to involve mature miRNAs, among which 751 precisely matched experimentally validated records in miRTarBase version 2025. Notably, the remaining 1,735 regulatory associations—absent from miRTarBase—represent novel insights captured by the miCDER. By examining the original publications corresponding to the source sentences, we further confirmed that these extracted targeting relationships are supported by experimental evidence, as demonstrated by the representative case below: miR-214-3p targets BMP2: “Together, miR-214-3p could prevent AS fibroblast osteogenic differentiation by targeting BMP2 and blocking BMP–TGFβ axis” (PMID: 33935961). miR-214-3p targets BMP2: “Together, miR-214-3p could prevent AS fibroblast osteogenic differentiation by targeting BMP2 and blocking BMP–TGFβ axis” (PMID: 33935961). miCDER extracted a target relationship between miR-214-3p and BMP2. Ding et al. [56] experimentally validated this interaction, demonstrating that miR-214-3p binds to the 3′UTR of BMP2 mRNA and significantly reduces luciferase activity in wild-type reporter constructs (p < 0.001), with no effect observed in mutated constructs. Overexpression of miR-214-3p decreased BMP2 mRNA and protein levels in ankylosing spondylitis fibroblasts (P < 0.001), while its inhibition led to increased BMP2 expression. These results confirm the validity of the extracted relationship. Collectively, miR-214-3p hinders osteogenic differentiation by targeting BMP2. (2)miR-182-5p targets TRIM8: “In conclusion, our study indicates that miR-182-5p restricts TNF-α-induced ASM cell proliferation and migration through downregulation of NF-κB activation via targeting TRIM8” (PMID: 32283508). miR-182-5p targets TRIM8: “In conclusion, our study indicates that miR-182-5p restricts TNF-α-induced ASM cell proliferation and migration through downregulation of NF-κB activation via targeting TRIM8” (PMID: 32283508). miCDER identified a target interaction between miR-182-5p and TRIM8. Dang et al. [57] validated this regulatory relationship using bioinformatic prediction and dual-luciferase reporter assays, showing that miR-182-5p significantly reduced TRIM8 luciferase activity (P < 0.001), with no suppression observed in constructs harboring mutated binding sites. Overexpression of miR-182-5p also downregulated TRIM8 expression at both mRNA and protein levels, leading to reduced NF-κB activation and suppression of abnormal ASM cell proliferation and migration. In summary, miCDER effectively identified 30.2% of experimentally validated target interactions and additionally uncovered a significant number of literature-supported target regulatory relationships that are not presently documented in miRTarBase. These results underscore the precision and robustness of miCDER in extracting biologically meaningful and functional regulatory interactions from textual data, offering novel insights into disease mechanisms and establishing a valuable groundwork for the development of future therapeutic strategies. Discussion In the present paper, we designed the miCDER model grounded in a Transformer architecture to perform automated joint extraction of intricate, multi-level miRNA–disease regulatory relationships. This model effectively addresses the limitations of traditional methods by leveraging contextual encoding and cross-entity attention mechanisms, enabling it to capture long-distance dependencies and complex multi-entity interactions. Based on the entities and relationships automatically extracted by our model, we constructed a miRNA–disease regulatory knowledge graph, providing a structured and queryable resource to facilitate biomedical research. However, this study is subject to several limitations: (1) The core Transformer-based architecture requires substantial computational resources during training, particularly when handling large-scale data inference tasks. (2) The annotation of high-quality, fine-grained biomedical text is both time-intensive and costly, which constrains the model’s scalability to larger datasets. (3) As the dataset is annotated only on segments of individual literature paragraphs, the model encounters significant challenges in extracting miRNA–disease regulatory relationships across multiple documents or chapters. It is important to note that our model primarily extracts associations reported in the literature rather than directly predicting entirely new biological relationships, reflecting the inherent imbalance between known and unknown miRNA–disease associations in available databases. To address this challenge, future work will explore leveraging additional literature sources, integrating temporal dynamics and external knowledge graphs, and adopting semi-supervised, self-supervised, or incremental learning paradigms to enhance the identification of rare or unrecorded associations and improve the model’s generalization capabilities. In the future, our work will rely on the constructed MAAD-HCD-KG knowledge graph to provide high-quality training data for prediction models, thereby offering valuable support for miRNA–disease association prediction. Furthermore, the captured multi-level regulatory relationships create opportunities for developing advanced graph neural network methods, such as hypergraph neural networks for modeling complex multi-entity interactions and temporal graph networks for capturing dynamic regulatory patterns. Future research can integrate this rich semantic information with semi-supervised learning frameworks to systematically enhance the ability of prediction models to capture complex regulatory relationships, thereby advancing the development of more comprehensive and accurate miRNA-disease association prediction methods. Conclusions In this study, we propose miCDER, a context-aware transformer model for fine-grained and directional extraction of multi-level miRNA–disease regulatory relationships. Additionally, we developed the inaugural dedicated biomedical dataset specifically aimed at capturing complex miRNA–disease regulatory interactions. Our experimental results show that miCDER achieves robust performance, with a named entity recognition (NER) precision of 91.61% (F1 = 87.34%) and a relation extraction (RE) precision of 82.99% (F1 = 77.28%), outperforming existing baselines. Leveraging our model, we constructed MAAD-HCD-KG, an extensive miRNA–disease regulatory knowledge graph implemented using Neo4j. MAAD-HCD-KG incorporates 93,221 high-confidence regulatory triplets identified from 27,051 curated PubMed text segments, covering 5,764 unique miRNAs and 6,437 distinct diseases. This knowledge graph facilitates structured storage, flexible querying, and network-based analysis of fine-grained regulatory pathways. Notably, miCDER identified 1,735 novel targeting relations absent from miRTarBase, highlighting its capability to uncover literature-supported associations not yet included in manually curated databases. These results demonstrate that our model effectively extracts complex regulatory information from biomedical literature, providing a reliable foundation for downstream knowledge discovery and biomedical research. Supplementary Information Supplementary Material 1. Acknowledgements Not applicable. Abbreviations microRNA Named Entity Recognition Relation Extraction microRNA Target Database Human microRNA Disease Database database of Differentially Expressed miRNAs in human Cancers Natural Language Processing Authors’ contributions Jiangcheng Shi : Conceptualization, Project administration, Supervision, Writing – review and editing.Lijun Wang : Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft.Lixue Liu : Data curation, Software, Visualization.Jinhao Su : Data curation. Funding This project was supported by the Natural Science Foundation of China (62202330). Data availability The Human microRNA Disease Database (HMDD, version 4.0) can be freely accessed at [http://www.cuilab.cn/hmdd] (http://www.cuilab.cn/hmdd). The miRNA–target interactions for comparison were obtained from the microRNA Target Database (miRTarBase, version 2025), available at [https://mirtarbase.cuhk.edu.cn/~miRTarBase/miRTarBase_2025] (https://mirtarbase.cuhk.edu.cn/~miRTarBase/miRTarBase_2025).The code of miCDER and extracted regulatory triplets are publicly available at: [https://github.com/miCDER-Bio/miCDER] (https://github.com/miCDER-Bio/miCDER). Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1. Data Availability Statement The Human microRNA Disease Database (HMDD, version 4.0) can be freely accessed at [http://www.cuilab.cn/hmdd] (http://www.cuilab.cn/hmdd). The miRNA–target interactions for comparison were obtained from the microRNA Target Database (miRTarBase, version 2025), available at [https://mirtarbase.cuhk.edu.cn/~miRTarBase/miRTarBase_2025] (https://mirtarbase.cuhk.edu.cn/~miRTarBase/miRTarBase_2025).The code of miCDER and extracted regulatory triplets are publicly available at: [https://github.com/miCDER-Bio/miCDER] (https://github.com/miCDER-Bio/miCDER).