Text-Attributed Knowledge Graph Enrichment with Large Language Models for Medical Concept Representation Thanks: Corresponding author. Mohsen Nayebi Kerdabadi, Arya Hadizadeh Moghaddam Abstract In electronic health record (EHR) mining, learning high-quality representations of medical concepts (e.g., standardized diagnosis, medication, and procedure codes) is fundamental for downstream clinical prediction. However, robust concept representation learning is hindered by two key challenges: (i) clinically important cross-type dependencies (e.g., diagnosis-medication and medication-procedure relations) are often missing or incomplete in existing ontology resources, limiting the ability to model complex EHR patterns; and (ii) rich clinical semantics are often missing from structured resources, and even when available as text, are difficult to integrate with KG structure for representation learning. To address these challenges, we present MedCo , an LLM-empowered graph learning framework for medical concept representation. MedCo first builds a global knowledge graph (KG) over medical codes by combining statistically reliable associations mined from EHRs with type-constrained LLM prompting to infer semantic relations. It then utilizes LLMs to enrich the KG into a text-attributed graph by generating node descriptions and edge rationales, providing semantic signals for both concepts and their relationships. Finally, MedCo jointly trains a LoRA-tuned LLaMA text encoder with a heterogeneous GNN, fusing text semantics and graph structure into unified concept embeddings. Extensive experiments on MIMIC-III and MIMIC-IV show that MedCo consistently improves prediction performance and serves as an effective plug-in concept encoder for standard EHR pipelines. 1 Introduction Electronic health records (EHRs) encode a patient’s medical history as a high-dimensional and sparse sequence of diagnosis (dx), medication (rx), and procedure (px) concepts, which enable a broad range of clinical prediction tasks 8 ; 27 ; 26 ; 28 . A cornerstone in EHR mining is to learn medical concept representations that can capture the meaningful but complex dependency patterns (e.g., dx-rx treatment/contraindication, dx-px indication/care pathways, or dx-dx comorbidity) to benefit downstream outcome prediction. Therefore, utilizing knowledge graphs to model heterogeneous concepts and their relations becomes an appealing direction to improve concept representation learning. However, the KGs most commonly used in practice are largely derived from existing medical ontologies, which either encode within-type parent-child hierarchies (e.g., ICD) or provide limited cross-type semantics (e.g., UMLS) 1 1 1 A broader comparison of existing medical ontologies is provided in Appendix A.1 , Table 6 . . Although large language models (LLMs) have recently emerged as a promising tool for enriching KGs by proposing missing relations and generating structured semantic descriptions, unconstrained prompting can produce plausible but unsupported edges and inconsistent outputs for the same concepts. Furthermore, because many clinically meaningful relations are context-dependent and vary across cohorts, care settings, and time, LLM-based KG enrichment must validate proposed relations against what is actually observed in the target EHR dataset, ensuring that edges are not only clinically interpretable but also empirically supported. These requirements create a key tension: although LLMs encode broad biomedical knowledge, KG induction for clinical modeling must remain evidence-grounded, type-aware, and globally consistent. Beyond which relations a KG should contain, a second challenge is how to incorporate node and edge information into concept embedding. Since each medical concept can be associated with rich LLM-generated semantics, such as indications, mechanisms of action, roles in care, and contextual constraints, it is natural to model the KG as a text-attributed graph in which nodes and edges are paired with descriptive text. This representation has the potential to combine the relational expressiveness of KGs with the semantic richness of LLM-generated knowledge. However, effectively learning from an LLM-enriched text-attributed graph is nontrivial. Graph neural networks (GNNs) excel at aggregating graph structure but are not designed to interpret long-form medical language. In contrast, LLMs can encode nuanced textual semantics but do not explicitly enforce or exploit global relational constraints. This challenge motivates a co-learning mechanism in which (i) an LLM encodes node- and edge-level textual context, while (ii) a relation-aware GNN propagates and refines these representations over the KG structure. In this way, textual semantics and graph structure can be jointly captured and aligned under task supervision. In this work, we propose MedCo , which addresses the above challenges by constructing a clinically interpretable and empirically supported KG and learning text-attributed concept embeddings through a KG-LLM co-learning framework. Specifically, we first build a global heterogeneous KG over diagnosis, medication, and procedure codes by combining (i) statistically reliable associations from EHR data, such as intra-visit co-occurrences and cross-visit temporal transitions, with (ii) structured, type-constrained LLM prompting that assigns directed relation types and calibrated confidence scores. Using the resulting text-attributed KG, enriched with node descriptions and edge rationales, we then jointly train a LoRA-tuned LLaMA text encoder and a heterogeneous GNN to fuse text-derived clinical semantics with graph-derived relational structure into unified concept embeddings for downstream EHR prediction. Our contributions are as follows: • We introduce an LLM-assisted pipeline to construct a global heterogeneous KG by combining EHR-derived co-occurrences and temporal statistics with type-constrained relation inference, yielding clinically meaningful relations among diagnosis, medication, and procedure codes. • We enrich the KG into a text-attributed graph with LLM-generated node and edge semantics, including textual rationales, relation labels, confidence scores, and EHR-derived statistics. • We propose MedCo , a KG-LLM co-learning framework that jointly fine-tunes a LLaMA text encoder and a heterogeneous GNN to learn unified medical concept embeddings. • We show that MedCo serves as an effective plug-in concept encoder for standard EHR backbones through extensive experiments on MIMIC-III and MIMIC-IV for sequential diagnosis prediction. 2 Methodology 2.1 Notation and Problem Definition EHRs. We represent the EHR record for patient j\in\mathcal{J} as a sequence of T_{j} clinical visits, denoted by \mathcal{X}_{j}=\{V_{j,t}\}_{t=1}^{T_{j}} . Each visit V_{j,t} is a set of N_{j,t} medical codes, V_{j,t}=\{c_{i}\}_{i=1}^{N_{j,t}} , where each code c_{i} corresponds to a diagnosis ( dx ), a medication ( rx ), or a procedure ( px ). For brevity, we omit the patient index j and visit index t . Healthcare Predictive Tasks. Given a patient’s visit sequence \mathcal{X}_{j}=\{V_{1},V_{2},\ldots,V_{T_{j}}\} , the objective is to predict clinical outcomes. These may involve binary tasks (e.g., mortality) or multi-label classification (e.g., diagnosis prediction). In this paper, we focus on predicting diagnosis codes for the next visit V_{T_{j}+1} , a comprehensive task aimed at identifying potential diseases for future encounters. Figure 1: Overview of MedCo . (1) Extract intra-visit co-occurrence and next-visit transition evidence from EHRs, and retain well-supported code pairs. (2) Use type-constrained, evidence-conditioned LLM prompting to assign directed relation types (with confidence and rationales), thereby constructing a heterogeneous KG. (3) Enrich nodes with LLM-generated concept descriptions and edges with relation metadata. (4) Jointly train a LoRA-tuned LLaMA encoder and a relation-aware GNN to learn medical concept embeddings for downstream prediction. 2.2 Method Summary We present MedCo , which constructs an evidence-grounded, LLM-guided heterogeneous KG and learns plug-in medical concept embeddings through an LLM-GNN co-learning framework. As illustrated in Figure 1 , the framework consists of four steps: (1) Evidence extraction. Compute co-occurrence and next-visit transition statistics, and retain statistically significant code pairs. (2) KG induction. Use type-constrained, evidence-conditioned LLM prompting to assign directed relation types, confidence scores, and rationales, thereby constructing a heterogeneous KG. (3) KG enrichment. Enrich the KG with LLM-generated node descriptions and edge metadata, including relations, supporting evidence, confidence scores, and rationale embeddings. (4) Co-learning. Jointly train a LoRA-tuned LLaMA encoder and a relation-aware GNN to fuse textual semantics and graph structure into concept embeddings for downstream EHR prediction. 2.3 LLM-Guided Construction of Global Health Knowledge Graph We construct a clinically grounded heterogeneous KG by inducing relations among diagnosis, procedure, and medication codes using longitudinal EHRs. The pipeline combines EHR-derived co-occurrence and temporal evidence with evidence-conditioned LLM prompting to infer directed relation types. It proceeds in two stages: (1) extract pair-level statistical evidence and (2) perform evidence-supported LLM relation inference. 2.3.1 Pairwise Statistical Evidence Extraction from EHRs We extract empirical evidence for candidate code pairs from EHRs in two steps: (i) computing intra-visit co-occurrence and next-visit transition statistics, and (ii) filtering to retain well-supported, clinically meaningful pairs. Each patient is represented as a sequence of visits, where each visit is a de-duplicated set of diagnosis (dx), procedure (px), and medication (rx) codes. These visit-level code sets are used to compute the corresponding marginal frequencies and transition statistics. Pair-level evidence. For a directed pair (c_{i}\!\rightarrow\!c_{j}) , we compute three signals: a smoothed conditional probability, pointwise mutual information (PMI), and a chi-square test of dependence. We use a common formulation for both intra-visit co-occurrence and next-visit transitions. Let x(\cdot) denote the appropriate count function: for co-occurrence, x(c_{i}) is the number of visits containing c_{i} , and x(c_{i},c_{j}) is the number of visits in which c_{i} and c_{j} co-occur; for transitions, x(c_{i}) is the number of source-visit occurrences of c_{i} , and x(c_{i},c_{j}) is the number of observed transitions from c_{i} in V_{t} to c_{j} in V_{t+1} . Let \mathcal{C} denote the set of distinct codes and |\mathcal{C}| its cardinality. (1) Smoothed conditional probability. P(c_{j}\mid c_{i})\;=\;\frac{x(c_{i},c_{j})+\alpha}{x(c_{i})+\alpha|\mathcal{C}|}, where \alpha is a Laplace smoothing constant used to mitigate sparsity and avoid zero probabilities. (2) PMI-style association. \mathrm{PMI}(c_{i},c_{j})\;=\;\log_{2}\!\left(\frac{p(c_{i},c_{j})}{p_{\text{src}}(c_{i})\,p_{\text{tgt}}(c_{j})}\right), where p(\cdot) denotes empirical probabilities estimated from counts, and p_{\text{src}} and p_{\text{tgt}} are the corresponding marginal probabilities. For intra-visit co-occurrence, p_{\text{src}}=p_{\text{tgt}} . (3) Statistical significance. We compute a chi-square p -value from a 2{\times}2 contingency table to test dependence between c_{i} and c_{j} for both intra-visit co-occurrence and next-visit transition settings. Evidence Filtering. For each unordered pair (c_{i},c_{j}) , we compute intra-visit and temporal evidence (support counts, conditional probabilities, PMI, and \chi^{2} p -values) and consolidate them into a unified table covering both directions (c_{i}\!\rightarrow\!c_{j}) and (c_{j}\!\rightarrow\!c_{i}) . We discard pairs with low support, weak association (low conditional probability/PMI), or non-significant dependence (e.g., p>0.05 ), yielding a set of statistically reliable candidates. These pairs are then passed to the LLM for relation-type assignment to construct a clinically meaningful, evidence-grounded KG. A detailed breakdown is provided in Appendix A.2 . 2.3.2 Evidence-Supported LLM Prompting for Medical Relationship Induction Given the final set of code pair candidates, the last stage of our pipeline assigns each pair a clinically meaningful semantic relationship using an LLM. We design a structured, evidence-supported prompting framework that guides the LLM to choose the most appropriate relation between two medical concepts from a predefined, type-specific relation pool, explicitly grounding its decisions in clinical knowledge and EHR-derived evidence. Type-Constrained Relationship Pool. Each code-type combination (dx–dx, rx–dx, px–dx, rx–rx, px–px, px–rx) is associated with a curated set of candidate relations grounded in clinical practice (Appendix A.3 ). The resulting relation inventory spans key semantic axes, including causality (e.g., causes ), temporal progression (e.g., leads_to ), therapeutic intent (e.g., treats , combination_therapy_with ), diagnostic usage (e.g., diagnostic_of ), workflow structure (e.g., prerequisite_for ), and safety considerations (e.g., contraindicated_for ). When neither clinical priors nor statistical evidence supports a confident assignment, the relation inventory provides conservative abstention labels ( no_significant_relation , cannot_decide ). Collectively, these relations define a concise, interpretable semantic space for modeling clinically meaningful dependencies in EHRs. Figure 2: Prompt for Medical Relationship Induction. Structured Prompting. For each code pair, we construct a prompt (Figure 2 ) that includes: (i) the code identifiers, textual names, parent categories, and marginal frequencies for both codes; (ii) directional statistical evidence from intra-visit co-occurrence and next-visit transitions (8 metrics) together with a brief metrics glossary; (iii) candidate relations permitted by the corresponding code-type combination (e.g., dx-rx); and (iv) explicit decision rubrics and confidence-calibration guidelines. The LLM is instructed to prioritize clinical plausibility, using statistical signals as supporting evidence, and to return a strict JSON object containing: (1) a single relation label, (2) an oriented KG triple, (3) a calibrated confidence score, and (4) a 50–60 word reasoning explaining the clinical rationale and how the co-occurrence and temporal statistics support or challenge the decision. This inference stage transforms medical knowledge encoded in the LLM, guided by statistically validated evidence, into a structured, relation-typed, and clinically interpretable knowledge graph that unifies EHR-derived evidence with clinical reasoning. Clinical expert audit of induced edges. To assess potential hallucination in LLM-inferred relations, we conducted a targeted clinical expert audit of the constructed KG. We sampled 50 edges by selecting 5 edges from each of the 10 most frequent predicted relation types. Two independent frontline clinicians from the University of Kansas Health System reviewed each edge under general clinical knowledge and rated its correctness on a 1–5 scale. The mean score, averaged across the two reviewers for each edge, was 4.84\pm 0.29 , indicating high agreement with the clinical validity of the sampled relations. Full protocol details and the relation-wise summary are provided in Appendix A.5 . 2.4 LLM-Based Contextual KG Enrichment Starting from the constructed medical KG, we further enrich nodes and edges with contextual information derived from a large language model (LLM). The LLM is treated as a high-coverage medical knowledge base: it provides clinically detailed descriptions for individual medical concepts and augments edge semantics using relation reasoning and EHR-derived evidence. The resulting features supply a semantic and knowledge-grounded context for downstream graph learning. Node-Level Enrichment. For each medical code (diagnosis, procedure, or medication), we use a type-specific prompt to generate a concise, clinically focused description using an LLM (e.g., typical presentation, indications, role in care, and key nuances; see Appendix A.6 ). We attach this description to the corresponding KG node as textual attributes, which are subsequently encoded by MedCo during joint KG-LLM co-learning. Edge-Level Enrichment. For each candidate code pair, the previous subsection yields (i) an oriented relation label, (ii) an LLM-generated confidence score and free-text rationale, and (iii) eight EHR-derived statistical metrics (co-occurrence and temporal transition statistics). We combine these signals to construct informative edge features. 2.5 MedCo : Co-Learning of LLM and GNN for Medical Concept Representations Given the constructed KG, we learn unified node representations with MedCo , a co-learning medical concept representation framework that jointly fine-tunes a large language model (LLM) and a heterogeneous graph neural network (GNN). The LLM encodes each node’s textual description into an initial embedding, which is then passed to the GNN for neighborhood message passing over the KG. We train both components end-to-end; for the LLM, we use LLaMA-1B and fine-tune it with low-rank adaptation (LoRA). This co-learning setup combines the LLM’s semantic knowledge with the GNN’s ability to incorporate relational structure and EHR-derived evidence. Architecture. Let G=(V,E) denote the global heterogeneous medical knowledge graph, where V is the set of medical concepts and E is the set of directed edges. Each node i\in V represents a medical concept with node type \tau(i)\in\mathcal{T} , where \mathcal{T} denotes the set of node types (diagnosis, medication, and procedure), and is associated with a textual description s_{i} (Appendix A.6 ). Each directed edge (j\!\to\!i)\in E is associated with a relation type r_{ji}\in\mathcal{R} , where \mathcal{R} is the set of relation types, and an edge feature vector \mathbf{e}_{ji}\in\mathbb{R}^{d_{e}} . A LLaMA-based text encoder f_{\theta} , fine-tuned with LoRA adapters, maps the description s_{i} to a text embedding, which is then projected into the GNN hidden space using a type-specific linear map \mathbf{W}_{\tau(i)}\in\mathbb{R}^{d\times d_{L}} : \displaystyle\mathbf{z}_{i}^{\text{text}} \displaystyle=f_{\theta}(s_{i})\in\mathbb{R}^{d_{L}}, (1) \displaystyle\mathbf{h}_{i}^{(0)} \displaystyle=\mathbf{W}_{\tau(i)}\,\mathbf{z}_{i}^{\text{text}}\in\mathbb{R}^{d}. These embeddings initialize a relation-aware heterogeneous GNN g_{\phi} that incorporates both relation types and edge features during message passing. Let \mathcal{N}(i)=\{j\in V:(j\!\to\!i)\in E\} denote the set of in-neighbors of node i . For layer \ell\in\{0,\dots,L-1\} , node states are updated as \displaystyle\mathbf{u}_{ji}^{(\ell)} \displaystyle=\mathrm{MSG}^{(\ell)}\!\left(\mathbf{h}_{i}^{(\ell)},\mathbf{h}_{j}^{(\ell)},\mathbf{e}_{ji},r_{ji}\right), (2) \displaystyle\mathbf{m}_{i}^{(\ell)} \displaystyle=\mathrm{AGG}^{(\ell)}\!\left(\left\{\mathbf{u}_{ji}^{(\ell)}:j\in\mathcal{N}(i)\right\}\right), \displaystyle\mathbf{h}_{i}^{(\ell+1)} \displaystyle=\mathrm{UPDATE}^{(\ell)}\!\left(\mathbf{h}_{i}^{(\ell)},\mathbf{m}_{i}^{(\ell)}\right). Here, \mathrm{MSG}^{(\ell)}(\cdot) denotes the message function that computes the edge-specific message \mathbf{u}_{ji}^{(\ell)} from node j to node i ; \mathrm{AGG}^{(\ell)}(\cdot) denotes a permutation-invariant aggregation operator over incoming messages; and \mathrm{UPDATE}^{(\ell)}(\cdot) denotes the node-state update function at layer \ell . Intuitively, the LLM provides semantic information from textual concept descriptions, while the heterogeneous GNN incorporates relational structure and EHR-derived evidence. Joint end-to-end training fuses these two sources into clinically meaningful concept representations. LLM-GNN Training Strategy. We jointly train the LLM-based node encoder and the GNN, but a practical challenge arises during mini-batch training. Each batch contains multiple patients, and the union of diagnosis, medication, and procedure codes appearing across their visit sequences can be large. If we update the LLM representations for all such codes at every iteration, the cost becomes prohibitively expensive, especially given the size of the medical code vocabulary. A naive workaround is to update only a random subset of batch-activated codes per iteration, but this has two drawbacks: (i) rare yet clinically important codes may receive very few or no updates, and (ii) highly frequent codes may be updated disproportionately often, leading to unbalanced adaptation of the LLM. To address this issue, we maintain lightweight per-code statistics that track how often each code appears in the training data and how many times its LLM representation has been updated. We then adopt a two-phase sampling schedule for LoRA updates. In the early phase, we prioritize codes with the fewest updates (“least-updated-first”), encouraging broad coverage and ensuring that all observed codes receive at least a few LLM updates. In the later phase, each iteration mixes a subset of the least-updated codes with a subset of the most frequently occurring codes from the current batch, allowing the model to further refine high-impact nodes without neglecting the long tail. This strategy amortizes LLM computation while enabling controlled, coverage-aware fine-tuning of the LLM representations used by the GNN. 2.6 Integrating MedCo into EHR Models As a medical concept encoder, MedCo can be integrated into standard EHR models and trained end-to-end; it improves concept representation learning and consequently downstream predictive performance. Given the learned concept (code) embeddings \mathbf{z}_{1},\mathbf{z}_{2},\dots,\mathbf{z}_{N} , we form the embedding matrix \mathbf{Z}\in\mathbb{R}^{d\times N} , where \mathbf{z}_{i} is the i -th column of \mathbf{Z} . This matrix is then used by a downstream task model. Considering sequential diagnosis prediction, which maps a sequence of visits to the diagnoses of the next visit: f:\{V_{1},V_{2},\ldots,V_{t}\}\rightarrow\mathbf{\hat{y}}_{t+1} , where \mathbf{\hat{y}}_{t+1}\in\mathbb{R}^{N_{dx}} is a multi-hot vector and N_{dx} denotes the total number of diagnosis codes: \displaystyle\mathbf{Z}=[\mathbf{z}_{1},\mathbf{z}_{2},\ldots,\mathbf{z}_{N}] \displaystyle\leftarrow\text{{MedCo}{}}(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{N}) (3) \displaystyle\mathbf{v}_{1},\mathbf{v}_{2},\ldots,\mathbf{v}_{t} \displaystyle=\mathbf{Z}[\mathbf{u}_{1},\mathbf{u}_{2},\ldots,\mathbf{u}_{t}] \displaystyle\mathbf{h}_{p} \displaystyle=\text{Model}(\mathbf{v}_{1},\mathbf{v}_{2},\ldots,\mathbf{v}_{t}) \displaystyle\mathbf{\hat{y}}_{t+1} \displaystyle=\text{Sigmoid}(\mathbf{W}\mathbf{h}_{p}+\mathbf{b}) For each visit V_{t} , we compute \mathbf{v}_{t}=\mathbf{Z}\mathbf{u}_{t} , where \mathbf{u}_{t}\in\{0,1\}^{N} is a multi-hot code indicator and \mathbf{Z} is the learned concept embedding matrix. The sequence \{\mathbf{v}_{1},\ldots,\mathbf{v}_{t}\} is encoded by a backbone model to produce a patient representation \mathbf{h}_{p} . The final prediction is obtained by a linear classification layer followed by a sigmoid activation. We train with multi-label cross-entropy between \hat{\mathbf{y}}_{t+1} and \mathbf{y}_{t+1} at each timestep. 3 Experimental Setting Figure 3: Performance gains from integrating MedCo into four diagnosis prediction backbones (plug-in analysis) on MIMIC-III and MIMIC-IV. Table 1: Performance comparison on MIMIC-III and MIMIC-IV. We report overall AUPRC, F1, and Acc@k, and stratified AUPRC by diagnosis label frequency quartiles (0–25%, 25–50%, 50–75%, 75–100%). The first row corresponds to the base Transformer, and subsequent rows denote plug-in variants (e.g., GRAM = Base + GRAM). Model General Performance Label Category Performance (AUPRC) AUPRC F1 Acc@15 Acc@20 Acc@30 0–25% 25–50% 50–75% 75–100% MIMIC-III Base 41.00 33.16 47.20 50.40 58.80 40.60 47.30 72.80 78.40 GRAM 41.70 34.60 48.60 51.90 59.80 42.20 50.10 74.10 79.30 MMORE 42.60 35.30 49.20 52.60 60.40 42.80 51.20 74.80 80.00 KAME 42.18 35.07 48.97 52.06 60.06 41.84 49.76 74.06 79.14 G-BERT 42.47 35.38 49.26 52.33 60.27 42.06 50.04 74.24 79.28 HAP 42.36 35.17 49.11 52.18 60.18 41.95 49.92 74.13 79.24 ADORE 42.58 35.43 49.33 52.47 60.32 42.18 50.09 74.29 79.31 KAMPNet 43.06 35.96 49.88 53.07 60.86 42.63 50.83 74.77 79.89 GraphCare 43.35 35.46 52.76 56.00 62.75 44.80 58.16 70.97 65.72 LINKO 44.91 38.20 52.30 55.62 62.80 44.80 55.20 76.43 83.41 MedCo 47.21 42.28 54.20 57.40 64.39 47.67 66.29 76.02 86.70 MIMIC-IV Base 45.10 39.10 54.80 58.10 64.70 46.80 53.60 53.90 77.20 GRAM 46.30 40.00 55.60 58.90 65.70 47.70 54.60 54.80 77.90 MMORE 46.90 40.60 56.10 59.40 66.10 48.20 55.10 55.70 78.40 KAME 45.97 39.66 55.27 58.57 65.26 47.16 54.24 54.28 77.63 G-BERT 46.37 40.07 55.68 58.96 65.66 47.58 54.73 54.87 78.04 HAP 46.33 39.88 55.57 58.92 65.58 47.52 54.57 54.83 77.92 ADORE 46.52 40.23 55.84 59.09 65.79 47.74 54.84 55.06 78.13 KAMPNet 47.08 40.96 56.46 59.77 66.47 48.26 55.57 56.16 78.68 GraphCare 46.90 40.12 55.89 59.14 65.71 48.31 60.31 73.70 66.33 LINKO 48.14 42.05 57.78 61.21 67.97 49.61 56.87 57.58 80.27 MedCo 50.00 45.25 59.22 62.54 68.71 52.01 57.70 59.35 81.19 Datasets. We evaluate on two public EHR benchmarks, MIMIC-III ( 19 ) and MIMIC-IV ( 18 ) . We map ICD diagnosis/procedure codes to CCS categories and medications from National Drug Codes (NDC) to the Anatomical Therapeutic Chemical (ATC) classification. Table 2 reports cohort statistics. The task is next-visit diagnosis prediction over imbalanced label spaces (515 codes in MIMIC-III; 562 in MIMIC-IV). We use Llama-3.2-1B ( 11 ; 25 ) as the text encoder and fine-tune it with LoRA using rank r{=}8 and \alpha{=}32 . The heterogeneous GNN encoder has 2 layers with 1 attention head per layer and dropout 0.4 . The source code is publicly available. 2 2 2 https://github.com/mohsen-nyb/MedCo.git Table 2: Data statistics for MIMIC-III and MIMIC-IV. Metric MIMIC-III MIMIC-IV # Patients 7,515 18,829 # Visits (samples) 12,430 25,028 # Labels/sample 12.31 10.56 # Unique conditions (ICD) 515 562 # Conditions/sample 26.85 59.50 # Drugs/sample 70.10 118.16 # Unique drugs 471 510 # Procedures/sample 6.49 5.41 # Unique procedures 280 322 Evaluation Metrics. We report mean results over 5 folds for AUPRC (area under the precision-recall curve; also stratified by label frequency to assess performance across code rarity levels), Acc@k (top- k accuracy normalized by \min(k,|\mathbf{y}_{t+1}|) ), and F1 (harmonic mean of precision and recall). 4 Evaluation Results To assess MedCo , we address the following research questions: RQ1: Does MedCo improve downstream EHR prediction when used as a plug-in medical concept encoder? RQ2: How does MedCo compare with existing medical code encoders? RQ3: How do individual components of MedCo and different KG edge categories contribute to overall performance? RQ4: How well does MedCo mitigate data scarcity and improve performance on rare conditions? Figure 4: Performance evaluation across different training set sizes using the MIMIC-III and MIMIC-IV datasets. An asterisk (*) next to each encoder indicates the integration of MedCo to that model, e.g., Transformer* = Transformer + MedCo . Table 3: Component-wise ablation of MedCo on MIMIC-IV and MIMIC-III. Model PRAUC F1 Acc@20 MIMIC-IV Base (Transformer) 45.10 39.10 58.10 Base + KG 48.55 42.60 60.90 Base + KG + Edge feat. 48.72 42.85 61.10 Base + KG + Edge feat. + LLM (freeze) 49.38 43.71 61.47 Base + KG + Edge feat. + LLM (LoRA) 50.00 45.25 62.54 MIMIC-III Base (Transformer) 41.00 33.16 50.40 Base + KG 45.79 38.90 55.60 Base + KG + Edge feat. 45.91 39.20 55.85 Base + KG + Edge feat. + LLM (freeze) 46.10 40.45 56.44 Base + KG + Edge feat. + LLM (LoRA) 47.21 42.28 57.40 4.1 RQ1: Plug-in Enhancement Evaluation We hypothesize that integrating MedCo , our medical concept encoder, into standard EHR backbones improves downstream performance by enhancing concept representations. To test this hypothesis, we incorporate MedCo into four representative models: (1) AdaCare 24 , an explainable architecture that models multi-scale biomarker variations; (2) Transformer ( 35 ) , based on self-attention; (3) RETAIN ( 8 ) , an RNN with reverse-time, two-level attention; and (4) TCN ( 3 ) , which uses causal convolutions for temporal modeling. We compare each backbone with and without MedCo . As shown in Figure 3 , MedCo consistently improves performance across all models, demonstrating its effectiveness as a plug-in concept encoder. 4.2 RQ2: Baseline Comparison We compare MedCo against widely used medical concept encoders, including the base Transformer 35 (i.e., no concept encoder), hierarchy-driven ontology methods such as GRAM ( 7 ) , MMORE ( 33 ) , KAME ( 23 ) , and HAP ( 41 ) , as well as KG-based approaches that leverage external knowledge resources. Among these, ADORE ( 5 ) relies on SNOMED, while GraphCare 16 leverages UMLS-derived edges together with LLM-derived edges. KAMPNet ( 2 ) and LINKO 28 construct cross-type connections but do not explicitly model rich semantic relation types. All encoders are integrated into the same downstream backbone for a controlled comparison. As shown in Table 1 (General Performance), MedCo achieves the best overall performance, indicating that our evidence-grounded, heterogeneous, text-attributed KG provides richer and more task-relevant semantics than existing medical concept baselines. Furthermore, to evaluate performance under data scarcity, we stratify diagnosis codes into four frequency quartiles (0–25%, 25–50%, 50–75%, 75–100%), where 0–25% contains the rarest codes. This analysis probes whether MedCo can better propagate informative signals to low-frequency concepts. Table 1 (Label Category Performance) shows that MedCo improves AUPRC across all bands, with pronounced gains on rare codes. Figure 5: Ablation study showing the impact of removing each edge type on PRAUC for MIMIC-III. The dashed vertical line indicates the full-model baseline. 4.3 RQ3: Ablation Study Table 3 shows progressive gains as components are added to MedCo . Starting from the base Transformer, adding the induced KG with a relation-aware GNN provides the largest boost, highlighting the value of explicit cross-domain structure. Edge features yield additional improvements by injecting LLM clinical reasoning and EHR-derived association evidence into message passing. Adding LLM-based node descriptions (frozen encoder) consistently boosts performance, indicating that clinical semantics from natural language complements graph signals. Finally, enabling LoRA fine-tuning delivers the best results, showing that adapting the LLM to the cohort and task produces more informative node representations and maximizes downstream accuracy. Figure 5 presents an edge-category ablation of the induced KG. For each bar, we remove all edges belonging to a given relation family (e.g., rx-px removes every medication-procedure edge) while keeping the rest of the graph unchanged, and report the resulting PRAUC. Larger drops indicate edge categories that contribute more to downstream performance. 4.4 Sensitivity to the LLM Backbone We further evaluate whether MedCo is sensitive to the choice of LLM backbone for node-text encoding. While our main experiments use Llama-3.2-1B , the framework is model-agnostic and can be instantiated with other open-source LLMs. Under the same LoRA fine-tuning setup, we additionally test Gemma-2-2B ( 9 ; 10 ) , Qwen2.5-1.5B ( 15 ; 31 ) , Qwen2.5-0.5B ( 15 ; 30 ) , and SmolLM2-1.7B ( 1 ; 14 ) on MIMIC-IV. As shown in Table 4 , performance remains consistently strong across backbones, indicating that MedCo is not tied to a specific LLM family. Larger backbones provide only modest gains, suggesting that downstream performance is driven primarily by the evidence-grounded KG construction and graph propagation components. LLM backbone AUPRC Acc@20 Llama-3.2-1B 50.00 62.54 Gemma-2-2B 50.31 62.89 Qwen2.5-1.5B 50.16 62.67 Qwen2.5-0.5B 49.31 61.44 SmolLM2-1.7B 50.11 62.61 Table 4: Sensitivity of MedCo to the choice of LLM backbone on MIMIC-IV. Variant AUPRC Acc@20 Train(s) Infer(s) Peak train mem (MiB) Peak infer mem (MiB) Base (no KG, no LLM/GNN) 45.10 58.10 22.0 1.30 169 106 GNN only (KG, no LLM) 48.72 61.10 42.7 3.19 1,273 518 LLM frozen ( K{=}0 ) + GNN 49.38 61.47 40.0 3.19 1,273 518 LLM LoRA + GNN ( K{=}5 ) 49.70 62.35 183.4 3.19 14,477 518 LLM LoRA + GNN ( K{=}10 ) 50.00 62.54 275.4 3.19 24,260 518 Table 5: Cost-accuracy trade-off on MIMIC-IV with batch size 128. Experiments were run on a machine with an Intel Xeon Silver 4214R CPU (24 cores), 256 GiB RAM, and one RTX A6000 GPU (48 GB). For compactness, Train(s) denotes training time per epoch, and Infer(s) denotes inference time per epoch for the test set. 4.5 RQ4: Data Insufficiency Analysis To evaluate MedCo under data scarcity, we subsample the training set to simulate limited-data regimes. In the plug-in setting, Figure 4 shows that MedCo delivers substantial gains for downstream EHR models even when training data are reduced, highlighting its robustness to data insufficiency. 5 Cost-Accuracy Trade-off Although jointly training the LLM and GNN yields the best performance, it can be computationally expensive. MedCo mitigates this cost in two ways: (i) LoRA fine-tunes only a small set of adapter parameters while freezing the backbone LLM, and (ii) node representations are cached and only a selected subset of codes is updated per epoch. At inference time, the LLM is removed from the loop, and prediction uses the cached concept representations together with the GNN and downstream EHR encoder, making deployment much lighter than training. Table 5 summarizes the efficiency-accuracy trade-off on MIMIC-IV. The results show a clear frontier: KG-based variants substantially improve performance over the base model with moderate additional cost, while LoRA-based joint training achieves the highest accuracy at greater training cost. Inference remains lightweight and identical across KG-based variants because the LLM is not used at test time. This suggests that practitioners can prefer frozen-LLM or smaller- K settings when efficiency matters most, and larger- K LoRA training when accuracy is the priority. 6 Related Work The growing availability of EHR data has driven rapid progress in clinical prediction, evolving from early sequential architectures ( 6 ) to attention-based methods ( 8 ; 13 ) , transformer models ( 20 ; 27 ) , and graph neural networks ( 22 ; 39 ; 40 ; 29 ) . A line of work strengthens code embeddings by injecting hierarchical ontology structure into models. GRAM ( 7 ) represents each concept by mixing it with its ancestors, while MMORE ( 33 ) extends this idea with multiple parent representations to better handle mismatch between ontologies and observed EHR patterns. KAME ( 23 ) incorporates ontology knowledge beyond embedding learning into the prediction pipeline. To better exploit hierarchy, HAP ( 41 ) propagates information through top-down and bottom-up attention. Beyond pure hierarchies, G-BERT ( 32 ) combines ontology graphs with BERT-style encoders, ADORE ( 5 ) leverages relational ontologies such as SNOMED to integrate heterogeneous code systems, and KAMPNet ( 2 ) adopts graph contrastive learning for EHR representations. More recent work augments structured EHRs with external signals such as clinical text and web knowledge. GCL ( 21 ) and RAM-EHR ( 38 ) incorporate unstructured text and retrieved medical knowledge, while retrieval-augmented frameworks such as KARE ( 17 ) and GraphCare ( 16 ) integrate multi-source knowledge for representation learning. LINKO 28 leverages multiple ontologies with intra- and inter-ontology propagation via multi-level graph attention. 7 Conclusion We presented MedCo , a framework that integrates evidence-grounded KG construction, LLM-based semantic enrichment, and joint LLM-GNN co-learning for medical concept representation. MedCo builds a heterogeneous diagnosis–medication–procedure KG from EHR-derived evidence, enriches it with LLM-inferred relations and textual semantics, and learns unified concept embeddings through a LoRA-tuned LLM encoder and a heterogeneous GNN. Experiments on MIMIC-III and MIMIC-IV show that MedCo consistently improves sequential diagnosis prediction across multiple backbones and outperforms strong baselines, especially on rare labels and in limited-data settings. Additional analyses show MedCo ’s strong clinical validity, robustness across diverse LLM encoders, and high inference efficiency. 8 Limitations While MedCo amortizes LLM computation via a coverage-aware update schedule, joint KG–LLM training can still be resource-intensive, and scaling to larger vocabularies, longer contexts, or larger LLM backbones may increase computational cost. 9 Ethical Considerations We comply with the ACL Ethics Policy throughout this study. All datasets used are publicly available and contain de-identified patient records, providing strong privacy protections. We do not send any patient-level data to external or public LLM services; all LLM interactions are restricted to general, concept-level information (e.g., medical code descriptions and relation prompts). 10