KEDRec-LM: A K nowledge-distilled E xplainable D rug Rec ommendation Large L anguage M odel DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06

Kai Zhang, Rui Zhu, Shutian Ma, Jingwei Xiong, Yejin Kim, Fabricio Murai, Xiaozhong Liu · 2025

Read KEDRec-LM: A K nowledge-distilled E xplainable D rug Rec ommendation Large L anguage M odel DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 by Kai Zhang, Rui Zhu, Shutian Ma, Jingwei Xiong, Yejin Kim, Fabricio Murai, Xiaozhong Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20350.txt.

Introduction

Kai Zhang and co-authors taught a language model to recommend drugs and explain itself in 2025. Drug find is a core language task in biomedicine, yet systems that must justify a recommendation, the trait that makes a suggestion usable in research or care, remain underexplored. The authors first built the teaching substance: a clear drug recommendation data set built from open drug knowledge graphs, medical trial records. And the PubMed literature, tying likely drugs to the evidence for them.

On that base they trained KEDRec-LM, an instruction-tuned large language model that distills knowledge from the built medical corpus. The model does two things at once: it recommends drugs for a stated need and generates the rationale behind each recommendation. So a human can audit the thinking rather than merely the answer. The data set and model are offered as infrastructure for further work on clear biomedical recommendation.

Catalog record

CREATORKai Zhang, Rui Zhu, Shutian Ma, Jingwei Xiong, Yejin Kim, Fabricio Murai, Xiaozhong Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20350

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

KEDRec-LM: A K nowledge-distilled E xplainable D rug Rec ommendation Large L anguage M odel DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Kai Zhang, Rui Zhu, Shutian Ma, Jingwei Xiong, Yejin Kim, Fabricio Murai, Xiaozhong Liu Abstract . Drug discovery is a critical task in biomedical natural language processing (NLP), yet explainable drug discovery remains underexplored. Meanwhile, large language models (LLMs) have shown remarkable abilities in natural language understanding and generation. Leveraging LLMs for explainable drug discovery has the potential to improve downstream tasks and real-world applications. In this study, we utilize open-source drug knowledge graphs, clinical trial data, and PubMed publications to construct a comprehensive dataset for the explainable drug discovery task, named expRxRec . Furthermore, we introduce KEDRec-LM , an instruction-tuned LLM which distills knowledge from rich medical knowledge corpus for drug recommendation and rationale generation. To encourage further research in this area, we will publicly release 1 1 1 A copy is attached with this submission both the dataset and KEDRec-LM. 1. Introduction The complexity of drug discovery lies in understanding the intricate relationships between drugs and diseases, making the identification of potential therapeutic uses a challenging and resource-intensive endeavor. In recent years, the availability of large-scale biomedical knowledge graphs ( Xu et al. 2024 ; Anand et al. 2024 ; Luo et al. 2023 ) , such as the Drug Repurposing Knowledge Graph (DRKG) ( Ioannidis et al. 2020 ) , has enabled significant advances in this field by linking vast amounts of biomedical entities and relationships. These structured databases capture a wealth of information across drug interactions, disease associations, and biological pathways. Yet, fully exploiting this information for drug discovery, and more specifically drug repurposing, requires efficient methods to extract meaningful insights that can guide therapeutic reasoning. With the rapid growth of biomedical literature, particularly in repositories like PubMed, there is an unprecedented opportunity to tap into this body of knowledge to inform drug-disease relationships. However, manually curating, understanding, and drawing conclusions from this literature is impractical due to its sheer volume and the specificity of each study. Traditional knowledge graphs provide a static representation of these connections, but their utility is limited when it comes to reasoning about complex therapeutic mechanisms or the nuanced interplay between drug efficacy and disease pathology. This gap highlights the need for automated approaches that can contextualize and synthesize existing literature to support and enhance the process of drug design and discovery. The challenges in leveraging biomedical literature for drug discovery are multifaceted. Firstly, the process of identifying and validating drug-disease pairs from a vast knowledge graph is inherently complex, given the heterogeneity of relationships and the potential noise in the data ( Peng et al. 2019 ; Wang et al. 2020 ; Zhu et al. 2019 ) . Secondly, even when relevant pairs are identified, understanding their context and implications requires sifting through large volumes of literature, which can vary greatly in quality, relevance, and specificity ( Haddaway et al. 2020 ; UnfoldAI 2023 ) . Standard retrieval-based systems often fall short in providing insightful reasoning over the retrieved content. Moreover, transforming this wealth of information into a format that can be easily used for computational models in drug design remains an unmet challenge. In essence, there is a critical need for a systematic, automated approach to extract, understand, and reason over biomedical literature in a way that is aligned with the complexities of drug-disease associations ( Huang et al. 2024 ; Authors 2022 ; Gendrin-Brokmann et al. 2023 ) . To address these challenges, we propose a novel framework that integrates knowledge graph extraction, literature mining, and language model-based reasoning. Our approach begins by extracting a subset of drug-disease pairs from the DRKG, thereby forming a focused sub-knowledge graph tailored for drug discovery. For each pair in this graph, we employ a Retrieval-Augmented Generation (RAG) ( Lewis et al. 2020 ) technique which searches for relevant literature content in PubMed and Clinical Trials, leveraging the wealth of biomedical literature to contextualize each drug-disease relationship. To facilitate reasoning over these pairs, we design a set of domain-specific questions that are crucial for understanding the therapeutic potential of each relationship. An instructional template is then used to fit the pairs as well as retrieved background information for a teacher model generating responses to these questions. These responses provide rich, contextually informed insights into each pair, transforming unstructured text into structured, reasoned answers. Building on this, we use these generated responses to train a LLaMA model ( Touvron et al. 2023 ) as the local model for learning those distilled knowledge. This model is designed to understand and reason over drug-disease relationships, providing a tool that not only synthesizes biomedical knowledge but also supports hypothesis generation and decision-making in drug design. By leveraging both structured knowledge graphs and unstructured literature, our approach enables a deeper understanding of drug-disease associations, facilitating the discovery of novel therapeutic opportunities. This paper makes the following contributions: • RAG-Based Literature Reasoning We design domain-specific questions and utilize a GPT-based Retrieval-Augmented Generation (RAG) pipeline to extract and reason over biomedical literature from PubMed for each drug-disease pair. • Domain-Specific local LLM We use the structured responses generated by RAG and a powerful teacher model to train a specialized LLaMA model, tailored for drug design reasoning. This model captures nuanced drug-disease relationships and supports inference-making in therapeutic research. • Experimental Resources We demonstrate the effectiveness of our approach in explainable drug recommendation, open-sourcing a new dataset and a local model to the community. Our work represents a step forward in the integration of knowledge graphs, literature mining, and language model-based reasoning for explainable drug recommendation. 2. Background Large Language Models and Retrieval-Augmented Generation. Large Language Models (LLMs) are deep neural networks trained on massive corpora of text data to generate and understand human-like language. By learning statistical patterns in text, LLMs, such as GPT and BioGPT, are capable of tasks like text generation, summarization, and reasoning. Given an input sequence q , an LLM generates a probability distribution over possible outputs y , modeled as: p(y|q)=\prod_{t=1}^{T}p(y_{t}|q,y_{<t}), where T is the output sequence length, and y_{<t} represents the tokens generated up to time t . However, standard LLMs are limited by the knowledge encoded during training and may struggle with domain-specific tasks such as biomedical reasoning. Retrieval-Augmented Generation (RAG) addresses this limitation by combining retrieval-based and generation-based methods, enabling LLMs to incorporate external knowledge in real-time. The workflow of RAG can be broken into two stages: (1) Retrieval : For a given query q , relevant documents are retrieved from an external corpus \mathcal{D} . Each document d_{i}\in\mathcal{D} is scored based on the similarity between the query embedding \mathbf{e}(q) and document embedding \mathbf{e}(d_{i}) . Cosine similarity is used for scoring: \text{sim}(q,d_{i})=\frac{\mathbf{e}(q)\cdot\mathbf{e}(d_{i})}{\|\mathbf{e}(q)\|\|\mathbf{e}(d_{i})\|}. The top- k documents with the highest scores are selected as the retrieval set \mathcal{R} . (2) Generation : The query q and the retrieved context \mathcal{R} are concatenated and fed into the LLM. The LLM generates an output y conditioned on both the query and the retrieved documents: p(y|q,\mathcal{R})=\prod_{t=1}^{T}p(y_{t}|q,\mathcal{R},y_{<t}). This approach improves the factual accuracy and interpretability of LLM outputs

Read the complete text