A Dataset for Distilling Knowledge Priors from Literature for Therapeutic Design

Haydn Thomas Jones, Natalie Maus, Josh Magnus Ludan Maggie Ziyu Huan, Jiaming Liang, Marcelo Der Torossian Torres, Jiatao Liang, Zachary Ives, Yoseph Barash, Cesar de la Fuente-Nunez, Jacob R. Gardner, Mark Yatskar · 2025

Read A Dataset for Distilling Knowledge Priors from Literature for Therapeutic Design by Haydn Thomas Jones, Natalie Maus, Josh Magnus Ludan Maggie Ziyu Huan, Jiaming Liang, Marcelo Der Torossian Torres, Jiatao Liang, Zachary Ives, Yoseph Barash, Cesar de la Fuente-Nunez, Jacob R. Gardner, Mark Yatskar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10899.txt.

Introduction

Drug designers who let models roam free often propose molecules that no sane chemist would make, some of them likely mutagenic. Haydn Thomas Jones and co-authors mine the literature in 2025 for the missing common sense, building Medex, a set of thirty-two million short facts about compounds used in real labs. Language model pipelines extracted and condensed the facts.

Models pretrained on Medex beat a far larger drug model on standard therapeutic tasks, despite a fraction of the size. Used as guardrails during molecule generation, the priors cut dangerous proposals while keeping potency. The authors release the data set openly and promise updates as literature grows. They present literature itself as the cheapest safety data there is.

Catalog record

CREATORHaydn Thomas Jones, Natalie Maus, Josh Magnus Ludan Maggie Ziyu Huan, Jiaming Liang, Marcelo Der Torossian Torres, Jiatao Liang, Zachary Ives, Yoseph Barash, Cesar de la Fuente-Nunez, Jacob R. Gardner, Mark Yatskar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10899

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Dataset for Distilling Knowledge Priors from Literature for Therapeutic Design Haydn Thomas Jones, Natalie Maus, Josh Magnus Ludan Maggie Ziyu Huan, Jiaming Liang, Marcelo Der Torossian Torres, Jiatao Liang, Zachary Ives, Yoseph Barash, Cesar de la Fuente-Nunez, Jacob R. Gardner, Mark Yatskar Abstract AI-driven discovery can greatly reduce design time and enhance new therapeutics’ effectiveness. Models using simulators explore broad design spaces but risk violating implicit constraints due to a lack of experimental priors. For example, in a new analysis we performed on a diverse set of models on the GuacaMol benchmark using supervised classifiers, over 60% of molecules proposed had high probability of being mutagenic. In this work, we introduce Medex , a dataset of priors for design problems extracted from literature describing compounds used in lab settings. It is constructed with LLM pipelines for discovering therapeutic entities in relevant paragraphs and summarizing information in concise fair-use facts. Medex consists of 32.3 million pairs of natural language facts, and appropriate entity representations (i.e. SMILES or refseq IDs). To demonstrate the potential of the data, we train LLM, CLIP, and LLava architectures to reason jointly about text and design targets and evaluate on tasks from the Therapeutic Data Commons (TDC). Medex is highly effective for creating models with strong priors: in supervised prediction problems that use our data as pretraining, our best models with 15M learnable parameters outperform larger 2B TxGemma on both regression and classification TDC tasks, and perform comparably to 9B models on average. Models built with Medex can be used as constraints while optimizing for novel molecules in GuacaMol, resulting in proposals that are safer and nearly as effective. We release our dataset at huggingface.co/datasets/medexanon/Medex , and will provide expanded versions as available literature grows. † † † Corresponding authors : {haydnj,jacobrg,myatskar}@seas.upenn.edu, ∗ equal contribution. 1 Introduction AI-driven scientific discovery within chemistry and biochemistry for therapeutic design has become one of the most exciting areas of growth for the field, with promising successes in protein folding ( abramson_accurate_2024 ; lin_evolutionary-scale_2023 ) , antibody and de novo protein design ( watson_novo_2023 ; thrift_graph-pmhc_2024 ) , antibiotic discovery ( stokes_deep_2020 ) , and many others. The success of these computational, data-driven approaches is fueled by the wealth and variety of publicly accessible large-scale data. Curated repositories like RCSB PDB ( burley_updated_2025 ) , ClinVar ( landrum_clinvar_2014 ) , PubChem ( 10.1093/nar/gkae1059 ) , UniProt ( 10.1093/nar/gkae1010 ) , OAS ( olsen_observed_2022 ) , the Therapeutic Data Commons (TDC) ( DBLP:conf/nips/HuangFG0RLCXSZ21 ) , among others, have enabled easy access to data on the structure, function, and biological activity for proteins, small molecules, genetic variants, and other biological entities of interest. Although existing datasets contain a wealth of information, they are incomplete. The majority of our knowledge of chemistry, biology, and medicine remains “locked” in natural-language text found in publications, patents, and other articles. For example, while TDC distributes small- to moderate-scale labeled datasets for specific kinds of drug safety information, the ultimate source of ground truth is found in publications, data sheets, and other human readable resources. Because of this relative inaccessibility of knowledge about key drug design factors like safety, stability, pharmacodynamics, and developability, many drug design benchmarks and algorithms are developed using in silico simulation that outright ignores these factors ( maus_local_2022 ; lee_genmol_2025 ; lee2025latent ; chu_inversion-based_2024 ) . To make this concrete: in Section 2 , we demonstrate that a wide variety of recent work optimizing the GuacaMol benchmark suite of drug design tasks would have a large fraction of their highest scoring molecules filtered out by classifiers predicting safety characteristics considered by the TDC. To address the lack of resources for prior knowledge relevant for therapeutic design, we present Medex . Medex is a large-scale dataset of medically relevant entities–small molecules, proteins, diseases, genes, and so on–and facts about these entities distilled from publicly accessible or licensable literature and other text sources. Our freely available dataset comprises over two million unique entities paired with information from over 200 million unique passages. For release, our dataset can be accessed as a large-scale set of succinct facts , along with normalized IDs and DOI sources, about the entities distilled from the literature. Medex was created by leveraging recent advances in large language models (LLMs) and multimodal language modeling Llava ; li2023llavamed ; CLIP . We have created and validated a mixture of supervised and zero-shot LLM components for discovering therapeutic entities in relevant paragraphs and summarizing information in concise facts. Ultimately, our pipeline offers a solution for transforming unstructured data of academic literature into tagged pairs of therapeutically relevant entities (small molecules, proteins, genes, and so on) linked with text found describing those entities. Medex can then be used in a variety of downstream multimodal models–for example using contrastive learning techniques–to build representations of entities and associated facts. Our key contributions are as follows: 1. We release Medex , a dataset of medical entities , associated text and 32.3 M extracted facts . This data represents a significant step towards enabling machine learning models to leverage the rich biological, chemical, and medical knowledge contained in scientific literature. 2. We demonstrate the potential for our data to greatly improve supervised and multimodal learning. Leveraging our data, we train small multimodal models with 15M learnable parameters that outperform the larger 2B TxGemma model across TDC classification benchmark tasks, and achieve 33% lower MAE on regression benchmark tasks. Our models perform comparably to the larger 9B parameter models on average across the TDC tasks. To further highlight the value of our knowledge extraction alone without access to additional TDC labels, we demonstrate 74% improved zero-shot performance over baselines. 3. We demonstrate that models built with our data can be used to constrain molecular optimization algorithms . We optimize 4 Guacamol benchmark tasks using safety and toxicity constraints, and demonstrate proposals that are safer and nearly as high scoring as unconstrained solutions. 2 GuacaMol analysis Figure 1 : Frequency at which candidates are deemed unsuitable by our classifiers in the Guacamol drug design benchmark ( Brown2019-zc ) . If we require drug safety confidence of only 60\% , more than 80\% of drugs across all methods are removed. Methods are retrieved from Gao2022SampleEM . Many existing approaches for therapeutic candidate optimization are benchmarked primarily on in silico simulation: given a set of design goals expressed as a fitness function (i.e. binding affinity), the goal is to propose de novo high scoring molecules, often with the fewest number of tests against the fitness function ( Gao2022SampleEM ) . Many approaches have been proposed lee_genmol_2025 ; lee2025latent ; maus_local_2022 ; chu_inversion-based_2024 , and methods are increasingly able to produce very high scoring, high precision candidate lists. For example, many of the Guacamol ( Brown2019-zc ) molecular design benchmark tasks can now be optimized in hundreds of evaluations ( lee2025latent ) . Results on Guacamol would seem to imply that, soon, given an entirely novel design problem, such methods could be used to propose a small number of candidates for scientists to test iteratively in a lab. While results are promising and many recent papers showcase the potential of computational approaches with in vitro and in vivo data, common in silico benchmarks may

Read the complete text