AssayMatch: Learning to Select Data for Molecular Activity Models

Vincent Fan, Regina Barzilay · 2025

Read AssayMatch: Learning to Select Data for Molecular Activity Models by Vincent Fan, Regina Barzilay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16087.txt.

Introduction

Vincent Fan and Regina Barzilay introduce AssayMatch, a data selection method for molecular activity models in drug discovery. Model quality rests on training data. Yet public bioactivity sets merge assays run under varied protocols, which adds noise. Rather than train on all data, AssayMatch builds smaller, more uniform training sets tuned to the test set at hand, whose labels are unknown in real campaigns.

Data attribution scores how much each training assay helps performance. Those scores fine-tune language embeddings of text assay descriptions, so they capture assay fit, not just word likeness. At test time, the tuned embeddings rank all training data for selection. Tests show the gain from picking data by fit rather than by volume.

Catalog record

CREATORVincent Fan, Regina Barzilay
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-16087

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AssayMatch: Learning to Select Data for Molecular Activity Models Vincent Fan, Regina Barzilay Abstract The performance of machine learning models in drug discovery is highly dependent on the quality and consistency of the underlying training data. Due to limitations in dataset sizes, many models are trained by aggregating bioactivity data from diverse sources, including public databases such as ChEMBL. However, this approach often introduces significant noise due to variability in experimental protocols. We introduce AssayMatch , a framework for data selection that builds smaller, more homogenous training sets attuned to the test set of interest. AssayMatch leverages data attribution methods to quantify the contribution of each training assay to model performance. These attribution scores are used to finetune language embeddings of text-based assay descriptions to capture not just semantic similarity, but also the compatibility between assays. Unlike existing data attribution methods, our approach enables data selection for a test set with unknown labels, mirroring real-world drug discovery campaigns where the activities of candidate molecules are not known in advance. At test time, embeddings finetuned with AssayMatch are used to rank all available training data. We demonstrate that models trained on data selected by AssayMatch are able to surpass the performance of the model trained on the complete dataset, highlighting its ability to effectively filter out harmful or noisy experiments. We perform experiments on two common machine learning architectures and see increased prediction capability over a strong language-only baseline for 9/12 model-target pairs. AssayMatch provides a data-driven mechanism to curate higher-quality datasets, reducing noise from incompatible experiments and improving the predictive power and data efficiency of models for drug discovery. AssayMatch is available at https://github.com/Ozymandias314/AssayMatch . keywords American Chemical Society, L a T e X † † affiliation: Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139 † † email: [email protected] † † email: [email protected] † † affiliation: Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139 † † affiliation: Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, 02139 † † abbreviations: IR,NMR,UV 1 Introduction Property prediction models are widely used for virtual screening in modern drug discovery. However, model performance can vary substantially depending on the target or endpoint being modeled. A major factor influencing prediction accuracy is the quality of the training data, which consists of molecules paired with a property value. When large amounts of diverse and consistently measured experimental data are available, the models learn generalizable and robust features. For many properties, such comprehensive data is not readily accessible and researchers commonly combine smaller datasets instead. These measurements are recorded by laboratories or manually curated from literature and deposited in publicly available resources such as ChEMBL 6 . While aggregated datasets readily increase the size of training data, they present significant challenges due to well-documented inconsistencies between assays. Figure 1: CYP3A4 IC50 measurements for the same molecule taken from ChEMBL assays 2296579, 2296580, 2296590. The measurements differ vastly depending on specific details in the assay description, such as the probe substrate or timing conditions. Combining data from heterogeneous experimental settings introduces confounding factors that obscure patterns and hinder generalization. Landrum et al. 11 analyzed IC50 affinity data in ChEMBL and demonstrated that over 25% of reported measurements for the same target and small molecule differ by more than one order of magnitude. Figure 1 depicts three inconsistent IC50 measurements of the same molecule with respect to Cytochrome P450 3A4 along with their assay descriptions in ChEMBL. There are over 1000 unique assay descriptions recorded for this target, reflecting substantial variability in experimental protocols, including differences in timing, substrate selection, expression systems, and detection methods. Past research has shown that incorporating extra text-based assay metadata can decrease noise or improve model performance. Niu et al. extract keywords from assay descriptions in ChEMBL for manual review by experts 15 . They generated a filtered set of data exhibiting reduced inter-assay variation. Other approaches automatically combine and filter assays for training through heuristics based on measurements and metadata. However, these rule-based consistency checks are often challenging to design due to sparse overlap of identical compounds between assays. We find that fewer than 0.5% of ChEMBL assay pairs share even a single molecule. In the “maximal curation” scheme devised by Landrum et al., over 99% of all IC50 data is discarded to pass hand crafted rules. Schoenmaker et al. offer a deep learning approach by directly including language embeddings of assay context as input to their bioactivity model, showing improvement for specific targets 19 . In contrast, CLAMP, TwinBooster, and MoleculeSTM use machine-learning models that leverage unsupervised relationships within text descriptions to enhance molecular representations 21 ; 20 ; 13 . However, language models are not explicitly trained to reflect the differences between chemical assays. The descriptions in Figure 1 are worded almost identically and have very similar language model embeddings, but clearly produce disparate readouts. Another orthogonal assay-aware approach is to treat each batch as a completely separate entity, and only learn to model and predict activity differences of compounds within the same assay 17 ; 5 . In this work we present AssayMatch , a model agnostic framework for data selection designed to learn the functional relationships between different text-based assay conditions using data attribution. AssayMatch takes an unseen assay description as input and generates a ranking of all training assays based on their predicted contribution to model performance on the test assay. This contrasts with traditional data attribution methods, which are unsuitable for data selection, since they require the true label of every point in advance. However, signals from data attribution methods are able to explain the effects of different assays on model performance. AssayMatch utilizes these signals to finetune the language embeddings of a large curated set of ChEMBL metadata without inspecting the held-out test set to improve upon simple semantic similarity. AssayMatch enables the construction of smaller, more efficient, and higher-quality training sets for improved model performance and generalizability. Using AssayMatch to select data for two common machine learning architectures, we were able to surpass the performance models trained on the full dataset by removing the 10% of samples identified as least informative among other gains in data efficiency. We make our implementation available at https://github.com/Ozymandias314/AssayMatch . 2 Related Work Data Attribution Methods Our work relates to research in data attribution. These methods seek to quantify the contribution of each point in the training set to the model’s performance. They can identify points that introduce noise to the training process. Specifically, they build upon classical influence functions, which measure the leave-one-out effect of removing each individual training point from the dataset. In deep learning, however, training a separate model with each individual point held out from the dataset is computationally prohibitive. Therefore, most of the existing methods focus on estimating quantities in the mathematical formula for leave-one-out scores. Koh et al. first extended influence functions to simple deep learning models by making first and second order approximations on the loss function to estimate the leave-one-out effect 10 . Other approaches repeatedly sample different subsets of the training set to empirically estimate the leave-one-out effect. Sampling based approaches achieve better attribution performance for more

Read the complete text