Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset

Ming-Hsiu Wu, Ziqian Xie, Shuiwang Ji, Degui Zhi · 2025

Read Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset by Ming-Hsiu Wu, Ziqian Xie, Shuiwang Ji, Degui Zhi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00708.txt.

Introduction

Ming-Hsiu Wu, Ziqian Xie, Shuiwang Ji, and Degui Zhi built a new dataset for drug discovery in a 2025 paper. Models that predict how well a drug candidate binds to a protein are key in early drug hunting. Yet current models often overfit to simplified datasets that miss how real proteins change.

Real proteins carry modifications, such as swapped, added, or missing building blocks, and added phosphate groups. The team rebuilt the widely used DAVIS dataset in a complete, modification-aware form. It adds thousands of kinase-ligand pairs that involve such changes. A harder, more honest test set should show which models truly work.

Catalog record

CREATORMing-Hsiu Wu, Ziqian Xie, Shuiwang Ji, Degui Zhi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00708

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset Ming-Hsiu Wu, Ziqian Xie, Shuiwang Ji, Degui Zhi Abstract Advancements in AI for science unlocks capabilities for critical drug discovery tasks such as protein-ligand binding affinity prediction. However, current models overfit to existing oversimplified datasets that does not represent naturally occurring and biologically relevant proteins with modifications. In this work, we curate a complete and modification-aware version of the widely used DAVIS dataset by incorporating 4,032 kinase–ligand pairs involving substitutions, insertions, deletions, and phosphorylation events. This enriched dataset enables benchmarking of predictive models under biologically realistic conditions. Based on this new dataset, we propose three benchmark settings—Augmented Dataset Prediction, Wild-Type to Modification Generalization, and Few-Shot Modification Generalization—designed to assess model robustness in the presence of protein modifications. Through extensive evaluation of both docking-free and docking-based methods, we find that docking-based model generalize better in zero-shot settings. In contrast, docking-free models tend to overfit to wild-type proteins and struggle with unseen modifications but show notable improvement when fine-tuned on a small set of modified examples. We anticipate that the curated dataset and benchmarks offer a valuable foundation for developing models that better generalize to protein modifications, ultimately advancing precision medicine in drug discovery. The benchmark is available at: https://github.com/ZhiGroup/DAVIS-complete 1 Introduction Measuring protein-ligand binding affinity is a critical task in drug development, as it directly determines the therapeutic efficacy and selectivity of potential drug candidates [ 37 ] . AI breakthrough has revolutionized protein folding [ 5 , 13 , 16 , 22 ] , protein design [ 19 , 18 ] , and even protein-ligand binding [ 5 , 14 , 46 , 8 ] . However, even with breakthroughs like AlphaFold [ 5 , 13 , 16 ] and DiffDock [ 14 , 8 ] , the protein-ligand affinity prediction problem is not solved yet. First, structural predictions from models like AlphaFold are AI estimations, not always equivalent to experimentally solved crystal structures [ 33 , 17 ] . Second, even with solved protein structures, co-crystalized ligand-bound structures are often unavailable. Third, factors beyond direct structural complementarity, such as pH [ 30 ] and solvent effects [ 3 ] , also significantly influence binding affinity. Current AI-driven affinity prediction methods are still in what might be termed a ’pre-AlphaFold era’; a significant portion operate as ’structure-free’ (using 1D protein amino acid sequences) [ 7 , 8 , 11 , 9 ] or ’docking-free’ even with the incorporation of structural information [ 25 , 10 , 44 ] . A more pressing challenge in the field is the lack of large, diverse, and experimentally homogeneous training datasets [ 20 ] that adequately capture biological realities. In particular, protein modifications—such as substitutions, insertions, deletions, and post-translational modifications (PTMs)—can drastically alter protein structure and ligand interactions [ 38 , 26 , 48 ] . While generating such comprehensive datasets is challenging, current AI-driven models [ 21 , 2 , 34 , 14 , 52 , 13 , 53 , 44 , 28 ] only focus on wild-type proteins, overlooking modified protein versions or applying a simplistic "one-size-fits-all" approach to variants within datasets like DAVIS [ 3 ] . This oversight creates a significant gap in understanding how these models perform in real-world biological contexts, where proteins naturally undergo structural or chemical modifications. Models trained solely on such data may overfit to wild-type proteins and fail to generalize to more complex, yet practical, scenarios. This study aims to bridge this critical gap. We introduce DAVIS-complete, a curated and complete version of the DAVIS dataset [ 3 ] that explicitly accounts for protein modifications, as illustrated in Fig. 1 (a). Building upon this, we design three novel benchmark frameworks inspired by realistic drug discovery scenarios: (1) Augmented Dataset Prediction: We augment the previously used DAVIS dataset with modified protein–ligand pairs and evaluate model performance across three standard train-test splits, assessing general predictive capability in a diverse setting (Fig. 1 (c)). (2) Wild-Type to Modification Generalization: This benchmark assesses a model’s ability to generalize from wild-type proteins to unseen modified variants in a zero-shot setting, reflecting practical cases where experimental data for modified proteins are unavailable (Fig. 1 (d)). (3) Few-Shot Modification Generalization: We further evaluate model adaptability by fine-tuning on a small number of modified protein–ligand pairs (Fig. 1 (e)). This scenario mirrors precision medicine applications, where individualized therapies frequently rely on accurately predicting drug responses from limited genetic or proteomic data unique to each patient. Together, these benchmarks, for the first time, provide a more comprehensive and biologically relevant framework for evaluating binding affinity prediction models. Our contributions are: • The curation and public release of DAVIS-complete, a comprehensive dataset incorporating protein modifications for binding affinity prediction benchmark. • The design and proposal of three biologically relevant benchmarks built upon DAVIS-complete. • An extensive evaluation of existing state-of-the-art methods using these new benchmarks, highlighting current limitations (e.g., overfitting to wild-type proteins) and demonstrating the potential improvement (e.g., through fine-tuning strategies). 2 Related Works 2.1 Docking free-based models In scenarios where high-resolution co-crystallized three-dimensional protein structures are unavailable, most existing deep learning approaches for predicting protein-ligand binding affinity operate without considering explicit binding poses—commonly referred to as docking-free methods. These models often represent proteins using amino acid sequences or predicted protein contact maps, while ligands are depicted as SMILES strings or molecular graphs. Deep neural networks are then employed to extract latent features from these representations to predict binding affinities. Notable models in this category include DeepDTA [ 7 ] , AttentionDTA [ 8 ] , GraphDTA [ 9 ] , DGraphDTA [ 10 ] , and MGraphDTA [ 11 ] . These methods have significantly advanced the field by circumventing the high cost of experimentally determining protein–ligand binding conformations. However, due to the nature of their input representations, these models are inherently limited in their ability to capture structural alterations caused by protein mutations or PTMs. This limitation is particularly important, as such modifications frequently occur in biological systems and could substantially influence protein-ligand binding affinity. 2.2 Docking-based models When high-resolution co-crystallized three-dimensional structures are available, docking-based approaches have also made strides in modeling protein-ligand interactions by explicitly considering atom-level interaction details. Unlike docking-free models, these methods incorporate spatial information about the binding pose, allowing for a more accurate depiction of the interaction landscape. Notable examples include SchNet [ 36 ] , EGNN [ 35 ] , and GIGN [ 15 ] , which leverage 3D convolutional networks or equivariant graph neural networks to process molecular structures and predict binding affinity directly from geometric configurations. However, the applicability of docking-based methods is limited by the availability of high-quality co-crystallized 3D structures. Building on this direction, the Folding-Docking-Affinity (FDA) [ 12 ] provides a framework for binding affinity prediction in scenarios where experimentally determined co-crystallized structures are unavailable. It unifies protein structure prediction, molecular docking, and binding affinity estimation into a single pipeline. FDA employs predicted 3D protein structures (e.g., from AlphaFold [ 13 , 5 ] ) and ligand binding poses generated through docking methods (e.g., DiffDock [ 14 ] ) to construct realistic protein-ligand complexes at scale. Despite potential noise, FDA explicitly models atom-level interactions within predicted complexes to capture

Read the complete text