Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models.

Jiang D, Wu Z, Hsieh CY, Chen G, Liao B, Wang Z, Shen C, Cao D, Wu J, Hou T. · 2021

Read Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. by Jiang D, Wu Z, Hsieh CY, Chen G, Liao B, Wang Z, Shen C, Cao D, Wu J, Hou T. (2021) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc7888189.txt.

Introduction

Jiang and co-authors report a comparison study on how computers represent molecules for drug discovery. A model must first turn a molecule into numbers it can learn from. Older methods use fixed descriptors. Newer graph neural networks learn the form straight from the molecule's structure. The team tested which approach works better.

They ran eight machine learning algorithms on 11 public datasets. Four models used descriptors, and four used molecular graphs. On average, the descriptor-based models won on both accuracy and speed. Support vector machines did best on the regression tasks. Random forests and XGBoost were strong and fast on classification tasks. Some graph models still shone on larger datasets. The authors also tested the models in virtual screening against HIV.

Catalog record

CREATORJiang D, Wu Z, Hsieh CY, Chen G, Liao B, Wang Z, Shen C, Cao D, Wu J, Hou T.
YEAR2021
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC7888189

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models By Dejun Jiang, Zhenxing Wu, Chang-Yu Hsieh, Guangyong Chen, Ben Liao, Zhe Wang, Chao Shen, Dongsheng Cao, Jian Wu, Tingjun Hou Published in: Journal of Cheminformatics DOI: 10.1186/s13321-020-00479-8 ABSTRACT Abstract Graph neural networks (GNN) has been considered as an attractive modelling method for molecular property prediction, and numerous studies have shown that GNN could yield more promising results than traditional descriptor-based methods. In this study, based on 11 public datasets covering various property endpoints, the predictive capacity and computational efficiency of the prediction models developed by eight machine learning (ML) algorithms, including four descriptor-based models (SVM, XGBoost, RF and DNN) and four graph-based models (GCN, GAT, MPNN and Attentive FP), were extensively tested and compared. The results demonstrate that on average the descriptor-based models outperform the graph-based models in terms of prediction accuracy and computational efficiency. SVM generally achieves the best predictions for the regression tasks. Both RF and XGBoost can achieve reliable predictions for the classification tasks, and some of the graph-based models, such as Attentive FP and GCN, can yield outstanding performance for a fraction of larger or multi-task datasets. In terms of computational cost, XGBoost and RF are the two most efficient algorithms and only need a few seconds to train a model even for a large dataset. The model interpretations by the SHAP method can effectively explore the established domain knowledge for the descriptor-based models. Finally, we explored use of these models for virtual screening (VS) towards HIV and demonstrated that different ML algorithms offer diverse VS profiles. All in all, we believe that the off-the-shelf descriptor-based models still can be directly employed to accurately predict various chemical endpoints with excellent computability and interpretability. Keywords: Graph neural networks, Extreme gradient boosting, Ensemble learning, Deep learning, ADME/T prediction INTRODUCTION Molecular property modelling, which assists in hunting for chemicals with desired pharmacological and ADME/T (absorption, distribution, metabolism, excretion, and toxicity) properties, is one of the most classical cheminformatics tasks [1, 2]. A variety of machine learning (ML) approaches, such as Naive Bayes (NB) [3–5], k-Nearest Neighbors (k-NN) [6], logistic regression (LR) [7, 8], support vector machine (SVM) [9–13], random forest (RF), [10, 14, 15] artificial neural network (ANN) [13] and more, have been widely employed in property prediction. More recently, the emergence of deep learning (DL) methods has revolutionized this traditional cheminformatics task due to their extraordinary capacity to learn intricate relationships between structures and properties [16–23]. The models developed by DL can be roughly classified into two categories: descriptor-based models and graph-based models [24]. As to descriptor-based DL models, molecular descriptors and/or fingerprints commonly used in traditional quantitative structure–activity relationship (QSAR) models are used as the input, and then a specific DL architecture is employed to train a model [25]. As to graph-based DL models, the basic chemical information encoded by molecular graphs is used as the input, and then a graph-based DL algorithm, such as graph neural networks (GNN), is used to train a model. Similar to the convolutions on the regular data such as images and texts, GNN generalizes this operation to the irregular molecular graph that is a natural representation for chemical structures. More specifically, a graph G = (V, E) can be defined as the connectivity relations between a set of nodes (V) and a set of edges (E). Naturally, a molecule can also be considered as a graph consisting of a set of atoms (nodes) and a set of bonds (edges). Essentially, GNN aims to learn the representations of each atom by aggregating the information from its neighboring atoms encoded by the atom feature vector and the information of the connected bonds encoded by the bond feature vector through message passing across the molecular graph recursively (Fig. 1), followed by the state updating of the central atoms and read-out operation. Then, the learned atom representations can be used for the prediction of molecular properties through the read-out phase [19, 26]. The key feature of GNN is its capacity to automatically learn task-specific representations using graph convolutions while does not need traditional hand-crafted descriptors and/or fingerprints. The state-of-the-art accuracy of GNN models in property prediction has been well represented [17, 24, 27–32]. The representative GNN models and their statistical performances on the MoleculeNet benchmark datasets [32] are summarized in Table 1. As we can see, their performances on the benchmark datasets vary from one to another, which may be attributed to the discrepancies on the model architectures, evaluation methods, training strategies and so on. Recently, a GNN method: Attentive FP, has gained increasing attention from the scientific community [27]. As shown in Table 1, Attentive FP yields the best predictions to 6 out of 11 benchmark datasets, including 2 regression tasks (ESOL and FreeSolv) and 4 classification tasks (MUV, BBBP, ToxCast and ClinTox), highlighting its impressive performance in modelling a variety of chemical properties in comparison with several other graph-based methods. A majority of those studies claimed that graph-based models are typically superior or comparable to traditional descriptor-based models [24, 30–35], and only a few studies gave the opposite conclusions [36]. For example, in 2017, Wu et al. reported MoleculeNet, a large benchmark for molecular machine learning, and the evaluation results illustrated that graph-based methods outperformed descriptor-based methods on most datasets [32]. Similarly, in 2019, Yang et al. introduced a novel GNN framework named directed message passing neural networks (D-MPNN), and the extensive evaluation on a large dataset collection indicated that D-MPNN consistently matched or outperformed descriptor-based methods on most datasets [24]. More recently, Korolev et al. reported a universal graph convolutional networks (GCN) architecture for the predictions of various chemical endpoints [33], and the application of GCN illustrated that its performance was comparable to state-of-the-art ML algorithms such as SVM, RF, and gradient boosting decision trees (GBDT). [Figure: The general workflow of GNN in molecular property prediction] [Table: The reported GNN models in molecular property prediction] In most of these reported studies, traditional ML models such as LR, RF, SVM (especially ‘gold standard’ RF) [31, 37] were employed to develop the prediction models based on a set of individual fingerprints (especially Extended Connectivity Fingerprints, ECFP) [31–33]. However, it is well known that the performance of descriptor-based models is highly depending on the descriptors used in training and many previous studies have highlighted that ML models only based on molecular fingerprints are not such well-performing [4, 5, 38, 39]. In addition, little attention was paid to several newly state-of-the-art ML algorithms, such as XGBoost and LightGBM, which have illustrated great potentials for modelling various molecular properties [39–42]. Accordingly, the conclusion that graph-based methods outperform traditional descriptor-based methods still remains controversial. The present study attempts to give a comprehensive evaluation of descriptor-based and graph-based models on 11 public datasets with different property endpoints. Four ML algorithms were used to develop the descriptor-based models, including SVM, extreme gradient boosting (XGBoost), RF and deep neural networks (DNN). In order to better represent the chemical and structure features of the molecules for the descriptor-based models, the combination of one set of molecular descriptors (206 MOE 1-D and 2-D descriptors) and two sets of fingerprints (881 PubChem fingerprints and

Read the complete text