Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations.
Winter R, Montanari F, Noé F, Clevert DA. · 2019
Read Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations. by Winter R, Montanari F, Noé F, Clevert DA. (2019) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc6368215.txt.
Introduction
Winter and co-authors present a research article in Chemical Science on learning molecular descriptors with neural networks. A molecule can be written in several text notations, such as SMILES and InChI. The team trained a model to translate between equivalent notations, forcing it to compress chemical meaning into a fixed-size numeric vector.
The model was pretrained on about 72 million compounds, with no labels needed. The resulting descriptors matched or beat standard circular fingerprints on prediction tasks for activity and physical properties. On two virtual screening benchmarks they significantly outperformed rival descriptors. The continuous space also lets users decode small moves into new chemical structures.
Catalog record
| CREATOR | Winter R, Montanari F, Noé F, Clevert DA. |
|---|---|
| YEAR | 2019 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations† By Robin Winter, Floriane Montanari, Frank Noé, Djork-Arné Clevert Published in: Chemical Science DOI: 10.1039/c8sc04175j ABSTRACT Translation between semantically equivalent but syntactically different line notations of molecular structures compresses meaningful information into a continuous molecular descriptor. INTRODUCTION Molecular descriptors play a crucial role in chemoinformatics, since they allow representing chemical information of actual molecules in a computer-interpretable vector of numbers.1 While chemical information can be represented by experimental measurements such as physico-chemical property measurements,2 a lot of work has been done to derive molecular descriptors from a symbolic representation of a molecule. A widely used concept to generate such theoretical molecular descriptors is molecular fingerprints. Molecular fingerprints encode structural or functional features of molecules in a bit string format and are commonly used for tasks like virtual screening, similarity searching and clustering.3–5 In particular, circular fingerprints like the extended-connectivity fingerprints (ECFPs) were introduced to model quantitative structure–activity relationships (QSAR) for biological endpoints by way of classical machine learning approaches as well as for ligand-based virtual screening (VS).6,7 Recent advances in the field of Deep Neural Networks (DNNs)8,9 also showed an impact in chemoinformatics-related tasks such as molecular property and activity prediction.10–12 The proposed DNNs have in common that they use pre-extracted molecular descriptors (mostly ECFPs) as input features. This, however, contradicts the fundamental idea of representation learning: DNNs should learn a suitable representation of the data from a simple but complete featurization, rather than relying on sophisticated human-engineered representations.9,13 Following these considerations, work was also done to apply DNNs directly on supposedly more complete and lower-level representations of a molecule such as the molecular graph14 or the sequential SMILES (Simplified Molecular Input Line Entry Specification) representation.15–17 By training a DNN directly on a comprehensive and low-level representation, it can automatically learn to extract useful descriptors best suited for the specific task it is trained on, resulting in a specific descriptor set for a given dataset. The downside, however, directly follows from this property. Since features have to be learned from scratch for every new dataset, these methods are prone to overfitting if trained on limited data. This is an issue when it comes to bioactivity data, due to the relatively high cost of generating a data point.11,18,19 Recently, work was also done to learn molecular descriptors in an unsupervised and data-driven way. Gómez-Bombarelli et al. proposed a variational autoencoder20 to convert the discrete SMILES representation of a molecule to and from a multidimensional continuous representation.21 Although their main purpose was to build a framework for de novo molecular design, the authors showed that the resulting representations could also be used as descriptors for a down-stream classification task. Xu et al. proposed a related unsupervised approach based on sequence to sequence learning.22,23 Both studies use an autoencoder24 methodology applied on the SMILES representation. An autoencoder comprises two neural networks, an encoder and a decoder. The encoder network transforms the input, here a SMILES sequence of variable length with discrete values, to a fixed size continuous representation (latent representation). The decoder network takes the latent representation as the input and aims at transforming it back to the input sequence. The whole autoencoder network is trained on minimizing the mean reconstruction error on a single-character level for each input sequence. By introducing an information bottleneck between the encoder and the decoder, the network is forced to compress the essential information of the input, so that the decoder still makes as few errors as possible in the reconstruction. If the trained autoencoder is able to encode all the necessary information of a given molecular representation to accurately reconstruct the original molecular representation, Xu et al. argue that it may also capture more general chemical information about the molecule and could be used as a molecular descriptor. However, training an autoencoder on reconstructing a sequence which represents a molecule bears the risk that the network solely focuses on syntactic features and repetitive patterns of this sequence, neglecting its semantics and failing to encode higher-level concepts such as molecular properties. In this work, we want to address this issue by proposing a method that is based on a translation rather than a reconstruction methodology (see Fig. 1). Similar to a human translating a sentence from one language to another by first reading the whole sentence to get a general understanding before starting translation, a so-called Neural Machine Translation (NMT)23 model first reads the whole input sequence and encodes it into an intermediate continuous vector representation (latent representation) which is then used by the decoder to emit a respective translation. This latent representation can be thought of as the model's “understanding” of the input sequence's “meaning”, incorporating all the semantic information shared by the input and output sequences. Here, we want to exploit this translation methodology to extract the “meaning” of a molecular representation like an InChI (International Chemical Identifier)25 by translating it to another syntactically different one, e.g. SMILES. Since the decoder uses the encoded latent representation to generate a semantically equivalent but syntactically different representation, the network does not benefit from encoding unnecessary information about the input sequence. However, the decoder can only succeed in generating the right translation for a given molecular representation if the encoder compresses a comprehensive description of the chemical structure in the latent representation. [Figure: General architecture of the translation model using the example of translating between the IUPAC and SMILES representations of 1,3-benzodioxole.] By training the translation model in a data-driven way on a large set of chemical structures, we propose a model that can extract the information contained in a comprehensive but discrete and variable-sized molecular representation (e.g. SMILES) and transform it into a continuous and fixed-sized representation. Once trained, the resulting model can be used to extract meaningful molecular descriptors for query structures without the need for retraining or including labels. To analyse the quality of the resulting molecular descriptors, we perform a variety of experiments on predictive QSAR and virtual screening tasks. Finally, we show that it is possible to navigate smoothly in this new continuous chemical descriptor space by modifying slightly the molecular representation of an existing compound in a given direction and using the decoder to obtain new chemical structures. METHODS Molecular representations 2.1. While translation could be performed between arbitrary molecular representations, in this work we focus on the sequence-based SMILES and InChI representations. The InChI notation represents molecular structures as a sequence of characters divided into layers and sub-layers providing different types of information such as the chemical formula, bonds and charges. The SMILES notation also represents molecular structures as a sequence of characters. In contrast to the InChI notation, however, a SMILES is not divided into different information layers but encodes the whole molecular structure in one sequence of characters including identifiers for atoms as well as identifiers denoting topological features like bonds, rings and branches. Since a molecule can typically be represented by many of equally valid SMILES, various algorithms have been developed to guarantee the uniqueness of a SMILES notation for a molecule. In this work we use the library RDKit26 to generate such canonical SMILES. Table 1 visualizes how the different notations differ in their syntax while