Multi-Objective Reinforcement Learning for Generating Covalent Inhibitor Candidates

Renee Gil · 2026

Read Multi-Objective Reinforcement Learning for Generating Covalent Inhibitor Candidates by Renee Gil (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20019.txt.

Introduction

Gil presents a machine learning pipeline that builds covalent inhibitor candidates against EGFR and acetylcholinesterase. Design must balance binding strength, target selectivity, and electrophilic reactivity at once, a task screening alone handles poorly. A SMILES-based LSTM builds candidates, tuned by policy gradient methods with Pareto crowding across scores for ease of making, covalent action, residue binding, and docking. The pipeline finds known inhibitors again and invents warhead motifs absent from training, including allenes, 3-oxo-beta-sultams, and alpha-methylene-beta-lactones, all backed by prior literature.

Catalog record

CREATORRenee Gil
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27388
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20019

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Objective Reinforcement Learning for Generating Covalent Inhibitor Candidates Renee Gil Abstract Rational design of covalent inhibitors requires simultaneously optimizing multiple properties, such as binding affinity, target selectivity, or electrophilic reactivity. This presents a multi-objective problem not easily addressed by screening alone. Here we present a machine learning pipeline for generating covalent inhibitor candidates using multi-objective reinforcement learning (RL), applied to two targets: epidermal growth factor receptor (EGFR) and acetylcholinesterase (ACHE). A SMILES-based pretrained LSTM serves as the generative model, optimized via policy gradient RL with Pareto crowding distance to balance competing scoring functions including synthetic accessibility, predicted covalent activity, residue affinity, and an approximated docking score. The pipeline rediscovers known covalent inhibitors at rates of up to 0.50% (EGFR) and 0.74% (ACHE) in 10,000-structure runs, with candidate structures achieving warhead-to-residue distances as short as 5.5 Å (EGFR) and 3.2 Å (ACHE) after further docking-based screening. More notably, the pipeline spontaneously generates structures bearing warhead motifs absent from the training data - including allenes, 3-oxo- β \beta -sultams, and α \alpha -methylene- β \beta -lactones - all of which have independent literature support as covalent warheads. These results suggest that RL-guided generation can explore covalent chemical space beyond its training distribution, and may be useful as a tool for medicinal chemists working on covalent drug discovery. 1 Introduction Covalent inhibitors form a permanent or long-lived bond with a nucleophilic residue in the target protein through a two-step mechanism: initial non-covalent recognition positions the molecule at the binding site, after which the electrophilic warhead reacts irreversibly with the target residue. Compared to non-covalent inhibitors, this confers advantages in potency, selectivity, and residence time, and enables applications where non-covalent binding is insufficient - including targets with shallow binding pockets, acquired drug-resistance mutations, and cases requiring sustained target suppression. [ 2 , 11 , 16 ] The clinical success of ibrutinib and osimertinib demonstrated that covalent inhibitors can be rationally designed with high target selectivity, reinvigorating this class of therapeutics after decades of concern over off-target reactivity. [ 1 , 13 ] Their design, however, presents an additional constraint absent in non-covalent drug discovery: the reactive warhead must be positioned close to the target residue, requiring careful integration of binding geometry and electrophilic reactivity. Conventional virtual screening approaches can assess existing compound libraries but are inherently limited to explored chemical space. Generative models can sample novel molecules that are not in any training set, and reinforcement learning provides a principled framework for steering generation towards user-defined objectives. [ 22 , 5 , 17 ] RL-guided molecular generation has shown promise for general drug design; its application to covalent inhibitors specifically remains underexplored. In this work, we develop a multi-objective RL pipeline tailored to covalent inhibitor generation, incorporating scoring functions specific to covalent drug discovery: a graph convolutional network covalent activity classifier (GCNII, from [ 12 ] ) and a graph attention network residue affinity classifier (GAT, developed in this work), both trained on ProteinReactiveDB [ 12 ] , alongside a new graph neural network (GNN) docking score approximation, synthetic accessibility, structural overlap and similarity to known inhibitors, and drug-likeness. We apply this pipeline to EGFR (targeting Cys797) and ACHE (targeting Ser200), demonstrate rediscovery of known inhibitors and generation of structures with suitable warhead-residue geometry, and report the spontaneous emergence of atypical warhead scaffolds with no precedent in our training data. 2 Methods 2.1 Pipeline Overview The pipeline consists of three components: a pretrained generative model, a set of molecular scoring functions, and a reinforcement learning algorithm that optimizes the generator to maximize the scores. A schematic is shown in Figure 1 . Figure 1 : Schematic of the generative RL pipeline. 2.2 Generative Model Molecule generation used the DrugEx [ 24 ] framework, which implements a SMILES-based Long Short-Term Memory (LSTM) [ 15 ] model trained to predict the next character of a SMILES string. Rather than training from scratch, we used a publicly available pretrained LSTM trained on the Papyrus [ 3 ] dataset (1,270,570 unique structures compiled from ChEMBL [ 10 ] , PubChem [ 19 ] , PDBBind [ 28 ] , and BindingDB [ 21 ] ), available in the DrugEx repository ( https://github.com/CDDLeiden/DrugEx , qsprpred extra). 2.3 Scoring Functions Table 1 lists all scoring functions. Scorers marked with * are included in all models and enforce minimum viability; the remaining scorers are added incrementally across models (see Section 2.5 ). Table 1 : Scoring functions used for evaluating generated molecules. Scorers marked * are included in all models and enforce minimum viability. All raw scores are clipped to [0,1] before being passed to the RL algorithm (see Figure 2 ); thresholds are applied to the clipped scores to define minimally desirable structures. Scorer Description Threshold Validity* SMILES grammar validity. must be valid Synthetic Accessibility (SA)* Synthesizability score of Ertl et al. [ 8 ] ; raw scale 1-10 (lower = easier to synthesize). raw \leq 6 Covalent Activity* Probability of covalent activity from the GCNII classifier of Gil & Rowley [ 12 ] . p\geq 0.75 Covalent Residue Affinity* Probability of reaction with the target residue class (Cys or Ser/Thr) from a GAT classifier developed in this work (Section 2.3.1 ). p\geq 0.75 Docking Score Approximation* GNN approximation of AutoDock Vina docking score for EGFR or ACHE (Section 2.3.2 ); raw scores in kcal/mol, more negative = stronger binding. \leq-6.0 kcal/mol Known Structure Overlap Crippen-weighted 3D alignment score (RDKit GetCrippenO3A [ 20 ] ) against a reference inhibitor. score \geq 100 Tanimoto Similarity Morgan fingerprint similarity ( r=2 , 2048 bits) to a reference scaffold. T\geq 0.1 QED [ 4 ] Quantitative drug-likeness estimate based on MW, LogP, H-bond counts, and related properties. none The clipping functions are shown in Figure 2 . Figure 2 : Clipped scoring functions for (a) SA, (b) Covalent Activity, (c) Residue Affinity, (d) Docking Score, (e) Overlap, (f) Tanimoto Similarity, (g) QED. Red lines mark desirability thresholds. All clipping functions map the raw score to [0,1] ; any generated structure falling below all active thresholds is counted as desirable. Threshold selection. Thresholds were chosen as follows. SA : the Ertl et al. scale characterizes raw scores above 6 as moderately to very difficult to synthesize (1-3 very easy; 3.1-5 moderately easy; 5.1-6.5 moderately difficult; 6.6-10 very difficult or impractical). A threshold of 6 was chosen to admit moderately challenging structures while excluding those likely to be impractical; the clipping function levels off gradually toward zero rather than applying a hard floor, so the optimizer is penalized but not blocked for slightly difficult structures. Covalent Activity and Residue Affinity : both classifiers output a probability in [0,1] . A threshold of p\geq 0.75 was chosen to ensure high confidence that generated structures are covalently reactive and target the intended residue type. The clipping function additionally applies a hard floor at p=0.5 : any structure predicted to be more likely non-covalent than covalent receives zero reward for that scorer. Docking Score Approximation : AutoDock Vina scores below -10 kcal/mol indicate very strong binding; -10 to -7 strong; -7 to -5 moderate; above -5 weak or unlikely. The threshold of \leq-6.0 kcal/mol (moderate-to-strong range) was set to match the upper tail of the docking score distribution

Read the complete text