ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design
R Yadunandan, Nimisha Ghosh · 2025
Read ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design by R Yadunandan, Nimisha Ghosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20958.txt.
Introduction
R Yadunandan and Nimisha Ghosh build a drug-design system that offers new compounds already shaped by how chemists in fact make them. ReACT-Drug needs no fresh training per target. It embeds the target protein with the ESM-2 model and finds like proteins in a structure database. Known drug ligands of those proteins are split into fragments that seed the search on relevant ground.
A reinforcement-learning agent then edits a ChemBERTa-coded compound through edits drawn from real reaction templates, so each step stays valid and plausible to make. On the MOSES benchmark the results show strong binding affinity and high ease of synthesis, with full validity and novelty. Joining structural biology, learned codes, and synthesis rules points toward machine-led design that a drug chemist could act on at once.
Catalog record
| CREATOR | R Yadunandan, Nimisha Ghosh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-20958
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design R Yadunandan, Nimisha Ghosh Abstract De novo drug design is a crucial component of modern drug development, yet navigating the vast chemical space to find synthetically accessible, high-affinity candidates remains a significant challenge. Reinforcement Learning (RL) enhances this process by enabling multi-objective optimization and exploration of novel chemical space - capabilities that traditional supervised learning methods lack. In this work, we introduce ReACT-Drug , a fully integrated, target-agnostic molecular design framework based on Reinforcement Learning. Unlike models requiring target-specific fine-tuning, ReACT-Drug utilizes a generalist approach by leveraging ESM-2 protein embeddings to identify similar proteins for a given target from a knowledge base such as Protein Data Base (PDB). Thereafter, the known drug ligands corresponding to such proteins are decomposed to initialize a fragment-based search space, biasing the agent towards biologically relevant subspaces. For each such fragment, the pipeline employs a Proximal Policy Optimization (PPO) agent guiding a ChemBERTa-encoded molecule through a dynamic action space of chemically valid, reaction-template-based transformations. This results in the generation of de novo drug candidates with competitive binding affinities and high synthetic accessibility, while ensuring 100% chemical validity and novelty as per MOSES benchmarking. This architecture highlights the potential of integrating structural biology, deep representation learning, and chemical synthesis rules to automate and accelerate rational drug design. The dataset and code are available at https://github.com/YadunandanRaman/ReACT-Drug/. I Introduction Development of a new drug where novel drug molecules are created is a time consuming and a very costly process. For such drug design, the number of feasible synthetic molecules is of the order of 10^{60}-10^{100} , out of which the suitable compound needs to be identified that should satisfy properties like bioactivity, drug metabolism and pharmacokinetic profile etc. [ 1 ] . This in itself is a very gigantic task and to mitigate such an issue, computational approaches can be very useful for de novo drug design which can help in significantly reducing the expenses and time of drug development. In the recent past, many deep learning algorithms have contributed significantly in molecule generation. In this regard, generative models have played a very important role wherein the models capture the inherent molecular distribution of the training dataset and then use this learned distribution to generate new molecules. AutoEncoder- (AE) [ 2 , 3 ] and Variational AE-based [ 4 , 5 , 6 , 7 , 8 , 9 ] models transform training molecules to a latent space using an encoder, generate new molecules in that latent space and then reconstructs the molecules using a decoder. Also, Generative Adversarial Network (GAN) [ 10 , 11 ] based models have also shown quite good results in molecule generation. Furthermore, Transformer-based generators [ 12 , 13 ] , Diffusion-based generators [ 14 , 15 ] and Recurrent Neural Network [ 16 , 17 ] -based generators have also shown immense potential and promising results. In the recent past, works like [ 18 ] have utilized Reinforcement Learning (RL) to generate drug-like molecules that are similar to the original molecule while [ 19 ] generate drug molecules from scratch. RL has shown promising results for de novo molecular generation where the objective is learning a policy than can sample sequences of tokens to generate simplified molecular-input line-entry system (SMILE) strings. RL methods with sequence-based approaches are also quite popular for controlling SMILES generation. These approaches employ the benefits of large language models that are pretrained on a huge number of chemical datasets, thereby capturing the chemical language quite well and providing encouraging results [ 20 , 1 ] . In [ 21 ] , the authors have proposed MOLRL ((Molecule Optimization with Latent Reinforcement Learning) that uses an RL paradigm called Proximal Policy Optimization (PPO) [ 22 ] to optimize molecules in the latent space of a pretrained generative model. This framework is agnostic to the architecture of the generative model although the characteristics of the latent space have an important effect on the optimization method. In [ 23 ] , the authors explore many policy optimization algorithms including PPO for de novo drug design in order to generate different types of molecules with high scores. Gupta et al. [ 24 ] have proposed Policy-guided Unbiased REpresentations (PURE) which employs semi-supervised and agnostic training design along with template-based molecular simulations for Structure-Constrained Molecular Generation (SCMG) tasks. SCMG is a method that uses existing molecules to guide the creation of new ones with similar structures but improved properties. Motivated by the literature, in this work, we propose the design of a de novo molecular drug discovery system based on an RL pipeline. It integrates ESM-2 [ 25 ] for protein embedding and ChemBERTa [ 26 ] for molecular representation which is a fragment-based approach for initial state generation, a novel reaction template library for molecular transformations, and an in silico docking oracle (AutoDock Vina) for reward calculation. The entire workflow is managed by a PPO agent designed to navigate a dynamic action space while optimizing molecules toward multiple objectives. Our contribution can be summarized as follows: • We propose a novel target-agnostic, policy-based reinforcement learning technique viz. ReACT-Drug for de novo drug design for a target protein by utilizing a fragment-based state generation. • ReACT-Drug eliminates the need for extensive supervised pre-training on specific target-drug pairs, relying instead on pre-trained generalist encoders and episodic reinforcement learning. • The experimental results demonstrate that the proposed framework shows superior or competitive performance as compared to the existing models. II Materials and Methods In this work, the discovery process begins by defining a target protein (PDB structure, sequence, and binding site) and subsequently the system identifies known drug ligands for functionally similar proteins using ESM-2 embeddings. These ligands are decomposed into a pool of chemical fragments. An RL episode starts by randomly selecting one fragment, which the PPO agent iteratively “grows” by applying chemically valid transformations from a pre-compiled library. At each step, the agent’s action (the choice of transformation) is guided by a policy network. The resulting molecule is evaluated by a multi-objective reward function, dominated by its docked binding affinity to the target, which is calculated using AutoDock Vina. The agent learns to select transformations that maximize this multi-objective reward, ultimately generating novel, high-affinity, and drug-like molecules. All computational experiments, including the training of the PPO agent and the docking simulations, are conducted using an NVIDIA L4 GPU. Figure 1 illustrates the overall workflow of the system, from protein target initialization to RL-driven molecular optimization. Fig. 1 : De novo molecular drug discovery pipeline workflow. The system initializes with a target protein, identifies similar proteins using ESM-2 embeddings, fragments their ligands, and uses a PPO agent to iteratively optimize molecules over 15-step episodes. The agent selects chemical transformations from a drug-relevant template library, with rewards based on binding affinity (AutoDock Vina), drug-likeness (QED), synthetic accessibility, and novelty. II-A Data and Model Preparation II-A 1 Protein-Ligand Knowledge Base The system leverages the PDBbind v2020 refined dataset as its primary source of protein-ligand interaction data. This dataset is processed to extract protein sequences, canonical ligand SMILES strings, and experimental binding affinity values. This processed data serves as a “knowledge base” for identifying relevant starting molecules.