Quantum Chemistry Driven Molecular Inverse Design with Data-free Reinforcement Learning
Francesco Calcagno, Luca Serfilippi, Giorgio Franceschelli, Marco Garavelli, Mirco Musolesi, Ivan Rivalta · 2025
Read Quantum Chemistry Driven Molecular Inverse Design with Data-free Reinforcement Learning by Francesco Calcagno, Luca Serfilippi, Giorgio Franceschelli, Marco Garavelli, Mirco Musolesi, Ivan Rivalta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12653.txt.
Introduction
Designing a molecule to order, rather than finding one by luck, has challenged chemists for decades. Francesco Calcagno and co-authors present a generator that learns without any training data set in a 2025 paper. A reinforcement learning system builds molecule codes step by step. Five linked models guide it to follow the grammar of a text code for molecules. Quantum chemistry checks score each new molecule as it is made.
The scorer runs on the fly and also samples the molecule's shapes. Known test problems were solved to their best known answers. New regions of chemical space yielded near-best finds as well. The method ran much faster than a standard baseline. The authors offer it as a data-free route to custom molecules.
Catalog record
| CREATOR | Francesco Calcagno, Luca Serfilippi, Giorgio Franceschelli, Marco Garavelli, Mirco Musolesi, Ivan Rivalta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-12653
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quantum Chemistry Driven Molecular Inverse Design with Data-free Reinforcement Learning Francesco Calcagno, Luca Serfilippi, Giorgio Franceschelli, Marco Garavelli, Mirco Musolesi, Ivan Rivalta Abstract The inverse design of molecules has challenged chemists for decades. In the past years, machine learning and artificial intelligence have emerged as new tools to generate molecules tailoring desired properties, but with the limit of relying on models that are pretrained on large datasets. Here, we present a data-free generative model based on reinforcement learning and quantum mechanics calculations. To improve the generation, our software is based on a five-model reinforcement learning algorithm designed to mimic the syntactic rules of an original ASCII encoding based on the SMILES one, and here reported. The reinforcement learning generator is rewarded by on-the-fly quantum mechanics calculations within a computational routine addressing conformational sampling. We demonstrate that our software successfully generates new molecules with desired properties finding optimal solutions for problems with known solutions and (sub)optimal molecules for unexplored chemical (sub)spaces, jointly showing significant speed-up to a reference baseline. 1 Introduction The inverse design of new molecules [ 1 ] is one of the leading challenges of chemistry this century [ 2 , 3 , 4 ] , aiming to generate de novo compounds with desired properties [ 1 ] . This is a fundamental paradigm shift in computational chemistry that promises a fast discovery of, e.g., new catalysts, drugs, molecular energy storage, and carbon-capturing systems. However, the complex structure-property relationship in molecules [ 5 ] and the lack of a unifying theory to solve this problem limit its development. Moreover, brute-force approaches are computationally infeasible due to the exponential size of the chemical space (CS) [ 6 ] . Despite significant efforts in developing physics-based methods as reported by Batista [ 7 , 8 ] and Reiher [ 9 , 10 ] , machine learning (ML)-based methods have recently emerged as powerful tools to accelerate the generation of molecules with predefined properties [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ] . However, ML models rely on large datasets and do not guarantee a thorough exploration of the CS, leaving a general and data-free approach to inverse design molecules an open question [ 15 ] . Among different generative models, those based on reinforcement learning (RL) [ 18 ] are extremely promising [ 16 , 15 , 17 ] . In RL, an artificial agent learns an optimal policy to exploit a task by interacting with its environment through a trial-and-error procedure. In the inverse design of molecules, an RL agent learns how to generate molecules that maximize desired properties, hereafter called chemical reward ( r_{c} ). To the best of our knowledge, a limited number of remarkable examples of data-free generations of molecules based on RL have been reported to date. Those models adopt reward metrics based on physicochemical properties, like the drug-likeliness (QED) [ 19 ] or lipophilicity (estimated by the logarithm of the partition coefficient, logP) [ 20 , 21 , 22 ] , which are not based on quantum mechanics (QM) first-principles. QM-driven generation was recently reported for organic electronic molecular design, but using pretrained language models [ 23 ] . Therefore, a method for QM-driven fully data-free generation of molecules is missing, representing a major gap in the field. In this work, we present PROTEUS, a new tool for data-free RL algorithm for molecular inverse design employing on-the-fly QM calculations of the target property. We successfully applied PROTEUS to design chemical substituents for a molecular backbone to maximize energy difference between two geometrical isomers – hereafter referred to as the isomerization energy . We demonstrated that PROTEUS enables an extensive and effective exploration of highly challenging and chemically-relevant fully-characterized spaces of solutions comprising up to 2,430,845 solutions, paving the way for a new paradigm in the de novo generation of molecules. 2 Results and discussion 2.1 The P-SMILES syntax and the isomerization energy problem In the present work we targeted the maximization of the isomerization energy of the double C=C bond of a styrene backbone by inversely designing tailored substituents (see the inset in Fig. 1 a). The structure-property relationship for such a relatively simple molecular system – i.e. the correlation between the structure of styrene derivatives and the corresponding isomerization energy – is not trivial since the geometrical isomerization involves a double bond that is conjugated with an aromatic ring. Molecules are encoded using P-SMILES, an ASCII encoding scheme here introduced (see Sec. 4 ). P-SMILES is a SMILES-based [ 24 , 25 , 26 ] syntax that encompasses a less complex and more compact syntax, and that mitigates sources of bias during the generative RL simulations (see Sec. 4 ). P-SMILES simplifies the encoding by limiting to two the maximum number of tokens required to define any structural moiety, i.e. it uses either single- or double-character notation. This reduces significantly the syntactical complexity of encoding geometrical isomers and aromatic rings (see Tab. 1 ), and, thus, the sources of bias in the generative RL procedure introduced by inequalities inherent to SMILES (see Sec. B.1 ). Thus, as detailed in Sec. 4 , PROTEUS’s generative model is designed to fit the well-defined syntax properties of P-SMILES. We approached a rigorous assessment of PROTEUS by a direct comparison between the set of molecules generated during RL experiments and the corresponding complete space of possible solutions. To keep this comparison computationally feasible, we considered complete chemical subspaces (SubCS) featuring different dimensions. The largest dataset considered, hereafter the reference ‘E/Z dataset’, involves all possible sets of R 1 and R 2 substituents for the styrene’s backbone resulting from the combinations of maximum 6 P-SMILES tokens. The reference dataset contains 1,628 chemically meaningful pairs of E/Z isomers out of all possible syntactic combinations, i.e. 1,948,716 pairs (see Appendix B ). The E/Z isomerization energy of each pair in the reference dataset was computed through a multi-step routine that accounts for QM calculations (see Sec. 4 ). Interestingly, the distribution of isomerization energies of the molecular pairs within the space of solutions highlights the complexity of the problem under investigation. Clustering of the molecules with principal component analysis (PCA) shows that the complete set of molecules can be grouped in four (Fig. 1 a) or three – when R 2 =H (Fig. 1 b) or R 1 =H (Fig. 1 c) – main clusters with similar molecular features (see Sec. B.3.4 ). In each cluster, however, the distribution of energy gaps is heterogeneous, i.e. clusters contain both positive and negative values. Figure 1: Chemical space analyses and P-SMILES syntax. Principal component analysis (PCA) of ( a ) the reference ‘E/Z dataset’, ( b ) the trans / cis subspace (i.e. with R 2 =H ) and ( c ) the cis / trans subspace (i.e. with R 1 =H ) the. PCAs use Morgan fingerprints of Z molecules with color coding based on the energy gaps computed on DFT-optimized geometries. The structural formula and the P-SMILES strings of the molecules with the largest energy gaps, i.e. Ca1C(CCCCa1)CECC(EC)F ( a and b ) and Ca1C(CCCCa1)CECCONECF ( c ), are shown. PCAs were done encoding molecules in bitvectors using the Morgan fingerprint scheme (radius =