Descriptor-based Foundation Models for Molecular Property Prediction

Jackson W. Burns, Akshat S. Zalte, William H. Green · 2025

Read Descriptor-based Foundation Models for Molecular Property Prediction by Jackson W. Burns, Akshat S. Zalte, William H. Green (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15792.txt.

Introduction

Jackson Burns and co-authors build a foundation model for chemistry from classic molecule descriptors, in a paper from two thousand twenty-five. Fast property guesses drive discovery across chemistry, yet deep models often lose to classic methods on small real data sets. Their model, CheMeleon, pretrains on low-noise descriptors rather than messy experiments or biased simulations.

A message-passing network then fine-tunes from that base. Across fifty-eight benchmarks it won about three quarters of one suite's tasks and nearly all of another's. It beat random forests and rival foundation models alike. The authors pitch descriptors as a cleaner foundation for chemical learning.

Catalog record

CREATORJackson W. Burns, Akshat S. Zalte, William H. Green
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Descriptor-based Foundation Models for Molecular Property Prediction Jackson W. Burns, Akshat S. Zalte, William H. Green Abstract Fast and accurate prediction of molecular properties with machine learning is pivotal to scientific advancements across myriad domains. Foundation models in particular have proven especially effective, enabling accurate training on small, real-world datasets. This study introduces CheMeleon , a novel molecular foundation model pre-trained on deterministic molecular descriptors from the Mordred package, leveraging a Directed Message-Passing Neural Network to predict these descriptors in a noise-free setting. Unlike conventional approaches relying on noisy experimental data or biased quantum mechanical simulations, CheMeleon uses low-noise molecular descriptors to learn rich molecular representations. Evaluated on 58 benchmark datasets from Polaris and MoleculeACE, CheMeleon achieves a win rate of 79% on Polaris tasks, outperforming baselines like Random Forest (46%), fastprop (39%), and Chemprop (36%), and a 97% win rate on MoleculeACE assays, surpassing Random Forest (63%) and other foundation models. However, it struggles to distinguish activity cliffs like many of the tested models. The t-SNE projection of CheMeleon ’s learned representations demonstrates effective separation of chemical series, highlighting its ability to capture structural nuances. These results underscore the potential of descriptor-based pre-training for scalable and effective molecular property prediction, opening avenues for further exploration of descriptor sets and unlabeled datasets. keywords foundation models, descriptors, chemistry † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Main Fast and accurate prediction of molecular properties is pivotal to scientific advancements in drug discovery, material design, and catalysis. In recent years, machine learning has emerged as a transformative tool for molecular property prediction, offering rapid and scalable alternatives to resource-intensive experiments and quantum chemistry calculations. These data-driven models have demonstrated remarkable success in predicting critical properties, such as Absorption, Distribution, Metabolism, and Excretion (ADME) properties Swanson et al. 2024 and physicochemical properties like lipophilicity and solubility Yang et al. 2019 ; Attia et al. 2024 , leveraging diverse architectures and molecular representations to accelerate discovery across myriad domains. The development of machine learning models relies on a diverse set of architectures, each tailored to balance predictive power and data efficiency. Widely accessible through user-friendly Python packages, representative models include the ubiquitous Random Forest Svetnik et al. 2003 , fastprop Burns and Green 2025 , and Chemprop Yang et al. 2019 ; Heid et al. 2024 . All these methods employ direct supervised fitting, where randomly initialized parameters are trained solely on the labeled target dataset. Random Forest (RF), an ensemble of decision trees trained on molecular fingerprints (e.g., Morgan Fingerprints Morgan 1965 ) or descriptors, serves as a robust baseline for its effectiveness on small datasets Svetnik et al. 2003 . Similarly, fastprop , a feedforward neural network using fixed molecular descriptors like topological or physicochemical features, achieves strong performance on small datasets but has a lower performance ceiling due to its reliance on predefined representations Burns and Green 2025 . In contrast, Chemprop , a graph neural network based on a Directed Message-Passing Neural Network (D-MPNN), excels by learning optimal feature representations from molecular graphs during training, offering generality across a wide range of tasks like solubility Chung et al. 2022 , radical thermochemistry Zalte et al. 2025 , and infrared spectra McGill et al. 2021 prediction, among others. Studies have shown that models leveraging learned representation (LR) tend to outperform those using fixed molecular representations Yang et al. 2019 ; Magar et al. 2022 ; Aldeghi and Coley 2022 . However, in the limit of small datasets ( \lesssim\mathcal{O}(1{,}000) samples), Chemprop and other LR models struggle, often outperformed by classical methods like Random Forest Xia et al. 2023 . Learning molecular representations from scratch using deep neural networks, such as in Chemprop , requires large datasets because the initial input features are sparse and uninformative Li et al. 2024 . This forces the model to simultaneously learn both a suitable representation and the target property mapping, which is challenging in low-data regimes and leads to poor generalization and overfitting. To unlock the performance potential of LR models without suffering these limitations, recent work has focused on learning meaningful, general representations using molecular foundation models. These models are typically pre-trained on large unlabeled datasets via self-supervised learning and aim to provide rich, transferable features that can be fine-tuned for various downstream tasks. We discuss several foundation models for chemistry in the following section and refer readers to the comprehensive review by Choi et al. 2025 for a more in-depth treatment of the literature. A wide range of approaches has been developed to create molecular foundation models, differing in both molecular representations and training strategies. Inspired by advances in natural language processing (NLP), many efforts rely on text-based representations such as SMILES (Simplified Molecular Input Line Entry System) Weininger 1988 ; Weininger et al. 1989 and SELFIES Krenn et al. 2020 , which naturally lend themselves to transformer-based architectures. Notable examples include ChemBERTa-2 Chithrananda et al. 2020 and MolFormer Ross et al. 2022 , both of which are pre-trained on \mathcal{O}(10M) or more SMILES strings to learn general-purpose molecular representations. More recent work has shifted toward graph-based representations, which enable more parameter-efficient models by naturally capturing molecular structure that is absent in SMILES. Galkin et al. 2023 ; Kläser et al. 2024 These methods have evolved through a range of training strategies, broadly categorized as supervised or unsupervised based on standard machine learning conventions. Among unsupervised approaches, MolCLR Wang et al. 2022 employs contrastive learning with graph neural networks, including graph convolution and graph isomorphism networks. Another model, GROVER Rong et al. 2020 , combines graph neural networks with Transformer-style attention and is trained using contextual property and motif prediction tasks. The best-performing models in the current literature have extended graph-based learning approaches using supervised pre-training. All supervised pre-training methods broadly follow the workflow illustrated in Figure 4 . A molecular representation method first embeds the input molecule into a numerical vector, which is then passed through a regressor to predict a set of target labels. The choice of these labels distinguishes different foundation model strategies. State-of-the-art pre-training labels, such as experimental results and computed quantum mechanics properties, dominate this space. Models such as MolE Méndez-Lucio et al. 2024 and the work of Beaini et al. 2024 adopt this strategy, reporting excellent results across many common benchmarks. While these labels are typically of high quality, their availability is limited, especially compared to the large unlabeled datasets leveraged in unsupervised learning. To attain the scale required for generalizable models, experimental datasets must be combined. This necessarily results in sparsity but also likely large stochastic and even systematic error from inter-laboratory variation. For instance, combining identical assays from different labs often results in low correlation coefficients, as shown by Landrum and Riniker 2024 . This poses a challenge for modeling as most methods struggle to regress in the presence of systematic error or implicitly rely on error being stochastic. An alternative strategy is to use only labels generated from quantum mechanical (QM) simulations. Prominent models like MolGPS Sypetkowski et al. 2024 and GraphQPT Fallani et al. 2025 are trained on such data. Although simulated datasets can be significantly larger, QM methods are

Read the complete text