Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design

Shriram Chennakesavalu, Kirill Shmilovich, Hayley Weir, Colin Grambow · 2026

Read Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design by Shriram Chennakesavalu, Kirill Shmilovich, Hayley Weir, Colin Grambow (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16279.txt.

Introduction

Chennakesavalu and seven co-authors test large language models for small-molecule drug design, where benchmarks matching real-world use have been missing. They build a suite of lab-grounded tasks with trait forecasting, form changes, and drug design, framed as reinforcement learning setups for unified testing and post-training. Across three model groups, frontier models grow more skilled at lab tasks, yet much room remains, above all with low data. Reward-based post-training lifts a smaller model to frontier-level standing.

Catalog record

CREATORShriram Chennakesavalu, Kirill Shmilovich, Hayley Weir, Colin Grambow
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78965
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16279

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design Shriram Chennakesavalu, Kirill Shmilovich, Hayley Weir, Colin Grambow Abstract Large Language Models (LLMs) have the potential to accelerate small molecule drug design due to their ability to reason about information from diverse sources and formats. However, their practical utility remains unclear due to the lack of benchmarks that reflect real-world scenarios. In this work, we introduce a suite of chemically-grounded tasks spanning molecular property prediction, molecular representation transformations, and molecular design. Importantly, we formulate these tasks as reinforcement learning (RL) environments, enabling a unified approach for evaluation and post-training. Across three model families, we find that frontier models are increasingly proficient at chemical tasks, but that there is significant room for improvement, especially in experimental settings with low data. Critically, we show that RL-based post-training can substantially improve performance. A smaller model post-trained on our environments becomes competitive with state-of-the-art frontier models, despite a significantly weaker base model. This suggests a practical route toward employing LLMs in drug discovery; by combining carefully-designed evaluation tasks with targeted post-training, we can both elucidate and close critical capability gaps. 1 Introduction Drug-discovery campaigns are truly multidisciplinary efforts that rely on a vast set of computational, experimental, and clinical approaches to successfully bring a drug to market ( Paul et al., 2010 ; Hughes et al., 2011 ; Macalino et al., 2015 ; Rácz et al., 2025 ) . Designing generalist systems that can seamlessly synthesize information and leverage tools across the full drug discovery pipeline can potentially reduce the time and cost of designing a drug and help us avoid failure modes that plague many campaigns today (e.g., clinical inefficacy, safety issues) ( Harrison, 2016 ; Paul et al., 2010 ) . Recently, Large Language Models (LLMs) have emerged as truly generalist models, capable of synthesizing knowledge and generating new ideas across many diverse disciplines ( Anthropic, 2026 ; OpenAI et al., 2023 ; Google, 2025 ) . In many real-world settings, LLMs are often augmented with tools, enabling these so-called agents to carry out more sophisticated tasks, via information acquisition from and/or modification of an external environment ( Yao et al., 2022 ; Bran et al., 2024 ; Boiko et al., 2023 ) . LLM-based agents have recently been deployed in many drug discovery contexts, including for target identification, lead optimization, and toxicity prediction ( Bran et al., 2024 ; Boiko et al., 2023 ; Ramos et al., 2024 ) . While these are compelling demonstrations, many of these agents are limited by the performance of the base LLM ( Bran et al., 2024 ; Putta et al., 2024 ) . Frontier models struggle at many of the fundamental biological and chemical tasks ( Narayanan et al., 2025 ; Anthropic, 2026 ) that underpin many of these drug discovery agents, and building truly robust agents for drug discovery will require stronger base models. Understanding both the current capabilities of existing LLMs and the expected capabilities of future LLMs is crucial to designing agents for drug discovery. While frontier LLMs have known limitations for drug discovery tasks ( Narayanan et al., 2025 ) , these models are rapidly advancing and this progression will ultimately inform efforts to build agents, including identifying realistic use-cases and understanding how to allocate resources between model training and harness engineering. Understanding this progression in capabilities of models is challenging, in large part, due to the closed nature of many of these models. Both open-weight and closed frontier models generally do not disclose their training recipes, making it difficult to forecast how models will evolve in the short and long-term. Early generations of LLMs were primarily improved through the scaling of pretraining, by increasing model size, compute, and dataset size ( Kaplan et al., 2020 ) . This approach enabled gains to be realized across tasks generally . More recent models have been improved by extensive post-training, which has resulted in a so-called “jagged frontier,” where model capabilities in certain domains far outstrip capabilities in other domains ( DeepSeek-AI et al., 2025 ; Team et al., 2025 ; Yang et al., 2025 ; Narayanan et al., 2025 ) . Current training recipes of LLMs can be decomposed into two parts: base-model training and post-training ( Dubey et al., 2024 ; NVIDIA et al., 2025 ; Olmo et al., 2025 ; Ren and Sutherland, 2024 ) . Base-model training, which includes pretraining on vast datasets and midtraining on large but higher-quality datasets, fundamentally increase the knowledge and skills of the model ( Olmo et al., 2025 ) . Post-training, such as with Reinforcement Learning (RL) methods like Direct Preference Optimization (DPO) ( Rafailov et al., 2023 ) and Group Relative Policy Optimization (GRPO) ( DeepSeek-AI et al., 2025 ; Shao et al., 2024 ) sharpen the capabilities of the model on particular skills acquired during base-model training. With base-model training already incorporating knowledge from diverse corpora, one perspective for evaluating the potential capabilities of frontier models is to assess how easily their performance can be “sharpened” for a domain of interest using a post-training-only training scheme ( Yue et al., 2025 ) . For open-weight models, we can directly assess this potential by measuring the gap between the base model and a post-trained model. For closed models, we assess their potential capabilities by comparing the performance of subsequent iterations of a model within a model family. While the differences between various model iterations are not publicly transparent, examining the differences in model performance over time can serve as a proxy for how easily the model can be improved. In this paper, we focus on a narrow drug discovery setting—small-molecule design—and examine the capabilities and trainability of three families of frontier models: GPT-5, Claude Opus 4, and Qwen-30B-A3B. We present a collection of tasks, ranging from property prediction to chemical language translation to molecular design. Importantly, we frame these tasks as Reinforcement Learning (RL) environments, enabling us to also post-train open-weight models on these tasks. We analyze the progression of model capabilities across model iterations within each of these families and find that generally subsequent iterations of models perform better on these tasks. Additionally, we find that our post-trained model is competitive with closed frontier models on a multi-turn molecular design task that simulates real-world lead optimization despite the base model being significantly weaker. Lastly, we find that frontier models and post-trained models struggle on experimental datasets with limited data suggesting that many of these tasks are out-of-distribution for LLMs currently and that we require further base-model training before LLMs can be meaningfully useful for these tasks. 2 Related Work Benchmarking LLMs. Benchmarking and evaluation have been a key driver of modern ML advancements across multiple application areas ( Deng et al., 2009 ; Rajpurkar et al., 2016 ; Huang et al., 2021 ; Hardt and Recht, 2022 , Chp.8 ), although such practices are not without their issues ( Dehghani et al., 2021 ; Bowman and Dahl, 2021 ) . As much of the development of LLMs has shifted into private settings, new releases of LLMs are often first evaluated and judged by their performance on a commonly used set of benchmarks (see, e.g., Table 2 of OpenAI

Read the complete text