Improving Chemical Understanding of LLMs via SMILES Parsing

Yunhui Jang, Jaehyung Kim, Sungsoo Ahn · 2025

Read Improving Chemical Understanding of LLMs via SMILES Parsing by Yunhui Jang, Jaehyung Kim, Sungsoo Ahn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16340.txt.

Introduction

Yunhui Jang and co-authors teach language models chemistry by parsing molecule codes in this 2025 paper. Molecules are often written as SMILES strings, a compact line code. Models today misread those strings, even failing to count rings in a molecule.

Their framework, CLEANMOL, turns parsing into a set of clean tasks. Tasks run from matching small parts to matching whole molecular graphs. Each task has a definite right answer, so training stays honest. The graded suite builds graph-level grasp step by step. Better molecule reading could lift drug and materials discovery with AI. It may also cut the silly errors that erode trust in such tools.

Catalog record

CREATORYunhui Jang, Jaehyung Kim, Sungsoo Ahn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16340

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving Chemical Understanding of LLMs via SMILES Parsing Yunhui Jang, Jaehyung Kim, Sungsoo Ahn Abstract Large language models (LLMs) are increasingly recognized as powerful tools for scientific discovery, particularly in molecular science. A fundamental requirement for these models is the ability to accurately understand molecular structures, commonly encoded in the SMILES representation. However, current LLMs struggle to interpret SMILES, even failing to carry out basic tasks such as counting molecular rings. To address this limitation, we introduce CleanMol , a novel framework that formulates SMILES parsing into a suite of clean and deterministic tasks explicitly designed to promote graph-level molecular comprehension. These tasks span from subgraph matching to global graph matching, providing structured supervision aligned with molecular structural properties. We construct a molecular pretraining dataset with adaptive difficulty scoring and pre-train open-source LLMs on these tasks. Our results show that CleanMol not only enhances structural comprehension but also achieves the best or competes with the baseline on the Mol-Instructions benchmark. 1 Introduction Molecular string representations such as SMILES ( Weininger, 1988 ) and SELFIES ( Krenn et al., 2020 ) have become a standard format for applying large language models (LLMs) to chemistry. These one-dimensional strings flatten molecular graphs by traversing atoms and bonds and are syntactically compatible with LLMs ( Xia et al., 2025 ; Taylor et al., 2022 ; Edwards et al., 2022 ; Christofidellis et al., 2023a ; Pei et al., 2023 ; Fang et al., 2024 ) . As a result, most molecular LLMs adopt training paradigms from the natural language processing domain, treating molecular strings as sequences of tokens analogous to sentences in natural language. However, molecular strings follow complex syntactic rules for encoding molecular structures, which LLMs often struggle to interpret. For instance, SMILES grammar includes specific conventions to denote rings and branches—often involving non-contiguous tokens to represent connected substructures. Additionally, SMILES representations must satisfy structural constraints such as proper valency and ring closure. As a result, current LLMs often misinterpret SMILES, which implies a failure to capture the underlying molecule represented by the SMILES string. This is reflected in their inability to perform even basic tasks, such as counting the number of rings or producing consistent outputs for different SMILES strings of the same molecule ( Jang et al., 2024 ; White et al., 2023 ; Ganeeva et al., 2024 ) . Our experiments revisit such limitations, as shown in Figure 1 and Section 2.2 . (a) Illustration of SMILES parsing tasks. (b) Failure of LLMs on SMILES parsing. Figure 1: Overview of SMILES parsing . ( 1(a) ) Each column visualizes one of the five SMILES parsing tasks: functional group matching , ring counting , carbon chain length measurement , SMILES canonicalization , and fragment assembly . The highlighted tokens in the SMILES correspond to the substructures involved in each task. ( 1(b) ) Recent LLMs fail for SMILES parsing while the model trained with our CleanMol shows improvement. One might expect such an understanding would “naturally emerge” from training LLMs on large corpora of SMILES strings for downstream tasks such as molecular generation and retrosynthetic analysis. However, high-quality data is limited and difficult to obtain. Unlike text or image data, which can be gathered at scale via web scrapping, chemical data often require expensive wet lab experiments or simulations for annotation. Although open-source datasets such as USPTO series ( Wei et al., 2010 ; Lu and Zhang, 2022 ) and MoleculeNet ( Wu et al., 2018 ) exist, their scale remains modest compared to datasets in other domains ( Deng et al., 2009 ; Raffel et al., 2020a ; Lozhkov et al., 2024 ) . Consequently, most chemical LLMs often rely on ambiguous and indirect pretraining objectives with non-deterministic and unclear tasks (e.g., masking each token in SMILES and reconstruct them or translation between a molecular string and its description) ( Pei et al., 2023 ; Edwards et al., 2022 ) , or focus on instruction tuning with limited-scale datasets ( Fang et al., 2024 ; Yu et al., 2024 ) . In response, we propose SMILES parsing —a suite of clean, deterministic, and scalable tasks that require models to extract structural information from molecular strings, as illustrated in Figure 1 . We argue that a natural and necessary candidate task for training LLMs to understand the SMILES representation is the extraction of deterministic graph-level information from molecular structures. To address this, we define five SMILES parsing tasks including subgraph matching (e.g., functional group, ring size, and chain length) and global graph matching (e.g., SMILES canonicalization and fragment assembly). Each task provides unambiguous supervision with deterministic answers. Based on these tasks, we construct the CleanMol dataset, consisting of 250K molecules annotated via lightweight molecular graph analysis tools such as RDKit ( Landrum et al., 2024 ) . Notably, our approach is scalable since the annotations for these tasks do not require any experiment or human annotation, in principle, SMILES parsing can be applied to all the existing molecules in the real world. To evaluate and demonstrate the benefit of our new CleanMol dataset, we also introduce a two-stage training framework: first, the model is pre-trained on the proposed SMILES parsing tasks and then fine-tuned on downstream chemical applications. To enhance data efficiency in the first stage, we propose a task-adaptive data pruning that selects structurally informative molecules and a curriculum learning framework that organizes them from easy to hard order. We empirically validate our approach by training recent LLM backbones ( Grattafiori et al., 2024 ; Yang et al., 2024 ) and evaluating them on three downstream tasks from the Mol-Instructions benchmark ( Fang et al., 2024 ) , including retrosynthesis, reagent prediction, and forward reaction prediction. Surprisingly, our clean and structure-aware CleanMol framework enables the models to achieve state-of-the-art or competitive results on the downstream tasks. This demonstrates that incorporating deterministic structural supervision via SMILES parsing can significantly enhance molecular generation capabilities, even without direct exposure to generation-specific training data. We summarize our contributions as follows: • We revisit the limitations of LLMs in interpreting molecular strings, highlighting the structural bottleneck. • We propose five deterministic and scalable SMILES parsing tasks and introduce the CleanMol dataset to bridge the gap between string-level and graph-level molecular understanding of LLMs. • We design a two-stage training framework incoporating a task-adaptive data pruning and curriculum learning strategy. • We validate the impact of CleanMol by demonstrating a consistent performance improvement across multiple downstream tasks. 2 SMILES parsing task In this section, we introduce five SMILES parsing tasks designed to enhance the mapping between molecular SMILES strings and their corresponding graph structures. We then highlight two key bottlenecks in applying LLMs to molecular tasks: (1) the inability of models to extract structural information from SMILES strings and (2) the lack of high-quality, scalable molecular datasets. To address the first bottleneck, we show that even advanced LLMs such as GPT-4o ( OpenAI and et al., 2024 ) and DeepSeek-V3 ( Liu et al., 2024 ) fail to perform well on simple SMILES parsing tasks, revealing the need for explicit structure-aware supervision. To address the second bottleneck, we explain the limitation of open-source molecular datasets, motivating the need for scalable molecular datasets that can be generated

Read the complete text