Generative Chemical Language Models for Energetic Materials Discovery
Andrew Salij, R. Seaton Ullberg, Megan C. Davis, Marc J. Cawkwell, Christopher J. Snyder, Cristina Garcia Cardona, Ivana Matanovic, Wilton J. M. Kort-Kamp · 2026
Read Generative Chemical Language Models for Energetic Materials Discovery by Andrew Salij, R. Seaton Ullberg, Megan C. Davis, Marc J. Cawkwell, Christopher J. Snyder, Cristina Garcia Cardona, Ivana Matanovic, Wilton J. M. Kort-Kamp (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03304.txt.
Introduction
Salij and colleagues build generative chemical language models to speed the hunt for energetic materials, where good data are scarce. Pretrained on broad chemical data and tuned on curated energetic sets, the models move beyond the drug-focused space where such models grew up. Fragment-based encodings also favor structures that are easier to make, aiding next-generation material design.
Catalog record
| CREATOR | Andrew Salij, R. Seaton Ullberg, Megan C. Davis, Marc J. Cawkwell, Christopher J. Snyder, Cristina Garcia Cardona, Ivana Matanovic, Wilton J. M. Kort-Kamp |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55786 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03304
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generative Chemical Language Models for Energetic Materials Discovery Andrew Salij, R. Seaton Ullberg, Megan C. Davis, Marc J. Cawkwell, Christopher J. Snyder, Cristina Garcia Cardona, Ivana Matanovic, Wilton J. M. Kort-Kamp Abstract The discovery of new energetic materials remains a pressing challenge hindered by limited availability of high-quality data. To address this, we have developed generative molecular language models that have been pretrained on extensive chemical data and then fine-tuned with curated energetic materials datasets. This transfer-learning strategy extends the chemical language model capabilities beyond the pharmacological space in which they have been predominantly developed, offering a framework applicable to other data-spare discovery problems. Furthermore, we discuss the benefits of fragment-based molecular encodings for chemical language models, in particular in constructing synthetically accessible structures. Together, these advances provide a foundation for accelerating the design of next-generation energetic materials with demanding performance requirements. keywords Machine learning, Energetic materials, Generative pre-trained transformers, Chemical language models, Fine-tuning, Low-rank adaptation † † email: [email protected] † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Weapon Stockpile Modernization Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Computing and Artificial Intelligence Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA † † email: [email protected] 1 Introduction Civilian and military applications frequently require the storage and release of chemical energy in energetic materials (EMs) 3 ; 78 , driving the discovery of successive generations of EM compounds, from the widespread TNT (trinitrotoluene) and RDX (hexogen) in the first half of the 20th century to more recent examples such as TNAZ (1,3,3-trinitroazetidine) and FOX-7 (1,1-diamino-2,2-dinitroethylene) 55 . To be effective in operational conditions, these materials must balance multiple properties such as high detonation velocities, thermal stabilities, and a desired insensitivity to shock waves, posing a persistent challenge for EM design. To avoid the substantial time, financial, and potential environmental costs of manual approaches to EM discovery, computational methods can efficiently evaluate many potential EM candidates 46 . Machine learning (ML) techniques have proven effective in connecting molecular structure to properties for EMs 75 ; 77 , highlighting the promise of artificial intelligence for materials discovery. Generative deep learning models have also begun to be explored for this domain 33 , though progress has been constrained by the scarcity of high-quality EM datasets 77 . Machine learning models ranging from those built using non-linear regression methods 5 ; 15 to multi-layer perceptrons 39 to direct message-passing neural networks 31 have demonstrated success in predicting EM properties such as detonation velocities and impact sensitivity. While predictions of properties from structures present opportunities in validating candidate materials, the reverse task of producing materials from an initial set of desired properties, or “inverse design” of materials 65 , enables the determination of materials from desired properties. Thus far, deep learning methods such as generative adversarial networks 41 and convolutional neural networks 40 have been applied for inverse design of systems relevant for EM deployment such as porous structures 14 . The inverse design generation of new molecules for energetic material systems, however, has remained relatively unaddressed, with notable exceptions in the recurrent neural networks (RNNs) developed by Xuemei Pu and coworkers 33 and a variational autoencoder produced by Wespiser and Mathieu 70 . Such approaches contribute toward the discovery of new molecules, but the complexities of molecular structure may require additional ML techniques to be properly expressed. For instance, recent advancements in large language models (LLMs) point toward language-based generative methods that leverage textual molecular encodings and transformer attention blocks 64 as a more flexible and data-efficient framework for advancing EM design. Chemical language models (CLMs) have recently emerged as tools for exploring chemical space in a general and scalable manner 19 . Here, we will view CLMs as specialized variants of general-purpose LLMs, which learn to predict sequences in natural language 38 . By focusing on molecular strings rather than words, permitting a truncated vocabulary and model size, CLMs hold the potential to achieve equal or greater quality at a fraction of the computational cost of general-purpose LLMs. By translating molecular representations into text sequences such as the Simplified Molecular Input Line Entry System (SMILES 69 ) or the Self-Referencing Embedded Strings (SELFIES 29 ), techniques honed for natural language processing become available for molecules. The three-dimensional data of the positions of atoms are compressed onto a one-dimensional textual representation that preserves bonds and bond order. Such molecular text inevitably discards some geometric information such as absolute atomic positions or relative orientation of unconnected groups, but it retains the essential chemical connectivity required for molecular modeling. The design paradigm of the generative pre-trained transformer (GPT) 51 , first developed for natural language generation, has recently been extended to small molecules and de novo molecular design 4 ; 67 . In these models, molecular text is broken into subunits, or “tokens,” which are linked through an attention mechanism that enables sequential generation of text and the completion of partial inputs 64 ; 51 . GPTs for chemistry, and transformer-based CLMs more broadly, provide robust and scalable alternatives to other generative approaches such as variational autoencoders and generative adversarial networks 72 ; 35 , producing candidate molecules with high uniqueness while maintaining chemical validity 4 ; 42 . As transformer-based CLMs train efficiently and can be reliably improved by increasing data and model sizes 17 , they offer a natural paradigm for leveraging large molecular datasets to generate plausible new compounds. Since these models learn internal chemical representations within their trained weights and biases, it is critical to ensure that essential chemical information is preserved not only during encoding and training, but also through model inference and decoding. Multiple strategies have been developed for converting molecular graphs into text suitable for machine learning. Owing to its long-standing use and general effectiveness, SMILES 69 has been a widely adopted representation in CLMs 37 . However, perceived limitations of SMILES, such as its fragility to mutation, have motivated alternative encodings, including DeepSMILES 44 , SAFE 42 , and SELFIES 29 . SELFIES enforces valency constraints on decoded graphs to preserve molecular validity following small changes to already valid structures 29 . For CLMs, SELFIES-based models have demonstrated performance comparable to, and in many cases better than, SMILES-based approaches across a range of tasks 76 , underscoring the advantages of alternative molecular encodings for chemical language modeling. As molecules often contain recurring motifs such as conjugated rings and functional groups, directly encoding fragments has been proposed to increase the semantic meaning of each unit in textual representations. Approaches such as SAFE 42 and GroupSELFIES 13 introduce vocabularies of chemically meaningful fragments, thereby improving the semantic richness of the encoding step. Much as morphemes or words