Training Text-to-Molecule Models with Context-Aware Tokenization
Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin · 2025
Read Training Text-to-Molecule Models with Context-Aware Tokenization by Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04476.txt.
Description
Seojin Kim and three co-authors change how molecule generators chop molecules into pieces in a 2025 research paper. Text-to-molecule models turn written descriptions into molecular structures, promising help for drug discovery. Molecules are usually tokenized atom by atom, which captures local connections but misses global structure like ring systems. Their model, CAMT5, tokenizes at the substructure level instead, treating meaningful chunks as units. An importance-based training strategy emphasizes the key substructures.
Experiments across generation tasks showed CAMT5 beating previous state-of-the-art methods while using only a small fraction of the training tokens. A simple ensemble that aggregates several models' outputs boosted results further. Code is public. Better tokenization, rather than bigger data, drove the gains.
Catalog record
| CREATOR | Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-04476