BiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning

Aditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora, Nitin Kumar, Prathosh A P, Sandeep Kumar · 2026

Read BiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning by Aditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora, Nitin Kumar, Prathosh A P, Sandeep Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24089.txt.

Introduction

Shahane and five colleagues present BiMol-Diff, a single diffusion framework for both molecule generation from text and molecule captioning. Standard diffusion corrupts all positions alike, which can damage structurally key tokens, while autoregressive models strain on long-range links. A token-aware noise schedule corrupts each position by how hard it is to recover, protecting tough substructures. On ChEBI-20 and M3-20M it gains about 15 percent in exact match for reconstruction and leads baselines on captioning scores.

Catalog record

CREATORAditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora, Nitin Kumar, Prathosh A P, Sandeep Kumar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53005
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.24089

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning Aditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora, Nitin Kumar, Prathosh A P, Sandeep Kumar Abstract Bridging molecular structures and natural language is essential for controllable design. Autoregressive models struggle with long-range dependencies, while standard diffusion processes apply uniform corruption across positions, which can distort structurally informative tokens. We present BiMol-Diff , a unified diffusion framework for the paired tasks of text-conditioned molecule generation and molecule captioning. Our key component is a Token-aware noise schedule that assigns position-dependent corruption based on token recovery difficulty, preserving harder-to-recover substructures during the forward process. On ChEBI-20 and M3-20M, BiMol-Diff improves molecule reconstruction with a 15.4% relative gain in Exact Match and achieves strong captioning results, attaining the best BLEU and BERTScore among compared baselines. These results indicate token-aware noising improves fidelity in molecular structure–language modeling. Code link GitHub . 1 Introduction Designing molecules requires balancing two distinct modalities: the molecular structure of atoms and bonds, and the linguistic descriptions of molecular properties (e.g., “a kinase inhibitor with good solubility”). Bridging this gap through cross-modal translation enables chemically grounded workflows where designs can be iteratively refined via natural language 42 ; 12 . This motivates two complementary directions: (i) molecule generation ( text \rightarrow molecule ) to materialize descriptions into structures, and (ii) molecule captioning ( molecule \rightarrow text ) to explain molecular structure and properties via natural-language descriptions 10 ; 39 . However, despite their symmetry, these tasks are typically developed in isolation with separate models and objectives, limiting consistency and reuse in iterative design loops. A common bridge between molecular structure and sequence models is the Simplified Molecular-Input Line-Entry System (SMILES) 37 ; 36 , which linearizes a molecular graph into a string. This linearization has enabled a large family of sequence-based molecular methods, including (1) conditional SMILES generation from text prompts 2 ; 44 ; 18 , and (2) SMILES or structure-conditioned captioning that maps molecules to natural-language descriptions 10 ; 39 . Many of these systems rely on autoregressive (AR) pretrained language models (PLMs) such as GPT 5 , T5 31 , and BART 21 , which generate tokens sequentially by conditioning each next token on the prefix 2 ; 44 ; 18 . In this work, while we retain SMILES as the standard input/output interface, we explicitly project these molecules into a Knowledge Graph (KG) view represented as a serialized sequence of atom-bond-atom triplets to achieve better generalization and reduce dependency on specific SMILES syntax rules (e.g., ring number). While effective, autoregressive models are not always well suited to SMILES-centric molecule–language modeling. First, decoding is inherently left-to-right, so early mistakes can propagate and are difficult to correct 27 ; 1 . Second, SMILES encodes long-range syntactic and chemical dependencies (e.g., ring closures, branching, and valid substructures) that require coordinated decisions across distant positions; locally plausible token choices can therefore cascade into globally invalid molecules. Third, many objectives are inherently structure-level such as scaffold preservation that are hard to enforce under strict prefix conditioning. These limitations motivate non-autoregressive generation mechanisms that can revise all positions jointly, raising a broader question: can we build models that generate and explain molecules while explicitly preserving global validity and structural controllability? Diffusion language models offer a promising non-autoregressive alternative by iteratively denoising a corrupted representation, enabling holistic updates over all positions at each step and better coordination of long-range dependencies 22 ; 27 . However, standard diffusion applies noise uniformly across tokens, which can corrupt chemically critical tokens (e.g., ring indices and branching markers) too early in the process and hinder recoverability. To address this, we propose BiMol-Diff , a unified diffusion framework that addresses the paired tasks of text \rightarrow SMILES generation and SMILES \rightarrow text captioning under a common denoising formulation. Unlike standard diffusion, BiMol-Diff employs a token-aware noising strategy that assigns token-dependent noise levels using the per-token training loss as a proxy for recovery difficulty. Intuitively, tokens that are consistently harder to denoise receive a more conservative corruption schedule, improving recoverability. We apply this mechanism consistently across both directions: for SMILES \rightarrow text , it helps preserve semantic tokens during denoising, and for text \rightarrow SMILES , it helps preserve chemically salient SMILES tokens during generation. Contributions. We make three primary contributions: (1) a token-aware noising strategy which preserves critical structural semantics during the diffusion process; (2) a unified framework that solves the bidirectional tasks of molecule generation and captioning, utilizing the same token-aware mechanism for both inverse problems; and (3) strong empirical performance across a wide range of metrics on both molecule generation and molecule captioning benchmarks. 2 Background and Preliminaries 2.1 Related Work Diffusion for Molecular Graphs (Unconditional / Property-based): Diffusion models are widely used for molecular graph generation, spanning unconditional sampling and property-conditioned design. Recent variants improve expressivity via transformer-style graph denoisers ( 23 ) , explore hierarchical/latent formulations for scalability and structure preservation ( 4 ) , and study control mechanisms for property prediction and guidance ( 40 ) . Text-guided Molecule Generation: Text-to-molecule generation is typically formulated as learning p(\tilde{\mathcal{G}}\mid\mathbf{S}) (or p(\text{SMILES}\mid\mathbf{S}) ), where \mathbf{S} describes desired structure or function. Recent work adapts PLMs to chemical strings, enabling text \leftrightarrow molecule translation with encoder–decoder pretraining ( 10 ) and improved multi-task / chemistry-aware extensions ( 20 ) . In parallel, diffusion-based text conditioning has emerged, including diffusion-LM style text-guided generation in discrete/embedded sequence spaces ( 13 ) and graph/latent diffusion variants that preserve molecular structure ( 6 ) . Molecule Captioning / Molecule-to-Text: Generating faithful descriptions from molecules (G2S) has been explored via translation-style baselines ( 10 ) and molecule captioning models that incorporate structural encoders to improve grounding ( 26 ) . More recent multimodal approaches further strengthen molecule–language alignment for captioning and related tasks. Unifying / Bidirectional Molecule–Text Modeling: While text-guided generation and captioning have progressed independently, fully bidirectional modeling that supports both p(\mathbf{S}\mid\tilde{\mathcal{G}}) and p(\tilde{\mathcal{G}}\mid\mathbf{S}) within a single generation framework remains relatively limited. Translation-based PLM systems offer practical bidirectionality ( 10 ) , and recent foundation-model efforts move toward broader cross-modal generalization ( 26 ) , but diffusion-based approaches are typically developed for one direction (most often S2G) and do not explicitly address token-level corruption heterogeneity. BiMol-Diff fills this gap by casting both directions under a unified conditional diffusion view and introducing a molecular graph-aware, token-dependent noising schedule to preserve chemically salient information during generation (see Appendix A.1 ). 2.2 Preliminaries Figure 1: The BiMol-Diff Framework. (A) Molecules are represented through canonical SMILES, AIS-aware tokens, and serialized graph-triplet sequences. (B) Training: the model is trained with token-aware noising that preserves chemically salient tokens, optimizing denoising, consistency, and rounding objectives. (C) Inference: starting from Gaussian noise, iterative denoising and rounding generate either a caption (G2S) or a serialized molecular graph sequence (S2G), which is then deterministically decoded into canonical SMILES. 2.2.1 Diffusion Models: Forward and Reverse Process Denoising diffusion probabilistic models (DDPMs) are generative models that learn a data distribution, often conditioned on some context \mathbf{c} , p(\mathbf{z}_{0}\mid\mathbf{c}) . They consist of a fixed forward process and a learned reverse process. Forward process : A standard DDPM forward process corrupts clean data \mathbf{z}_{0} through a Markov chain with noise-schedule coefficients \{\alpha_{t}\}_{t=1}^{T} controlling signal decay.

Read the complete text