OmniMol : Transferring Particle Physics Knowledge to Molecular Dynamics with Point-Edge Transformers
Ibrahim Elsharkawy, Vinicius Mikuni, Wahid Bhimji, Benjamin Nachman · 2026
Read OmniMol : Transferring Particle Physics Knowledge to Molecular Dynamics with Point-Edge Transformers by Ibrahim Elsharkawy, Vinicius Mikuni, Wahid Bhimji, Benjamin Nachman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10791.txt.
Introduction
Ibrahim Elsharkawy, Vinicius Mikuni, Wahid Bhimji, and Benjamin Nachman present OmniMol, a learned potential for molecules. Such potentials predict forces between atoms so studies can run fast. OmniMol began life far from chemistry. It adapts Omnilearned, a base model for particle jets from physics experiments like the Large Hadron Collider.
Omnilearned uses a Point-Edge-Transformer and was trained first on a billion particle jets. The team retunes that physics model into a potential for small molecules. The transfer tests whether learned patterns of many-body data cross fields. If it holds, chemistry gains a fast potential seeded by physics data at a scale few chemistry sets can match.
Catalog record
| CREATOR | Ibrahim Elsharkawy, Vinicius Mikuni, Wahid Bhimji, Benjamin Nachman |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-10791
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OmniMol : Transferring Particle Physics Knowledge to Molecular Dynamics with Point-Edge Transformers Ibrahim Elsharkawy, Vinicius Mikuni, Wahid Bhimji, Benjamin Nachman Abstract We present OmniMol , a state-of-the-art transformer-based small molecule machine-learned interatomic potential (MLIP). OmniMol is built by adapting Omnilearned , a foundation model for particle jets found in high-energy physics (HEP) experiments such as at the Large Hadron Collider (LHC). Omnilearned is built with a Point–Edge–Transformer (PET) and pre-trained using a diverse set of one billion particle jets. It includes an interaction-matrix attention bias that injects pairwise sub-nuclear (HEP) or atomic (molecular-dynamics) physics directly into the transformer’s attention logits, steering the network toward physically meaningful neighborhoods without sacrificing expressivity. We demonstrate OmniMol using the oMol dataset and find excellent performance even with relatively few examples for fine-tuning. This study lays the foundation for building interdisciplinary connections given datasets represented as collections of point clouds. I Introduction Machine learning (ML) has transformed how we represent, simulate, and analyze complex physical systems. Two domains where this transformation is particularly visible are particle physics 35 ; 21 and molecular chemistry 41 ; 4 (alongside related topics like molecular structure 19 ). The former pursues the fundamental structure of matter while the latter is the basis for materials design and a number of other highly impactful applications. Although different in energy scale, both subjects are governed by quantum theories and share a common data structure: (variably-sized) sets of particles in (phase) space. This is a type of structured point cloud . Our hypothesis is that a foundation model built for point clouds in particle physics will be useful for ML tasks operating on point clouds in computational chemistry. By construction, a foundation model for particle physics should improve diverse ML tasks across particle physics, but it is not known if transferability extends to completely different scientific disciplines. To test the hypothesis, we adapt the recent Omnilearned particle physics foundation model 33 ; 32 ; 6 to molecular dynamics (MD). Omnilearned is trained to classify and generate particle jets, which are a ubiquitous and central component of high energy reactions. Physically, these jets emerge from quantum processes related to the radiation from energetic quarks and gluons. Due to their prevalence and complexity, jets have been driving ML developments in particle physics 23 ; 7 ; 13 . We fine-tune Omnilearned on MD simulations to create OmniMol . Along the way, we explore multiple approaches to fine tuning the foundation model. Our goal is to develop a machine-learned interatomic potential (MLIP) 37 ; 44 ; 3 ; 36 ; 15 ; 2 ; 1 ; 25 ; 34 ; 14 . MLIPs aim to approximate potential energy surfaces and their gradients (i.e. forces) at a fraction of the computational cost of traditional methods, such as density functional theory (DFT) 28 . The goal is to enable large-scale and long-horizon MD. Given atomic types and coordinates \{\vec{r}\in\mathbb{R}^{3}\} , an MLIP predicts \mathbf{E}(\{\vec{r}_{i}\}) , the energy of a given configuration of atoms and \vec{F}_{i}(\{\vec{r}_{i}\}) , the forces on each atom due to the rest. These predictions can be used to ‘rollout’ the motion of a molecule over time. Accuracy is necessary but not sufficient. MD utility depends on stability, energy conservation (enforced or approximate), and consistency across molecular compositions. OmniMol is trained on OMoL25 24 , a large-scale molecular dataset, which provides atom positions in \mathbb{R}^{3} as inputs and \mathbf{E}(\{\vec{r}_{i}\}) and \vec{F}_{i}(\{\vec{r}_{i}\}) in \mathbb{R}^{3} as labels. In this paper, we develop a novel point-edge-transformer MLIP architecture derived from particle physics, containing an interaction-matrix attention bias for chemically informed attention, along with equivariant and conservation constraints. Using this architecture, we achieve state-of-the-art, transformer-based small molecule MLIPs. This is the first demonstration of cross-discipline transfer for scientific point-cloud foundation models. While industrial foundation models for natural language and images have found widespread use, there are fewer examples of cross-domain models specifically for scientific data. A number of recent works have explored multiphysics foundation models for image representations of dynamical systems 30 ; 16 ; 29 ; 26 ; 42 . Within the discipline of fundamental physics, the OmniCosmos model 31 has a similar setup to OmniMol , transferring particle physics point cloud pre-training to cosmological point clouds. This paper is organized as follows. Section Ref. II briefly reviews the Omnilearned model and training protocol. Next, Sec. III introduces OmniMol . Equivariance and energy conservation are discussed in Sec. IV . Model training is detailed in Sec. V . Results are presented in Sec. VI and the paper ends with conclusions and outlook in Sec. VII . II OmniLearn ( ed ) Backbone and heads. OmniLearned is composed of a general core called the the point-edge transformer (PET) body and a number of task-specific heads. The model couples local attention over k -nearest neighbors with physics-inspired pairwise features and global transformer blocks whose attention matrices are additively biased by the same interaction terms. Four learnable tokens summarize jet information. Two task heads share the PET body: a classifier and a generator trained with diffusion/flow-matching style objectives 5 . Inputs Each jet is represented as an unordered variable-sized set of its constituent particles. Each particle is specified by 1) kinematic information relative to the overall jet momentum, 2) discrete Particle Identification Numbers (PIDs) that indicate the type of particle (such as proton or neutron) and 3) additional per particle information that is not always available, such as the reconstructed origin of the particle in the detector. Embeddings Each jet constituent is embedded into a token space given by: \begin{split}\vec{x}_{embed}=\vec{x}_{embed}^{kinematic}+\vec{x}_{embed}^{PID}+\\ +\vec{x}_{embed}^{time}+\vec{x}_{embed}^{add}+\vec{x}_{embed}^{local}\,.\end{split} (1) Kinematic information is embedded to \vec{x}_{embed}^{kinematic} via a shared multilayer perceptron (MLP), and PID information \vec{x}_{embed}^{PID} is embedded via a learned lookup table. For generation, the diffusion/flow time variable is embedded via an MLP and injected through \vec{x}_{embed}^{time} . Finally, \vec{x}_{embed}^{add} is found by embedding the additional information via another MLP. Local attention. Figure 1: Local Embedding Block, where K-Nearest Neighbors for either Molecule or Jet point-clouds are computed along with pairwise features, these are fed to an MLP, and a small local transformer resulting in \vec{x}_{embed}^{local} The last component of the embedding is \vec{x}_{embed}^{local} and given by a local attention block. For each particle, a local neighborhood is built out of the K-nearest neighbors (with K {=}10 ) using a Euclidean L2 notion of distance. Omnilearned then builds a set of pairwise features f(x_{i},x_{j}) followed by an MLP that transforms each of the K 2 features f(x_{i},x_{j}) before being fed into a small local transformer block. This local block aggregates all pairwise information of the K-nearest neighbors for each jet constituent into a local embedding vector \vec{x}_{embed}^{local} . A diagram illustrating this block is found in Figure 1 . Global attention and Bias After \vec{x}_{embed} is computed for each jet constituent, the unordered set of \vec{x}_{embed} is fed into large global all-to-all transformer blocks. These global self-attention components add an explicit bias to the attention logits found by computing f(x_{i},x_{j}) for all pairs of particles, and embed via an MLP into f(x_{i},x_{j})\rightarrow B_{ij} , such that the logits are set to A_{ij}^{*}\;=\;A_{ij}\;+\,B_{ij} for B_{ij}\in\mathbb{R} . Output Heads OmniLearned is built with two output heads attached to the transformer body, a classifier head, and a generator head. The generator head