QE-Catalytic: A Graph–Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption

Yanjie Li, Jian Xu, Xueqing Chen, Lina Yu, Shiming Xiang, Weijun Li, Cheng-lin Liu · 2025

Read QE-Catalytic: A Graph–Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption by Yanjie Li, Jian Xu, Xueqing Chen, Lina Yu, Shiming Xiang, Weijun Li, Cheng-lin Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20084.txt.

Introduction

Yanjie Li leads a seven-author team fusing a language model with a geometry-native network. The task is to predict how molecules stick to catalysts. Adsorption energy is the key yardstick for screening catalysts. It is the gap between a relaxed molecule-surface system and its reference state. Errors there poison all that follows. Graph networks handle 3D coordinates superbly. Language models give readable text, but blur unlike setups of the same system.

Their model, QE-Catalytic, couples a Qwen language model with an equivariant graph transformer. Graph and text channels are aligned, so 3D facts flow into the text side. The model predicts relaxed adsorption energies with much lower error than past text-based tools. It can also write crystal structure files. That serves target-driven inverse design of new catalysts.

Catalog record

CREATORYanjie Li, Jian Xu, Xueqing Chen, Lina Yu, Shiming Xiang, Weijun Li, Cheng-lin Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20084

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

QE-Catalytic: A Graph–Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption Yanjie Li, Jian Xu, Xueqing Chen, Lina Yu, Shiming Xiang, Weijun Li, Cheng-lin Liu Abstract Adsorption energy is a key descriptor of catalytic reactivity. It is fundamentally defined as the difference between the relaxed total energy of the adsorbate–surface system and that of an appropriate reference state; therefore, the accuracy of relaxed-energy prediction directly determines the reliability of machine-learning-driven catalyst screening. E(3)-equivariant graph neural networks (GNNs) can natively operate on three-dimensional atomic coordinates under periodic boundary conditions and have demonstrated strong performance on such tasks. In contrast, language-model-based approaches, while enabling human-readable textual descriptions and reducing reliance on explicit graph—thereby broadening applicability—remain insufficient in both adsorption-configuration energy prediction accuracy and in distinguishing “the same system with different configurations,” even with graph-assisted pretraining in the style of GAP-CATBERTa. To this end, we propose QE-Catalytic, a multimodal framework that deeply couples a large language model ( Q wen) with an E(3)-equivariant graph Transformer ( E quiformer-V2), enabling unified support for adsorption-configuration property prediction and inverse design on complex catalytic surfaces. During prediction, QE-Catalytic jointly leverages three-dimensional structures and structured configuration text, and injects “3D geometric information” into the language channel via graph–text alignment, allowing it to function as a high-performance text-based predictor when precise coordinates are unavailable, while also autoregressively generating CIF files for target-energy-driven structure design and information completion. On OC20, QE-Catalytic reduces the MAE of relaxed adsorption energy from 0.713 eV to 0.486 eV, and consistently outperforms baseline models such as CatBERTa and GAP-CATBERTa across multiple evaluation protocols. keywords Multimodal Large Language Models, Catalytic materials; Property Prediction, Relaxed Energy Prediction, Adsorption Energy Prediction 1 Introduction In heterogeneous catalysis, adsorption energy is a key reactivity descriptor for catalyst screening. It essentially corresponds to the minimum-energy state attained by a given adsorbate–catalyst system across a diverse set of adsorption configurations that differ in adsorption sites, orientations, and related geometric degrees of freedom. Because the differences among these configurations are often subtle and their energies can be highly close—exhibiting near-degeneracy on the order of 0.1–0.3 eV around the global minimum—a model must not only predict the energy of an individual configuration, but also reliably discriminate near-degenerate configurations and identify the global minimum to ensure trustworthy adsorption-energy determination. However, achieving this objective within the DFT framework typically requires large-scale configuration enumeration and geometric relaxation, leading to prohibitive computational cost. Consequently, developing machine-learning models that can serve as efficient surrogates for DFT, accurately predicting relaxed energies across adsorption configurations and subsequently determining the minimum-energy state, has become a central direction for data-driven catalyst screening. In recent years, E(3)-equivariant graph neural networks (graph neural networks, GNNs), such as the Equiformer family, have achieved remarkable performance in property prediction for atomistic systems. These models act directly on three-dimensional atomic coordinates under periodic boundary conditions and learn geometric information—including local coordination environments, bond lengths, and bond angles—without explicitly constructing hand-crafted features, reaching state-of-the-art performance for energy and force prediction on large-scale catalysis adsorption benchmarks such as OC20 5 . However, this advantage also implies a strong dependence on high-quality three-dimensional structures: on the one hand, both training and inference require accurate or approximately relaxed atomic coordinates; on the other hand, it is difficult to naturally integrate extensive prior knowledge that exists in textual form—such as composition, Miller indices, and even experimental conditions—into the learned representations. In practical catalysis research, experiments and the literature typically provide “symbolic descriptions” (e.g., chemical formulas, facets, and adsorption-site types) rather than detailed three-dimensional coordinates, which limits the applicability of equivariant GNNs when high-quality geometric information is unavailable. In contrast, Transformer- or BERT-based language models 35 ; 17 are inherently well-suited to processing human-readable string inputs. With appropriately designed text templates that encode information such as “adsorbate + catalyst composition + Miller indices + configuration description” into structured sequences, language models can learn statistical associations between adsorption energies and structure–property relationships directly in text space, without explicit three-dimensional coordinates or topology. Such approaches have shown initial promise in computational catalysis: they readily incorporate discrete information from the literature, databases, and experimental records, and provide a unified interface for diverse downstream tasks. However, because many geometrically distinct adsorption configurations can correspond to similar or even identical textual descriptions, text-only models struggle to precisely discriminate subtle differences in “the same system with different configurations.” Moreover, their energy-prediction accuracy remains limited. To mitigate this issue, works such as GAP-CATBERTa introduced the idea of graph-assisted pretraining : first extracting representations from three-dimensional atomic coordinates using mature graph neural networks, and then “compressing” them into the corresponding text features through self-supervised alignment, thereby indirectly injecting geometric information during training. This strategy improves, to some extent, the sensitivity of text models to configuration differences, enabling better adsorption-energy prediction performance under text-only inputs. Nevertheless, such methods still rely on a single text modality at inference: even when three-dimensional structural information is available, it cannot explicitly participate in prediction; furthermore, when different configurations share similar textual descriptions, the model remains unable to fully exploit fine-grained geometric differences to distinguish near-degenerate configurations. The above observations expose a more fundamental issue: in complex catalytic systems, graph and text are not fully substitutable modalities. Three-dimensional atomic coordinates carry local structural information strictly constrained by E(3) symmetry, whereas textual descriptions are better suited to capturing high-level semantics such as composition, facets, defects, and processing conditions. If one relies on a single modality—either “graph-only” or “text-only”—some information crucial for accurate adsorption-energy prediction is inevitably lost. This naturally raises the following question: can we build a truly multimodal model that simultaneously perceives three-dimensional graphs and structured text within a unified framework, maintains strong predictive performance even when the graph is missing, and can even perform inverse design by generating CIF files to complement missing information? To address the above limitations, we propose QE-Catalytic: a multimodal framework that deeply couples an E(3) -equivariant graph Transformer with a large language model, enabling unified support for adsorption-configuration property prediction and inverse design on complex catalytic surfaces. Architecturally, QE-Catalytic adopts Equiformer-V2 as the geometric encoder to directly process atomistic structures under periodic boundary conditions, and uses structured configuration text—including the adsorbate, catalyst composition, and Miller indices, as well as primary/secondary neighbor configuration cues—as the language input. Through graph–text alignment training, the model constructs a shared graph–semantic latent space within a unified backbone, allowing geometric inductive bias to be injected into the language channel. Consequently, when three-dimensional coordinates are available, QE-Catalytic fully exploits geometric details; when precise coordinates are absent, it degrades to a high-performance text-based predictor. Moreover, leveraging the autoregressive generation capability of large language models, QE-Catalytic can automatically generate complete CIF files to complement missing configuration text information, and can even produce structural candidates such as CIFs conditioned on a target energy or a high-level system description, thereby forming an inverse structure-design pipeline oriented toward target adsorption energies. Overall, QE-Catalytic demonstrates a viable pathway for jointly leveraging three-dimensional graph and language representations in computational catalysis, providing methodological foundations for multimodal catalytic foundation models that achieve both high-accuracy property prediction and inverse-design capability. The contributions of this paper are as follows: • Multimodal architecture: We propose a multimodal large language model architecture for catalytic materials property prediction, QE-Catalytic , which

Read the complete text