UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials

Gongbo Zhang, Yanting Li, Renqian Luo, Pipi Hu, Yang Yang, Zeru Zhao, Lingbo Li, Guoqing Liu, Zun Wang, Ran Bi, Kaiyuan Gao, Liya Guo, Yu Xie, Chang Liu, Jia Zhang, Tian Xie, Robert Pinsler, Claudio Zeni, Ziheng Lu, Hongxia Hao, Yingce Xia, Marwin Segler, Maik Riechert, Wei Yang, Hao Jiang, Wen-Bin · 2025

Read UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials by Gongbo Zhang, Yanting Li, Renqian Luo, Pipi Hu, Yang Yang, Zeru Zhao, Lingbo Li, Guoqing Liu, Zun Wang, Ran Bi, Kaiyuan Gao, Liya Guo, Yu Xie, Chang Liu, Jia Zhang, Tian Xie, Robert Pinsler, Claudio Zeni, Ziheng Lu, Hongxia Hao, Yingce Xia, Marwin Segler, Maik Riechert, Wei Yang, Hao Jiang, Wen-Bin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06687.txt.

Introduction

Gongbo Zhang and a large team present a foundation model that designs proteins, molecules, and materials in one system, in this 2025 paper. In nature, a one-dimensional sequence folds into a three-dimensional structure, and the structure sets the function. Most generative models design for shape alone, leaving function as an afterthought. They also split their effort between letter-like sequences and continuous coordinates, optimizing the two apart.

Their model, UniGenX, couples sequence, structure, and function in a single generative frame. Training aims directly at function, not just at looking like past examples. One engine for three domains could transfer tricks learned on proteins to materials and back. The authors pitch it as a general design accelerator for the molecular sciences.

Catalog record

CREATORGongbo Zhang, Yanting Li, Renqian Luo, Pipi Hu, Yang Yang, Zeru Zhao, Lingbo Li, Guoqing Liu, Zun Wang, Ran Bi, Kaiyuan Gao, Liya Guo, Yu Xie, Chang Liu, Jia Zhang, Tian Xie, Robert Pinsler, Claudio Zeni, Ziheng Lu, Hongxia Hao, Yingce Xia, Marwin Segler, Maik Riechert, Wei Yang, Hao Jiang, Wen-Bin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-06687

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials Gongbo Zhang, Yanting Li, Renqian Luo, Pipi Hu, Yang Yang, Zeru Zhao, Lingbo Li, Guoqing Liu, Zun Wang, Ran Bi, Kaiyuan Gao, Liya Guo, Yu Xie, Chang Liu, Jia Zhang, Tian Xie, Robert Pinsler, Claudio Zeni, Ziheng Lu, Hongxia Hao, Yingce Xia, Marwin Segler, Maik Riechert, Wei Yang, Hao Jiang, Wen-Bin Zhang, Zhijun Zeng, Yi Zhu, Li Dong, Xiuyuan Hu, Li Yuan, Lei Chen, Haiguang Liu, Tao Qin Abstract Function in natural systems arises from one-dimensional sequences forming three-dimensional structures with specific properties. However, current generative models suffer from critical limitations: training objectives seldom target function directly, discrete sequences and continuous coordinates are optimized in isolation, and conformational ensembles are under-modeled. We present UniGenX, a unified generative foundation model that addresses these gaps by co-generating sequences and coordinates under direct functional and property objectives across proteins, molecules, and materials. UniGenX represents heterogeneous inputs as a mixed stream of symbolic and numeric tokens, where a decoder-only autoregressive transformer provides global context and a conditional diffusion head generates numeric fields steered by task-specific tokens. Besides the new high SOTAs on structure prediction tasks, the model demonstrates state-of-the-art or competitive performance for the function-aware generation across domains: in materials, it achieves “conflicted” multi-property conditional generation, yielding 436 crystal candidates meeting triple constraints, including 11 with novel compositions and 4 confirmed thermodynamically stable by DFT; in chemistry, it sets new benchmarks on five property targets and conformer ensemble generation on GEOM; and in biology, it improves success in modeling protein induced fit (RMSD < < 2 Å) by over 23-fold and enhances EC-conditioned enzyme design. Ablation studies and cross-domain transfer substantiate the benefits of joint discrete–continuous training, establishing UniGenX as a significant advance from prediction to controllable, function-aware generation. Keywords— Unified Generation, Autoregressive Diffusion Model, Next-token Prediction, AI for Science 1 Introduction The ability of one-dimensional sequences to form three-dimensional structures with specific functions is a fundamental principle in natural systems. Replicating this generative process is a central goal for artificial intelligence in scientific discovery, promising to unlock transformative applications in materials design, drug discovery, and protein engineering [ 1 , 2 ] . The intricate interplay between a system’s discrete sequence (e.g., chemical formula, amino acid chain) and its continuous coordinates (e.g., atomic positions, protein fold) dictates its essential properties and functions. For instance, a crystal’s atomic arrangement governs its electronic characteristics [ 3 ] , a molecule’s spatial conformation determines its pharmacological efficacy [ 4 ] , and a protein’s 3D shape enables its biological role [ 5 ] . Given this sensitivity, where even minute deviations in atomic coordinates can nullify function, the challenge is not just to predict structures, but to generate them with high numerical precision under desired functional constraints. However, current generative models face critical limitations that hinder this goal. First, most training objectives target structural accuracy in isolation, rather than the ultimate function or property of interest. Second, the generation of discrete sequences and continuous coordinates are often treated as separate tasks, failing to capture their co-dependent relationship. Finally, models typically generate a single static structure, under-modeling the dynamic conformational ensembles that are crucial for the function of many molecular and biological systems. These gaps have created a significant bottleneck, preventing a unified approach to controllable, function-aware generation across scientific domains. The current modeling landscape is dominated by two powerful yet incomplete paradigms. On one hand, autoregressive language models (LLMs) excel at processing and generating discrete token sequences, demonstrating remarkable flexibility, scalability, and long-context reasoning [ 6 , 7 ] . However, their token-based nature struggles to enforce the strict numerical precision required for generating physically plausible 3D atomic coordinates. On the other hand, diffusion models have achieved state-of-the-art performance in generating high-fidelity continuous data, such as images and 3D structures [ 8 , 9 , 10 ] . Yet, they are less adept at handling discrete sequence information and lack the flexible, token-by-token generative framework of autoregressive models, making the joint generation of sequence and structure challenging. To address these shortcomings, we introduce UniGenX, a unified generative foundation model that co-generates sequences and coordinates under direct functional and property objectives. UniGenX bridges the gap between discrete and continuous generation by integrating an autoregressive transformer with a conditional diffusion head. In our framework, heterogeneous inputs are represented as a mixed stream of symbolic and numeric tokens. A decoder-only transformer processes this stream to capture global context, while the conditional diffusion head generates precise numeric fields (i.e., coordinates) steered by task-specific functional tokens. This hybrid architecture synergistically combines the contextual power and flexibility of autoregressive models with the numerical precision of diffusion models, enabling a single framework to operate across proteins, molecules, and materials. UniGenX marks a significant advance from prediction to controllable, function-aware generation. Beyond setting new state-of-the-art benchmarks on structure prediction tasks, our model demonstrates unparalleled capabilities in conditional generation across domains. In materials science, it successfully navigates “conflicted” multi-property objectives to design 436 novel crystal candidates, including 11 with new compositions and 4 confirmed as thermodynamically stable by DFT calculations. In chemistry, it establishes new performance records on five property-targeting tasks and conformer ensemble generation. For biology, it improves the success rate of modeling protein induced fit (RMSD < 2 Å) by over 23-fold and enhances enzyme design conditioned on Enzyme Commission (EC) numbers. Through extensive ablation studies and cross-domain transfer experiments, we substantiate the benefits of our joint discrete–continuous training strategy, establishing UniGenX as a versatile and powerful foundation for AI-driven scientific discovery. 2 Methods UniGenX leverages the strengths of both autoregressive (AR) and diffusion models, achieving a natural unification. The diffusion head addresses the numerical precision limitations of general AR models by operating in a continuous vector space for numerical data. Conversely, the AR next-token prediction provides effective conditioning for the diffusion head, which diffuses only a low-dimensional variable (e.g., three dimensions for molecular 3D structures), significantly simplifying the diffusion training process. Our design seamlessly integrates symbolic (words) and numerical (numbers) data across scientific domains without sacrificing scientific precision. This section details the key features of the UniGenX architecture. 2.1 Sequentialize All for Multi-Domain/Task Compatibility Figure 1 : Sequentialization approach combines symbolic (words) and numerical data (numbers), separated by special tokens that identify the category of each data element. As illustrated in the figure for small molecules and materials, SMILES strings or chemical formulas are treated as words and delimited by special tokens such as < bos > (beginning of SMILES) or < bof > (beginning of formula). Corresponding coordinates are treated as numbers, delimited by tokens such as < boc > (beginning of coordinates). This method provides a general framework applicable to diverse scientific data formats. The success of multimodal GPT-like models in integrating language and images demonstrates the potential of sequence alignment for addressing diverse problems across various domains. To achieve alignment with language—a naturally sequential modality—images are encoded into a latent space using a pretrained encoder and subsequently aligned with language embeddings within this shared latent space. This process enables general comprehension and facilitates generation in both language and image modalities. This approach highlights the flexibility of sequentialization across

Read the complete text