LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble
Taoyu Wu 1 , Yu Guang Wang 2,3 , Yiqing Shen 3,4,* · 2024
Read LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble by Taoyu Wu 1 , Yu Guang Wang 2,3 , Yiqing Shen 3,4,* (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01737.txt.
Description
A 2024 arXiv research paper by Taoyu Wu 1 , Yu Guang Wang 2,3 , Yiqing Shen 3,4,*. Protein inverse folding aims to identify viable amino acid sequences that can fold into given protein structures, enabling the design of novel proteins with desired functions for applications in drug discovery, enzyme engineering, and biomaterial development. Diffusion probabilistic models have emerged as a promising approach in inverse folding, offering both feasible and diverse solutions compared to traditional energy-based methods and more recent protein language models.
Catalog record
| CREATOR | Taoyu Wu 1 , Yu Guang Wang 2,3 , Yiqing Shen 3,4,* |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble Taoyu Wu 1 , Yu Guang Wang 2,3 , Yiqing Shen 3,4,* Abstract Protein inverse folding aims to identify viable amino acid sequences that can fold into given protein structures, enabling the design of novel proteins with desired functions for applications in drug discovery, enzyme engineering, and biomaterial development. Diffusion probabilistic models have emerged as a promising approach in inverse folding, offering both feasible and diverse solutions compared to traditional energy-based methods and more recent protein language models. However, existing diffusion models for protein inverse folding operate in discrete data spaces, necessitating prior distributions for transition matrices and limiting smooth transitions and gradients inherent to continuous spaces, leading to suboptimal performance. Drawing inspiration from the success of diffusion models in continuous domains, we introduce the Latent Graph Diffusion Model for Protein Inverse Folding (LaGDif). LaGDif bridges discrete and continuous realms through an encoder-decoder architecture, transforming protein graph data distributions into random noise within a continuous latent space. Our model then reconstructs protein sequences by considering spatial configurations, biochemical attributes, and environmental factors of each node. Additionally, we propose a novel inverse folding self-ensemble method that stabilizes prediction results and further enhances performance by aggregating multiple denoised output protein sequence. Empirical results on the CATH dataset demonstrate that LaGDif outperforms existing state-of-the-art techniques, achieving up to 45.55% improvement in sequence recovery rate for single-chain proteins and maintaining an average RMSD of 1.96 Åbetween generated and native structures. These advancements of LaGDif in protein inverse folding have the potential to accelerate the development of novel proteins for therapeutic and industrial applications. The code is public available at https://github.com/TaoyuW/LaGDif . I Introduction Protein inverse folding, by predicting amino acid (AA) sequences that can fold into a given protein structure, is a fundamental challenge in computational biology with far-reaching implications for drug discovery, enzyme engineering and etc [ 1 ] . The ability to design proteins with specific structural features is important for designing novel proteins with desired functions, such as membrane proteins for targeted drug delivery or enzymes for industrial applications [ 2 ] . Recent advancements in deep learning have substantially improved protein structure modeling by automatically learning complex nonlinear relationships from protein data [ 3 ] , demonstrating success in predicting protein structures from sequences [ 4 ] . In parallel, current approaches to protein inverse folding rely on language models that treat the 20 types of AAs as tokens, effectively transforming the task into a sequence generation problem analogous to natural language processing [ 5 , 6 , 7 ] . These language-based models typically fall into two categories, namely the masked models and autoregressive models [ 8 ] . Masked models predict unknown AAs by leveraging contextual information from partially masked sequences, while autoregressive models sequentially predict the probability distribution of each subsequent AA based on the existing sequence [ 9 ] . However, language models are limited to capture the inherent one-to-many mapping between protein structures and their corresponding AA sequences [ 10 ] . The deterministic nature of these models, even when incorporating randomness through techniques like temperature sampling, struggles to fully represent the diversity of possible sequences that can fold into a given structure. Diffusion probabilistic models have emerged as a promising alternative to traditional language-based models for protein sequence generation, offering capabilities in modeling and generating diverse outputs [ 9 ] . These diffusion models generate data through a reverse process from noise to structured data, possessing a natural advantage in handling uncertainty and exploring complex, multimodal output spaces [ 11 ] . This characteristic makes them therefore well-suited for accurately capturing the relationship between protein structures and multiple possible AA sequences. Recent work has demonstrated the potential of diffusion models in protein inverse folding. For instance, GradIF [ 10 ] introduced a discrete diffusion model for protein inverse folding, utilizing the BLOSUM matrix as a transition matrix. However, discrete diffusion models operate in a discrete space, which can lead to less smooth distributions of generated samples compared to continuous space models [ 12 ] . It can affect the quality and diversity of the generated protein sequences. Furthermore, many sampling scheme, such as self-ensemble [ 13 ] , are challenging to apply to discrete diffusion models, potentially limiting the efficiency of the generation process. To address these limitations, we propose a novel approach that constructs a diffusion model in the AA latent space, employing a graph encoder-decoder architecture to transform discrete AAs into a continuous latent representation. The major contributions are four-fold. First, we introduce LaGDif, a latent space diffusion model based on Equivariant Graph Neural Network (EGNN) [ 14 ] , which excels in generating diverse protein sequences while maintaining structural integrity. LaGDif addresses the limitations of discrete diffusion models by operating in a continuous latent space, allowing for smoother distributions of generated samples and improved exploration of the sequence space. Secondly, to enhance the biological relevance of sample distributions in latent space, we leverage the ESM2 pretrained encoder [ 5 ] for AA encoding, incorporating valuable evolutionary information into our model. Additionally, we integrate secondary structure information using the DSSP method, enabling a more comprehensive understanding of protein spatial configuration. Thirdly, we propose a guided sampling approach with controlled noise, which ensures diversity in generated sequences while preserving essential structural integrity. This method facilitates a more efficient and stable denoising process by initializing from a point closer to the target distribution. Finally, we introduce a novel inverse folding self-ensemble method that stabilizes prediction results and further enhances performance, which aggregates multiple denoised output protein sequences at each sampling step, leading to more robust and accurate predictions by mitigating individual biases and errors. Fig. 1: Overview of LaGDif. The upper path illustrates the forward process, where the original protein graph is encoded into a latent space and progressively diffused with Gaussian noise. The lower path shows the reverse process, where a noised graph is iteratively denoised using an equivariant neural network, conditioned on structural information, to generate diverse protein sequences compatible with the given structure. The encoder-decoder architecture, based on ESM2, facilitates the transition between discrete amino acid sequences and the continuous latent space, enabling the application of diffusion models to protein inverse folding. II Methods In this section, we elaborate on LaGDif, a latent graph diffusion model for protein inverse folding, as shown in Fig. 1 . II-A Transition to Continuous Space Given a protein graph G=(V,E) , where V represents the feature matrix of the nodes and E is the adjacency matrix, and X_{pos} denotes the three-dimensional positional information of nodes, the encoding and decoding processes are defined as: \displaystyle z \displaystyle=\text{Encoder}(V,X_{pos},E) (1) \displaystyle\hat{x} \displaystyle=\text{Decoder}(z,X_{pos},E). Our encoder is based on the ESM2 [ 15 ] , a state-of-the-art protein language model pre-trained on a vast corpus of protein sequences. By leveraging ESM2, our encoder benefits from a deep understanding of AA relationships and protein structure, capturing rich semantic information from the context, allowing for a more informative and biologically relevant latent representation. The encoder transforms discrete amino acid sequences into a high-dimensional continuous latent space, preserving crucial structural and biochemical information. The decoder, implemented as a learnable linear layer,