Zero-Shot Learning with Subsequence Reordering Pretraining for Compound-Protein Interaction
Hongzhi Zhang, Zhonglie Liu, Kun Meng, Jiameng Chen, Jia Wu, Bo Du, Di Lin, Yan Che, Wenbin Hu · 2025
Read Zero-Shot Learning with Subsequence Reordering Pretraining for Compound-Protein Interaction by Hongzhi Zhang, Zhonglie Liu, Kun Meng, Jiameng Chen, Jia Wu, Bo Du, Di Lin, Yan Che, Wenbin Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20925.txt.
Introduction
Zhang and co-authors propose in 2025 a way to predict whether a new drug compound will bind a wholly new protein, a zero shot task in drug hunting. Chemical space is vast and fresh proteins keep surfacing, so tools must handle unseen pairs. Past methods read whole protein chains or chunks alone, missing how chunks depend on each other.
Their pretraining shuffles protein chunks and asks the model to restore the order, teaching those links. A length varying boost lets small data sets still train well. The learned codes then feed binding forecasts for new pairs. Tests probe both accuracy and the zero shot claim.
Catalog record
| CREATOR | Hongzhi Zhang, Zhonglie Liu, Kun Meng, Jiameng Chen, Jia Wu, Bo Du, Di Lin, Yan Che, Wenbin Hu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20925
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Zero-Shot Learning with Subsequence Reordering Pretraining for Compound-Protein Interaction Hongzhi Zhang, Zhonglie Liu, Kun Meng, Jiameng Chen, Jia Wu, Bo Du, Di Lin, Yan Che, Wenbin Hu Abstract Given the vastness of chemical space and the ongoing emergence of previously uncharacterized proteins, zero-shot compound-protein interaction (CPI) prediction better reflects the practical challenges and requirements of real-world drug development. Although existing methods perform adequately during certain CPI tasks, they still face the following challenges: (1) Representation learning from local or complete protein sequences often overlooks the complex interdependencies between subsequences, which are essential for predicting spatial structures and binding properties. (2) Dependence on large-scale or scarce multimodal protein datasets demands significant training data and computational resources, limiting scalability and efficiency. To address these challenges, we propose a novel approach that pretrains protein representations for CPI prediction tasks using subsequence reordering, explicitly capturing the dependencies between protein subsequences. Furthermore, we apply length-variable protein augmentation to ensure excellent pretraining performance on small training datasets. To evaluate the model’s effectiveness and zero-shot learning ability, we combine it with various baseline methods. The results demonstrate that our approach can improve the baseline model’s performance on the CPI task, especially in the challenging zero-shot scenario. Compared to existing pre-training models, our model demonstrates superior performance, particularly in data-scarce scenarios where training samples are limited. Our implementation is available at https://github.com/Hoch-Zhang/DrugDiscovery-DTI/ . 1 Introduction Figure 1: (a) Residue-level interaction diagram of the EJ4-4N6H protein-ligand complex. (b) Our method learns protein representations by capturing interdependencies among protein subsequences. Compound–protein interaction (CPI) prediction [ 1 , 2 , 3 , 4 ] offers a promising solution to the inefficiencies of traditional drug discovery, particularly in reducing time and cost [ 5 ] . In CPI prediction tasks, evaluation scenarios are typically classified into four categories: Seen-Both, Unseen-Compound, Unseen-Protein, and Unseen-Both based on whether the compounds and proteins involved have appeared in training data. Among these, apart from the Seen-Both scenario, the other three scenarios fall under the category of zero-shot learning. Considering the immense chemical space and the continuous emergence of novel proteins, zero-shot CPI prediction scenarios better reflect the real-world requirements faced in drug discovery. In the interaction between a compound and a protein, as shown in Fig. 1 (a), the amino acid residues involved in binding are often not sequentially adjacent, but are brought into close spatial proximity through the folding of the protein’s three-dimensional structure, thereby forming a binding pocket or active site. Therefore, effectively modeling CPIs requires capturing the relationships between protein subsequences that may cooperate in binding. This poses a significant challenge for traditional sequence-based models, which often struggle to model for such long-range and subsequence dependencies. Supervised learning techniques, including regression [ 6 , 7 ] and classification [ 8 , 9 ] models, have been extensively studied and have demonstrated adequate performance in predicting Seen-Both CPIs, where both compounds and proteins have been observed during training. However, supervised learning methods that utilize an end-to-end CPI framework for protein representation struggle to effectively capture the intricate interdependencies among protein subsequences, which are crucial for predicting spatial structures and binding properties. Additionally, the predictive performance of these methods heavily relies on a substantial amount of high-quality labeled data. All of the above make it challenging to generalize to zero-shot learning. Self-supervised learning (SSL) techniques have recently demonstrated excellent performance in zero-shot learning tasks [ 10 , 11 , 12 , 13 ] . In the context of CPI prediction, SSL-based approaches typically aim to enhance protein representation learning through two dominant paradigms: cross-modal contrastive learning and mask-based sequence reconstruction. Despite their success, these methods demonstrate significant limitations in the inadequate modeling of non-local subsequence dependencies. Contrastive learning methods focus on aligning identical protein subsequences across modalities (e.g., sequence-structure pairs) [ 4 ] , yet they inherently fail to capture complex interdependencies between distinct subsequences within the same protein. This limitation arises from their reliance on modality alignment objectives, which prioritize global feature consistency over local interaction patterns. Consequently, critical spatial or functional relationships between non-adjacent residues (e.g., discontinuous binding motifs) remain under-represented. Mask-based methods, particularly protein language models (PLMs) [ 14 , 15 , 16 ] , address local context learning by reconstructing masked amino acids from their neighbors [ 17 ] . These approaches predominantly model adjacent or proximal subsequences, neglecting long-range dependencies essential for holistic structural and functional representation. In addition, SSL-based methods often suffer from a heavy reliance on specialized or task-specific data. For instance, contrastive learning frameworks rely on high-quality multimodal protein data, such as sequence–structure pairs, to capture cross-modal correlations. However, such data are scarce, especially for understudied proteins, limiting their applicability in zero-shot scenarios. While mask-based methods are less dependent on multimodal data, they still require large-scale pretraining on massive sequence corpora (e.g., UniRef50/90) to learn robust and generalizable protein representations [ 14 ] . To address these challenges, we propose a P rotein encoder via S ubsequence R eordering P retraining method, specifically designed for CPI prediction (PSRP-CPI) . First, as shown in Fig. 1 (b), PSRP-CPI pretrains the protein encoder via subsequence reordering, explicitly capturing interdependencies among protein subsequences for protein representation learning. Then, it fine-tunes the encoder according to benchmark methods to adapt to the CPI task more effectively. In the pretraining phase, we use a multi-layer Transformer [ 18 ] as the protein encoder to effectively learn relationships between distant subsequences. In addition, the reordering task requires the model to predict the correct order of shuffled protein subsequences, forcing it to understand and model their structural and functional interdependencies. Second, we apply length-variable protein augmentation to improve protein subsequence reordering method’s pretraining performance even on small training datasets and enhance the model’s zero-shot learning ability. Finally, to evaluate the model’s validity and zero-shot learning ability, we divided the test data into four groups based on whether compounds and proteins have been observed during training. The experimental results demonstrate that PSRP-CPI significantly enhances the performance of baseline methods in zero-shot scenarios, highlighting our approach’s effectiveness in enhancing model generalization. Meanwhile, our approach exhibits enhanced efficacy relative to conventional pre-training methods, especially when operating with constrained training datasets that lack sufficient sample diversity. The proposed method serves as a powerful tool for CPI tasks, with broad applicability in drug discovery. The main contributions of this work are as follows: 1. We propose a pretraining method based on protein subsequence reordering, termed PSRP-CPI, which improves performance on the CPI prediction task by explicitly modeling the relationships between protein subsequences. 2. We develop a length-variable protein augmentation strategy that enables the model to achieve robust pre-training performance, even when trained on a small-scale dataset. 3. We conduct comprehensive experiments on four widely used CPI benchmark datasets to demonstrate the strong performance of PSRP-CPI in zero-shot CPI prediction. Furthermore, we compared PSRP-CPI with existing pretraining methods on small-scale datasets to evaluate its effectiveness in low-resource settings. 2 Related Work Compound-Protein Interaction Prediction. Clinical and laboratory studies focus on CPI prediction, primarily through molecular docking [ 19 , 20 ] and representation learning methods [ 1 , 7 , 21 ] . Molecular docking predicts protein-ligand binding structures, using energy functions and geometric deep learning to rank ligand interactions. However,