DeepFRI Demystified: Interpretability vs. Accuracy in AI Protein Function Prediction
Ananya Krishna, Valentina Simon, Arjan Kohli · 2025
Read DeepFRI Demystified: Interpretability vs. Accuracy in AI Protein Function Prediction by Ananya Krishna, Valentina Simon, Arjan Kohli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00642.txt.
Introduction
Protein function prediction tools are powerful black boxes, and their opacity slows adoption in drug discovery. Ananya Krishna, Valentina Simon, and Arjan Kohli demystify DeepFRI in 2025, weighing interpretability against accuracy. DeepFRI is a leading graph convolutional network tool for predicting protein function.
The authors evaluate it with explainability techniques including GradCAM, excitation backpropagation, and PGExplainer, plus adversarial robustness tests. Adoption of such methods depends on verifying their findings before targets are chosen. The study shows what explanation tools can and cannot certify about a model that proposes drug targets. Explainability techniques disagree with each other often enough that using several is prudent. Adversarial tests check whether the explanations survive deliberate attempts to fool them. For drug teams, the gain is a check they can run at low cost. If a tool's hot spots jump at small tricks, its claims should be read with care.
Catalog record
| CREATOR | Ananya Krishna, Valentina Simon, Arjan Kohli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00642
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DeepFRI Demystified: Interpretability vs. Accuracy in AI Protein Function Prediction Ananya Krishna, Valentina Simon, Arjan Kohli Abstract Machine learning technologies for protein function prediction are black box models. Despite their potential to identify key drug targets with high accuracy and accelerate therapy development, the adoption of these methods depends on verifying their findings. This study evaluates DeepFRI, a leading Graph Convolutional Network (GCN)-based tool, using advanced explainability techniques—GradCAM, Excitation Backpropagation, and PGExplainer—and adversarial robustness tests. Our findings reveal that the model’s predictions often prioritize conserved motifs over truly deterministic residues, complicating the identification of functional sites. Quantitative analyses show that explainability methods differ significantly in granularity, with GradCAM providing broad relevance and PGExplainer pinpointing specific active sites. These results highlight trade-offs between accuracy and interpretability, suggesting areas for improvement in DeepFRI’s architecture to enhance its trustworthiness in drug discovery and regulatory settings. 1 Introduction Explainability in artificial intelligence (XAI) is pivotal for adopting and effectively using AI in healthcare and biotechnology [ 1 ] . A Deloitte survey found that 75% of professionals in these fields consider explainability essential for AI adoption [ 3 ] . Additionally, about 90% of biopharmaceutical companies require explainable AI to comply with regulatory standards from agencies like the FDA and EMA, which increasingly mandate transparent AI models for drug development and approval [ 6 ] . Despite breakthroughs in protein structure prediction—most notably AlphaFold—functional annotation tools still lag in both performance and trustworthiness. Of existing methods, DeepFRI [ 4 ] is the most comprehensive: it uses graph convolutional networks trained on PDB-derived structural graphs to assign Gene Ontology (GO) terms. Alternative approaches such as ContactPFP [ 7 ] , which relies on residue–residue contact maps, and 3D CNNs [ 12 ] , which identify functional pockets, often miss crucial topological context. Homology-independent tools like FFPred 3 [ 2 ] (SVM-based) and multimodal frameworks like DeepGO [ 8 ] offer complementary strengths but still lack end-to-end interpretability. Although DeepFRI achieves state-of-the-art performance, its explainability and robustness beyond preliminary GradCAM analyses remain unexplored. Here, we apply a suite of post hoc explainability techniques to DeepFRI to pinpoint the most influential amino acid residues driving its predictions, uncover potential vulnerabilities, and enhance the transparency of protein function models [ 11 ] . 2 Methods We evaluate the DeepFRI model [ 4 ] , which predicts a protein’s function from its structure. Our analysis identifies which structural motifs are most influential to the model’s predictions across protein test cases. The creators of DeepFRI utilized GradCAM [ 11 ] for a qualititative understanding of explainability. We build upon this by reproducing their GradCAM results, corroborating these findings with other post-hoc explainability techniques (Excitation Backpropagation and PG Explainer [ 11 ] ), and introducing quantitative scores like sparsity [ 11 ] . We randomly selected a diverse set of 124 binding proteins and fed their native FASTA sequences to DeepFRI. We verified that none of the proteins were in DeepFRI’s training set. We focus on binding proteins due to their clear active sites and distinguishable functional residues. Prioritizing slightly different features of the model’s architecture in calculation, GradCAM, Excitation Backpropagation, and PG Explainer all produce saliency heat-maps that highlight the most relevant/deterministic input regions (ie. amino acids in the sequence) prioritized by DeepFRI for prediction. These amino acids correlate with the highest activations in the GCN. We also apply a modified version of DeepFool [ 10 ] to understand how changes in sequence information change DeepFRI’s behavior. Similar to DeepFool’s emphasis on iteratively perturbing a minimal number of pixels in an image such that the majority of the original input is preserved, we identify the smallest number of mutations necessary to cause DeepFRI to misclassify a protein’s function. After calculating the activation scores for each amino acid residue with GradCAM, we assign a directly proportional probability score to each residue in the sequence. Using hardcoded "dissimilar amino acids", a FASTA sequence is iteratively mutated. Only one residue is mutated at a time, with previous mutations preserved from prior iterations, before the sequence fed to the model for prediction. This iterative process is conducted 50 times to produce a plot that showcases the average "mutation threshold" for a particular prediction, or the number of point mutations necessary to cause a misclassification. This enables us to understand the model’s sensitivity and generalization. 3 Results and Discussion We have results for thousands of combinations of explainability methods, proteins, and GO terms, but we outline two representative examples here in detail. 3.1 Case Study 1: Lac Repressor Protein The lac repressor is a DNA-binding protein that inhibits transcription of lactose metabolism genes when sugar is absent [ 9 ] . We used GradCAM to identify regions relevant to DeepFRI’s predictions. Figure 1 : Above: GradCAM on 2PE5B, the Lac Repressor. The line graph shows neural network activations plotted for residues from beginning to end. The residues highlighted in red (60-65, 97-99, 124-6, 275) are the most important in DeepFRI’s prediction of the function of the Lac Repressor. Yellow spheres represent water and ligands. Below: Table with DeepFRI’s top three functional prediction for the Lac Repressor with confidence scores. All of these functions are consistent with the true function of the lac repressor. Shown in figure 1, the red/most-relevant residues correspond to a distinctive feature of the lac repressor, the singular beta pleated sheet found in its monomeric subunits. This sheet stabilizes the folded structure, but is surprisingly not related to the protein’s actual function. We found DeepFRI prioritized amino acids that control protein structure over protein function in over half the proteins we examined. This could result from DeepFRI being trained on homology data, a critical metric for determining biological function, but not always for determining functional residues. This exemplifies the tradeoff between accuracy and interpretability. Next, we investigated the robustness of DeepFRI with DeepFool, initially testing a baseline of two manually altered sequences. First, a completely scrambled version of the sequence in which all amino acids had been rearranged. This, as expected, yielded vastly different GO classifications related to metabolism. Second, we inputted a sequence in which only the red-labeled residues (the most statistically significantly relevant ones) were replaced with their least similar amino acid counterparts. Surprisingly, DeepFRI correctly predicted the same GO classifications as the native sequence, with the same prior confidence. When amino acids were mutated from most to least relevant (according to GradCAM), it took 206 mutations of the 330 amino acid lac repressor for misclassification to occur. However, when 50 iterations of stochastic mutations (sampling from a probability distribution of residues weighted by relevance) was performed, the mutation threshold approached 75. The fact that the model is more sensitive to mutations that are not the "most relevant" poses questions about the true relevance of each residue. It is highly possible that the amino acids prioritized in predictions do not have significant differences in relative weight, as supported by the earlier sparsity findings for GradCAM. This extreme robustness—needing to mutate over two-thirds of residues before the GCN’s output flips—suggests that the model’s message-passing is dominated by stable local neighborhoods. In other words, unless enough residues are changed to reconfigure those “neighborhood” features en masse, the graph convolution layers keep aggregating