Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data Robert Frank, Michael Widrich, Rahmad Akbar, Günter Klambauer, Geir Kjetil Sandve, Philippe A. Robert, Victor Greiff, Pingzhao Hu Abstract Generative machine learning models offer a powerful framework for therapeutic design, by efficiently exploring large spaces of biological sequences enriched for desirable properties. Unlike supervised learning methods, which require both positive and negative labeled data, generative models such as LSTMs can be trained solely on positively labeled sequences, for example, high-affinity antibodies. This is particularly advantageous in biological settings where negative data are scarce, unreliable, or biologically ill-defined. However, the lack of attribution methods for generative models has hindered the ability to extract interpretable biological insights from such models. To address this gap, we developed Generative Attribution Metric Analysis (GAMA), an attribution method for autoregressive generative models based on Integrated Gradients. We assessed GAMA using synthetic datasets with known ground truths to characterize its statistical behavior and validate its ability to recover biologically relevant features. We further demonstrated the utility of GAMA by applying it to experimental antibody-antigen binding data. GAMA enables model interpretability and the validation of generative sequence design strategies without the need for negative training data. We have developed a new way to look inside AI language models that create biological sequences such as antibodies, even when only examples of successful binders are available. Traditional design methods need both positive and negative examples, but in many biomedical settings negative data are scarce, unreliable, or undefined. By training a model on only the desirable (positive) sequences and then comparing the model’s internal “gradient” signals to those of an untrained reference, we can identify which positions in a protein sequence the model deems important for binding. This approach, which we call Generative Attribution Metric Analysis (GAMA), lets us recover known binding motifs from synthetic data, confirm biologically relevant positions in simulated antibody‐antigen interactions, and highlight key residues in real experimental antibody datasets. The method therefore adds a layer of interpretability to otherwise “black-box” AI-based protein design tools, helping researchers understand why a generated sequence is predicted to work and enabling more confident use of these models in therapeutic development and other areas where only positive examples exist. Introduction Deep generative neural networks are a class of models capable of learning complex data distributions and can enable the generation of novel instances from them. They do not require labels and can be trained on “positive-only samples” (hereafter referred to “1-class samples”). Some of the most common approaches used in generative modeling are Long Short-Term Memory networks (LSTM) [], state space models [], Generative Adversarial Networks (GAN) [], Variational AutoEncoder (VAE) [], diffusion models [], and transformer networks []. Several protein design approaches have succeeded using (deep) generative models [–]. Generative models have also been used to design DNA and RNA sequences, e.g., to design novel promoter sequences or RNA sequences with desired folding properties [–]. Previous reports have demonstrated that monoclonal antibodies can be synthetically generated using generative models and have a distribution of binding affinity and various biophysical parameters that is similar to the training data distribution [,,–]. The interpretability of deep learning methods is inherently challenging, as they process information in highly nonlinear hierarchical representations. Various attribution assignment methods have been developed to address this issue, aiming to elucidate the contribution of input features to the output decision [–]. These methods are instrumental in diagnosing individual classification errors and assessing the confidence level of model reliability []. Among these techniques, Integrated Gradients (IG) stands out as a particularly effective approach []. IG emphasizes the significance of input features in the prediction process and has been successfully employed in various supervised sequence prediction tasks. For instance, Senior and colleagues demonstrated IG’s utility in elucidating the influence of feature location on protein folding prediction from sequences [,]. Similarly, Preuer and colleagues showcased IG’s role in molecule design for drug discovery [], and Ishida and colleagues utilized IG to identify key atoms in molecules for reaction prediction []. The use of IG in supervised LSTM models is well-documented across multiple studies [,], including its application to DNA sequences [] and RNA sequences []. IG has also been implemented in the analysis of antibody sequences: our recent work revealed that IG, when employed in binary classification methods, uncovers structural patterns relevant to antibody binding [,]. Furthermore, Widrich and colleagues identified pertinent patterns in amino acid sequences within immune repertoire datasets using IG []. A detailed guide on integrating the attribution assignment method “Layer-wise Relevance Propagation” into LSTM models was provided by Arras and colleagues []. Extensive research has been performed on rendering supervised methods interpretable. However, there is only limited research available that has focused on the interpretability of generative sequence models [,], especially generative sequence models for 1-class data []. There are several papers on developing attribution assignment methods for transformers and large language models [–]. There exists some work on interpreting generative (sequence) models with a latent space [–]. Approaches that tackle attribution assignment methods for 1-class generative models are rare [,]. A survey paper from Tritscher et al. is available []. To our knowledge, there exists no prior work which uses gradient-based methods on LSTMs for attempting to identify essential patterns that are relevant for LSTM-based sequence generation. Interpretable methods would enable addressing the negative label bias problem, which primarily occurs in immunological datasets but poses a challenge in many supervised multiclass settings where the definition of negative data is ambiguous [,,–]. Specifically, supervised methods for immune receptor classification tasks require at least two labels in the training data, where one class usually consists of those receptor sequences that have been validated to bind the target antigen. The other, negative class, is challenging to define as it may be defined as non-binders or, for example, binders to another target antigen []. Indeed, we and others have previously shown that the training data composition strongly impacts the classification of the positive class by ML methods and best practices for defining negative datasets are still controversial [,,–]. Applying attribution assignment methods to solely 1-class (positive‐only) data (i.e., antigen-specific binders) using generative models would reduce the requirements for training data size and complexity, enabling the generation of both novel binders and an understanding of the underlying binding patterns. Furthermore, interpretable generative models would enable greater trust and confidence in sequences designed by such models, making them more viable for sensitive applications, such as drug design [,]. To elucidate the underlying determinants and biological principles that influence sequence generation by autoregressive generative models (), we have developed a novel attribution assignment method (GAMA) (), which highlights the importance of specific positions for the positive class. GAMA is based on Integrated Gradients (IG) for LSTMs. We demonstrated GAMA’s application to LSTMs trained on synthetic sequences and the results suggest that the importance of features gives us information on binding rules, i.e., amino acids that are important for binding. We define “Importance” as the difference between the integrated gradients (attribution assignment method) of a trained LSTM and a randomly initialized LSTM. We also show the applicability to antibody sequences. Briefly, LSTMs and other state-space models estimate a probability distribution for the next token in a sequence given the previous tokens []. In this way, LSTMs model the entire sequence distribution globally and unsupervised while learning a supervised model locally for next-token prediction []. IG works on the local level of the autoregressive model, i.e., predicting a single element from the previous element, by identifying the impact of one input-output token pair. By summarizing the IG results over all possible combinations of input-output pairs, GAMA obtains the global distribution that the model has learned. Monoclonal antibodies (mAb) are used for various applications, such as therapeutics or as diagnostic biomarkers []. Each individual drug and application requires a specifically designed mAb. MAb design is currently performed both in vivo or in vitro []. This is usually very time and cost-intensive, for example, lead times to mAb discovery and design are on average >3 years [, ]. Generative in-silico design of mAb sequences is a promising avenue for improving mAb design lead times and costs. Additionally, mAb sequences are a good modeling [] system for evaluating our methods due to the availability of synthetic ground truth data []. The structural binding of an antibody to a target protein is described by a set of amino acids (termed paratopes) that interact with the binding site of the target antigen (termed epitopes) []. Paratopes are typically localized in the complementarity-determining regions (CDR) of antibodies. Paratopes can have complex interactions with epitopes []. Consequently, attribution assignment methods must strongly consider the complexity [] of the datasets and the robustness towards noise. Paratopes are usually gapped motifs [] and we investigate the hypothesis that attribution methods will focus on binding residues preferentially, an analysis of attributions could be used to predict the paratope of antibody sequences. Successful attribution assignment of these motifs will pave the way for interpretable deep generative models []. In this work, we trained an LSTM with an autoregressive unsupervised objective on synthetic 1-class CDR3-antigen binding data at different levels of complexity as well as real-world data and identified the important features learned by the generative model using our newly developed method “Generative Attribution Metric Analysis” (GAMA) (). Results A method to quantify and highlight important motifs in generative models Generative models have been used to learn and generate antibody sequences with desired properties. We develop a tool, GAMA, to extract information from trained generative models (). In particular, we identify the features learned by a generative model as follows: First, we apply interpretability/attribution approaches, such as IG, to the trained autoregressive generative model “main model” to obtain the importance of each input/output pair - e.g., the LSTM predicting the next amino acid based on the previous ones (). Subsequently, we repeat this process for a “reference model” and use the difference between the main and the reference model to obtain the learned features that are particular to the main model. See Algorithm 1 for details. In the context where an antibody dataset contains binding sequences to an antigen, we hypothesize that if a generative model has learnt biological rules of antibodies, GAMA should reveal AAs important for the binding (), and define the capacity to identify motifs as the predictive power of the model. We define a motif as a set of repeating AAs at the same positions. We test our method by applying GAMA to models trained on datasets with known binding patterns. Since datasets with known binding patterns are rare, we first use 270 simple synthetic datasets with specific properties and implanted motifs to assess motif recovery under noise, motif position, inter-motif interactions, and motif length. We define motifs as short repeating elements of a sequence in a dataset. We assume this simulates important features of biological datasets that all share a single common gapped motif (paratope motifs are usually gapped motifs [, ]). Then we evaluate GAMA on complex simulation-derived Ab-Ag binding simulations [] with known binding motifs to assess if generative models can learn binding motifs. Lastly, we evaluate GAMA on an experimental dataset of HER-2 binding antibody sequences [] (the Trastuzumab paratope is a non-gapped motif of size four) where we are only certain of the motif of one of the sequences (). An LSTM is an autoregressive generative sequence model, which means it iteratively predicts the next element within the sequence given all previous sequences (next-character prediction), and is trained with teacher forcing. Such a model can sample novel sequences given a “start token”. GAMA is derived by comparing IG signals from a randomly initialized reference LSTM and a trained LSTM model; the reference model uses exactly the same weights that were later used for training, so that the only systematic difference between the two models is the effect of exposure to the true data distribution. The importance metric (GAMA) is derived by comparing IG signals from a randomly initialized reference LSTM and a trained LSTM model. The reference model is identical to the initialization used to train the model thereby giving us the difference the training data has induced into the model. Our similarity function M(t: = token position) is described in Equation 1. The similarity function measures the absolute median difference between the IG values for the reference model and the trained model (). The difference was weighted by the sum of the variance of the IG values in the reference model and the trained model and normalized by the total variance of the sequence.The metric was inspired by the two empirical observations of signal‐position behavior. - They exhibit a large absolute median shift between the trained and reference IG vectors. - IG values at those positions are more dispersed (higher variance) than at background positions. By explicitly modelling both aspects, Equation 1 captures the complete signature of a signal. The numerator is an estimate of the effect, while the denominator estimates the relative noise. Although the metric was initially motivated by the observed median‐difference/variance pattern; it follows a well‐established statistical recipe for comparing two empirical distributions: robust central‐tendency difference divided by a pooled dispersion measure. In the next section, we will benchmark GAMA on a simple synthetic dataset, a more challenging antibody synthetic data [], and antibody experimental data []. Equation 1 describes the computation of the similarity function M(s) that computes the similarity of our trained model IG distribution and our reference model IG distribution at token position t. [pcbi.1014805.e002.webp]Dt,trainedModel indicates all IG datapoints computed on the trained model at sequence token position t. N indicates the total sequence length. A large difference indicates an important sequence position. Benchmarking of LSTM-based motif recovery across three key parameters: noise ratio, motif position, and motif complexity on synthetic datasets We investigated how GAMA is affected by three variables: (i) noise ratio (ii) motif position (iii) motif logic. Noise ratio (i), is a ratio defined by the number of sequences containing motifs (motif_sequ) and the number of contaminating sequences not containing any motifs (no_motif): [pcbi.1014805.e003.webp]motif_sequmotif_sequ+no_motif Motif position (ii) refers to whether the amino acids of a motif are concentrated towards the beginning, middle, or end of a sequence. Motif logic (iii) designates how the amino acids within a motif are allowed to interact to be counted as a motif. Specifically, motif interaction is a function of all motif positions; each motif position either has the relevant AA (true) or not (false) if the function returns true then the sequence is considered to be part of the dataset. For example, a motif with “AND” logic must have all associated positions be set to true to be counted as a motif. We investigate three logic types: “AND”, “OR”, and exclusive or “XOR”. To investigate the impact of these three parameters (parameter 3-tuples), we explored all possible combinations of the parameter 3-tuples. Each experiment consisted of a synthetic dataset as described in Methods. Briefly, each sequence had a length of 16, a value chosen because it approximately represents the median length of human CDRH3 sequences [].the 3-tuples had the following values: three different logic motif interactions, three motif positions times three motif lengths, and ten noise conditions, which resulted in 3*(3*3)*10 = 270 experiments (). An example of such a 3-tuple is a “L” implanted at position two, a “Y” implanted at position four, with a motif logic of “OR”, and a signal ratio of 80% as seen in . Since we implanted the artificial motif into these datasets, we know the ground truth motif position. We investigated the usefulness of the GAMA importance values by applying it to LSTM models that were trained on datasets with known features and utilizing the importance values in a motif retrieval task. We demonstrated sufficient model convergence by sampling from the resulting models and plotting their frequency distribution (). We quantify GAMA performance by measuring the overlap between the top‐ranked positions (according to GAMA) and the known ground‐truth motif positions. Retrieval performance is the ability of GAMA to recover relevant motif sequences accurately. Therefore, high retrieval performance indicates high interpretability. The noise ratio tests whether GAMA is robust to noise within the dataset; in biological datasets noise can be introduced, for example, through imperfect FACS sorting or library amplification or sequencing-related errors. The motif position parameter investigates if the retrieval performance is position invariant because it could be degraded due to gradient vanishing. The motif logic investigates the ability of our method to identify motifs with non-linear amino acid interactions. For comparison, we define an “IG baseline” that summarizes Integrated Gradients computed only on the trained LSTM, without reference to a randomly initialized model. Concretely, for each sequence position t, we aggregate the absolute IG values over all input token types, all output token types, and all output positions, yielding a single baseline importance score per position. Unlike GAMA, which explicitly contrasts trained and reference IG distributions to highlight training-induced changes, the IG baseline reflects a conventional, reference-free attribution derived solely from the trained model. GAMA achieves high motif retrieval performance at high signal ratios We sought to understand how robust GAMA is in noisy datasets. Biological dataset creation is error-prone, i.e., sequences that are wrongly labeled as positive []. For example, FACS cell sorting can introduce noise when identifying high and low binding Ab binders, thus, introducing false positives into the dataset, we are not considering false negatives because we do not use negatively labeled data. Large numbers of sequences without motifs can also be inherent to certain datasets, for example, antibody repertoires sample all immune sequences from a subject with a disease status, but each subject will have many non-disease status-associated antigens. To assess how the presence of spurious (false‐positive) sequences influences motif recovery, we systematically introduced uniformly random amino‐acid sequences into the training set at varying proportions, which we refer to as the signal ratio. The signal ratio denotes the fraction of genuine signal sequences relative to the total dataset size. Starting from a fixed pool of 10 000 signal sequences, a signal ratio of 100% corresponds to a dataset composed exclusively of these sequences (no noise), whereas a signal ratio of 10% implies that only 10% of the final dataset (1 000 sequences) are true signal and the remaining 90% (9 000 sequences) are random noise. illustrates an example dataset constructed with a signal ratio of 80% (8 000 signal sequences and 2 000 noise sequences). We increase the signal ratio from 0% to 100% in increments of 10 percentage points. For each signal ratio we consider three separate motif-logic conditions (AND, OR, XOR) at each 9 different motif positions. This results in a total of 270 experiments (). In we show the results for all experiments with signal positions two and four for all motif logic types without averaging over them as an example. We can see that the GAMA attributions have the strongest distinction on the two and four positions for a signal ratio of 100% (no noise sequences) and “AND” motif logic. This distinction diminishes with more noise sequences and for “OR” and “XOR” logic datasets. We also created the same visualization with an IG baseline for comparison in . Further examples for GAMA and IG baseline for different motifs can be found in , , , , , , , and . In we show a synthetic experiment repeated with a different seed. Additionally, we assessed how well this synthetic data can be classified by the classical positive‐only algorithm (Isolation Forest []) . Because Isolation Forest operates at the sequence level and only distinguishes between noise and non‐noise sequences, its outputs are not directly comparable to the motif retrieval performance of our GAMA method. The Isolation Forest baseline achieves mostly close to random balanced accuracy, especially for longer motifs, indicating that sequence‐level positive‐only classification is mostly unable to leverage the implanted motifs. For comparison, we evaluated the performance of standard Integrated Gradients applied to LSTM models trained in a supervised manner on the same synthetic dataset . The results of this supervised baseline are directly comparable to the GAMA retrieval performance; however, the task it solves is trivial, so its superior performance relative to GAMA is unsurprising. Nevertheless, it highlights motif and logic configurations in the synthetic dataset that remain challenging even for supervised approaches, providing a useful reference for interpreting GAMA’s limitations. Supervised LSTM classifiers combined with Integrated Gradients perfectly recover motif positions under AND and, to a slightly lesser extent, OR logic, but still largely fail to correctly attribute positions under XOR logic. This pattern mirrors our GAMA results, where motif retrieval is strong for AND and OR conditions but substantially weaker for XOR, demonstrating the difficulty of capturing XOR‐type interactions for both supervised and unsupervised/one-class attribution methods. The drop in performance under high noise levels or when motifs follow more complex logical relationships stems from two factors. First, as the proportion of random noise sequences increases, the gradients that Integrated Gradients relies on become increasingly diluted; the signal contributed by genuine motif residues is averaged together with many irrelevant background contributions, lowering the contrast between true‐positive and background positions. Second, complex motif interactions (e.g., OR or XOR logic) create non‐linear dependencies among residues, so the attribution of a single token no longer captures the joint effect of the motif. Consequently, the IG‐derived importance for each individual position is weaker and more variable, leading GAMA to miss or mis‐rank those positions when identifying the underlying pattern. For every synthetic data set, we know exactly which positions in the sequence constitute the implanted motif; we call this set the ground‐truth motif. After GAMA is applied, each position in the sequence receives an importance score and the positions are ranked from most to least important. We then take the top‐ranked positions equal in number to the size of the ground‐truth motif (for example, if the motif occupies four positions, we examine the four highest‐scoring positions). The false‐negative‐rate is defined as the proportion of true‐motif positions that are missed. It is calculated by dividing the number of false negatives by the total number of motif positions. Consequently, a false-negative rate of 0% means that every true motif position was recovered, whereas a false-negative rate of 90% means that only 10% of the true motif positions were recovered (lower is better). We averaged the false-negative-rates for 9 position experiments and plotted them with respect to each signal ratio for motifs that were formed by one of the logic conditions (–). Since, to the best of our knowledge, there are no attribution assignment methods for 1‐class sequence generative models available, we computed a random baseline with a Monte Carlo simulation to derive the false-negative-rate, one would get if the motifs were retrieved at uniform random; resulting in a false-negative-rate 0.93. We also computed the false-negative-rate for the IG baseline. GAMA outperformed the random baseline in all experimental conditions. GAMA had the best performance for motifs with AND-logic, motifs were retrieved with 0% false-negative-rate for a signal ratio ≥80%. For a signal ratio of 40%–70%, the signal is identified with a false-negative-rate lower than 50% (). GAMA could still recover motifs with an OR-logic but the performance was worse for the entire signal ratio range. Motifs were retrieved with a false-negative-rate lower than 50% for signal ratio of 50%–100% (). Motifs with Exclusive-OR logic are the most difficult to retrieve with GAMA. The false-negative-rate is worse for the entire signal ratio range and the false-negative-rate is below 50% only for signal ratios of 90%-100%. To summarize, GAMA was able to identify the signal across all noisy dataset conditions to some extent. We investigated how effectively the GAMA function retrieves the motif positions, for all 270 datasets. Subsequently, we calculated and visualized the false-negative error rate (), and the TopK (until full retrieval) () for all motifs. The performance of GAMA is position-invariant We investigated whether the performance of our method is position-invariant, as motifs may vary in positions across biological sequences. Since our method was designed for autoregressive models, motif positions at the beginning of a sequence may be less reliably identified than those toward the end. The weaker gradients experienced for elements located towards the end of the computation graph likely contribute to this discrepancy []. To examine the reliability of motif identification, we implanted motifs at three different positions within the sequence: the front, middle, and end. We averaged the false-negative rate over the three motif positions (, ). We observed very similar false-negative-rate distributions for all three motif positions. Only slight shifts in the distributions are observable. Therefore, we concluded that the performance of GAMA is position-invariant. The length of the motif correlates positively with the GAMA false-negative-rate We investigated whether longer motifs render recovery more challenging. For instance, Akbar and colleagues showed that antibody binding sites (paratopes) can vary in amino length [, ]. Multiple relevant elements in our sequence may dilute the IG signal. To answer this question, we queried motifs of different lengths (binary, trinary, quaternary). Binary motifs have a lower false-negative rate than the larger motifs and the false-negative-rate is similar across all three motif lengths. (). This can be explained by the fact that larger motifs also have more possible combinations. GAMA false-negative-rate of retrieved motifs increases in the following order: AND-logic, OR-logic, XOR The function of a biological sequence is driven by inter-position dependencies. Individual amino acids within binding motifs of CDR sequences can have different effects on the binding affinity. For example, a CDR might only effectively bind an antigen if all amino acids of the motif have the correct binding structure (AND-interaction). In contrast to a binding motif that just requires one of the amino acids to be correct (OR-interaction). Different interactions between amino acids within the motif can be described by logical operations. We investigated the performance of our method on different motif interactions. We considered three logic operations AND, OR, and XOR. Valid signal sequences must conform to this logic otherwise they are considered noise sequences. For example, for the AND logic to be considered valid, all signal positions must be activated, but for the OR logic only one of the sequence positions must be activated. We also included longer signals that contain more than two sequence positions; we include binary, trinary, and quaternary. The non-binary sequences are created by chaining the respective logic operation (i.e., s1 AND s2 AND s3). The false-negative-rate rate for all experiments with the same signal logic is averaged. The results are plotted in ( “AND-logic”, “OR-logic”, “XOR-logic”). In summary, we evaluated the performance of our method across various motif interaction logics (AND, OR, XOR) and extended analyses to include longer signals, demonstrating how different logical dependencies influence false-negative rates. GAMA attributions on synthetic antibody binding data correlate with positional contribution to binding affinities GAMA attributions on synthetic antibody binding data correlate with positional contributions to binding affinity. We first characterized performance on a simple synthetic data set () and then evaluated GAMA using a more complex and realistic synthetic benchmark derived from antibody‐binding simulations generated with the Absolut! framework []. This framework provides binding affinity of antibody CDRH3 sequences together with the amino acid-wise binding energy values contributing to the total binding affinity [], serving as a proxy for the position-specific ground truth amino acid significance, which is largely unattainable in experimental data. The informativeness of binding locations is likely correlated with the binding energy if a specific binding motif mediates the physical interaction with the epitope. It has to be noted that amino acids not within the paratope may also have high information content, for example, an amino acid that causes a change in the 3D structure of the paratope. The Absolut! framework discretizes an antigen and systematically explores all possible CDRH3 binding configurations and computes the optimal binding affinity (). We investigated whether GAMA effectively highlights amino acid motifs that correspond to the binding energy contributions of each amino acid in the CDRH3 region. We generated a set of binding CDRH3 sequences for four different antigens (pdb-id 5CZV: 4823 CDRH3 sequences (), pdb-id 5KN5: 2047 CDRH3 sequences (), pdb-id 4K24: 2173 CDRH3 sequences (), pdb-id 3Q3G: 1013 CDRH3 sequences ()). We selected these CDRH3 sequences from the top 1% binders to a specific epitope on a given antigen, containing at least 1000 sequences with the lowest entropy in the dataset. (Information)-entropy quantifies the uncertainty within a dataset; the lowest entropy dataset is the one with the highest information. For the 5CZV derived dataset (frequency distribution plotted in ), we compared the Abolut! [] ground-truth data with the attribution assignments derived from GAMA. Our analysis successfully identified a high binding affinity amino acid at position six while the IG baseline mostly failed to identify important features, as shown in . To evaluate the consistency across datasets, we calculated the Spearman correlation between Absolut! binding affinities and GAMA attributions for all four datasets, marking statistical significance with asterisks (*) in . The non-perfect correlation may be explained by the fact that most informative locations should not necessarily be binding locations, for example, an amino acid that causes a change in the 3D structure of the paratope. Additionally, we computed one-sided 95% confidence intervals for the correlation coefficients using bootstrap sampling, also depicted in . The one-sided p-values for the four antigens were as follows: 5CZV p-value = 0.004 < 0.05; 4K24 p-value = 0.018 < 0.05; 5K5N p-value = 0.08; 3Q3G p-value = 0.196. To summarize, our analyses highlight significant correlations between Absolut! binding affinities and GAMA attributions for two antigens (5CZV and 4K24), with statistical support provided by one-sided p-values and confidence intervals, while variability in other datasets underscores the complex relationship between binding locations and structural influences. GAMA identifies the most important amino acid positions in the CDR3H region of the Trastuzumab antibody We demonstrated that GAMA effectively identifies significant motifs in synthetic sequences and synthetic antibody sequences, even under noisy conditions. Notably, GAMA achieved best performance with datasets exhibiting less than 50% noise ratio and motifs connected by “AND”-logic. Although most other conditions surpassed the random baseline, motif identifications remained prone to errors (). To extend our analysis, we examined the experimental Trastuzumab-HER-2 binding data from Mason et al. [] (hereafter referred to as the “Mason dataset”). The Mason dataset comprises of 8955 sequences that were generated by mutating the original trastuzumab CDRH3 and filtering them for sequences that bind the HER-2 antigen; the resulting frequency distribution is visualized in . Given the experimental nature of the dataset, the amount of noise in it is unknown. Nevertheless, the crystal structure of unmutated Trastuzumab bound to HER-2 (pdb-ID: 18NZ) identifies the key paratope amino acids for binding at positions 102, 103, 104, and 105 (). It remains uncertain whether these positions maintain their significance within the Trastuzumab variants. Since GAMA identifies averages over all sequences, different binding motifs within the same dataset may dilute the GAMA attributions across the sequence length. Therefore, the correlation between the informativeness and the binding motif might not be perfect. presents our findings with a logo plot of the amino acid frequency distribution shown in . The GAMA activations for the Mason dataset, highlighted in , underscore positions 103, 104, 105, and 107 as particularly important for the generative model, with some overlap between the predicted and known binding positions of the original Trastuzumab-HER-2 interaction. The IG baseline in fails to recover any of the four important positions. In summary, the analysis in reveals key positions identified by GAMA activations that align partially with known binding sites, emphasizing their relevance to the generative model and the Trastuzumab-HER-2 interaction. Discussion In this paper, we developed a novel analysis method, GAMA, to enhance the interpretation of generated sequences for autoregressive generative models and mitigate the effects of negative label bias. This advancement is particularly relevant for datasets devoid of negative label sets, known as “Positive and Unlabeled data” (PU-data), which find extensive applications beyond biological contexts, including recommender systems, fraud detection, information retrieval, and personal advertisement. Methods for PU-data typically employ semi-supervised strategies within an iterative “Two-step” learning process, wherein a supervised method classifies unlabeled data, subsequently using the new labels as training data for successive iterations [–]. While these PU-data approaches are versatile, they do not provide solutions specifically tailored for generative models. Recent generative approaches to antibody sequence design often fail to elucidate why a generative model considers a newly generated sequence to be optimal for a specific task [, ]. GAMA addresses this gap by offering interpretability, and it is applicable to a wide range of (differentiable) autoregressive models, including large language models (e.g., protein language models), which have demonstrated the most promising recent advances in generative antibody design [, ], as well as other enhancements in state space models such as xLSTM [], and Mamba []. GAMA could also be extended to address non-biological challenges, such as generative models for outlier detection employed in fraud detection, where the interpretability of model outputs is of importance. Utilizing simple synthetic sequences with well-defined motif signals, we observed that GAMA can recover binding motifs under various sequence noise conditions and remains invariant to motif position (, ). Our results indicate a strong correlation between GAMA and the ground-truth data in simulated antibody binding data (). We also demonstrated the efficacy of our approach using an experimental antibody binding dataset, and thus confirming its applicability to real-world data (). A direct comparison to published classification based attribution studies, for example, in terms of sensitivity, is not straightforward in our setting because those approaches rely on clearly defined positive and negative labels, whereas our models are trained on 1‐class, positive-only data and there is no well-defined negative class against which to compute true‐positive or false‐positive rates. Instead, we systematically compared GAMA against an Integrated Gradients baseline computed on the trained LSTMs without reference-model subtraction; across all synthetic conditions, simulated antibody datasets, and the experimental Trastuzumab–HER2 data, GAMA consistently outperformed this IG baseline in recovering known or proxy ground-truth positions (, , /, -, -). The evaluation of GAMA on experimental HER-2/Trastuzumab binding data revealed that positions 103, 104, 105, and 107 are particularly significant for the generative model. When comparing these results to the four binding positions in the unmutated Trastuzumab sequence, which indicate positions 102, 103, 104, and 105 as most important for binding with the HER-2 antigen (), we matched the positions 103, 104, 105 and 107, but failed to match position 102. This discrepancy between structural information and GAMA-identified positions could stem from several factors. Firstly, structural information is available only for the unmutated Trastuzumab sequence, making it uncertain to what extent mutated sequences retain the same binding amino acids. Secondly, GAMA provides average amino acid attributions over all sequences, meaning that multiple different motifs within the dataset could aggregate into a single “combination” motif. Thirdly, as demonstrated in with synthetic sequences, error increases with higher noise levels and more complex inter-position interactions. In this study, we benchmarked GAMA using simulated paratope motifs of up to four interacting amino acids, separated by gaps of up to one or two amino acids. These motifs encompass the most probable types in terms of length and number of gaps. The distributions of paratope motifs have been thoroughly investigated []. For instance, the CDRH3 region exhibits a broad range of motif lengths, from one to fifteen, with a median of five, while gap numbers range from zero to five, with a median of one. In contrast, the CDRL2 region features paratope motifs ranging from one to seven in length, with a median of two, and gap numbers ranging from zero to five, with a median of two. Applying GAMA to generative antibody sequence models represents a promising avenue for rational antibody design [,,]. Understanding the rules governing antibody-antigen binding, which operate on the antibody and (protein)-antigen sequences [], could facilitate the creation of novel, fit-for-purpose antibodies. Our work represents an initial step towards this goal. Currently, GAMA analyzes single positions within the sequence chain, which hampers its ability to identify more complex binding patterns, such as higher-order dependency patterns [], and may result in the erroneous identification of important positions []. GAMA offers an aggregated summary of the generated sequence patterns by integrating attributions across the entire dataset. This methodological approach, however, limits GAMA’s capacity to discern motifs in environments with very low signal ratios, thereby constraining its applicability to domains where motifs are anticipated to be sparse, such as Adaptive Immune Receptor Repertoire (AIRR) datasets. For example, only approximately 10% of TCR sequences in cytomegalovirus (CMV)-infected patients are associated with CMV []. Future work will focus on extending GAMA to single sequence analysis, enabling attributions to be computed for individual sequences. Currently, the noise of the integrated gradients for a single sequence is too high to allow for meaningful individual interpretation, necessitating the averaging of attributions over many sequences, but the gradients contain much more information in 4D, which should be further analyzed. Although the present study implements and validates GAMA using an LSTM, its computational principle does not inherently depend on the LSTM architecture: it requires a differentiable autoregressive model for which gradients can be computed with respect to the input embeddings. This suggests that GAMA could potentially be extended to other architectures, including transformers and state-space models. However, such extensions may require architecture-specific implementation and validation and remain an important direction for future work. Additionally, future research should incorporate pairwise and higher-order dependency information, as we know that CDR binding is mediated by the interaction of multiple amino acid positions within the CDR [,]. Methods Datasets Synthetic dataset generation Whether CDR sequences are associated with binding to an antigen usually depends on short sequences of amino acids that physically interact with the epitope (). To simulate this interaction, we created datasets containing sequences with uniform sampled amino acids, and containing implanted signal amino acids that designate a motif. Each dataset is associated with an experimental condition that is described by a 3-tuple: motif positions (list of motifs within the dataset, represented as a list of all sequence positions where the motif is implanted: (2, 4), (7, 9), (13, 15), (2, 4, 6), (7, 9, 11), (12, 14, 16), (2, 3, 4, 5), (6, 7, 8, 9), (11, 12, 13, 14)), motif logic (“AND”, “OR”, “XOR”), and noise ratio (1, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1). An example of how the dataset creation is structured can be seen in . Noise ratio is the ratio between sequences with an implanted motif and noise sequences devoid of any motif if motifs are present by chance the sequence is rejected and resampled. The randomly generated sequences without signals were inserted into the datasets to assess the model’s robustness against noise. Specifically, the noise sequences were composed of uniformly sampled tokens, while the signal sequences were also uniformly sampled but included implanted signal tokens representing the motif. A token represents an element of the sequence and corresponds to an amino acid. The synthetic datasets were generated using a deterministic seed to ensure reproducibility. Each sequence had a length of 16, a value chosen because it approximately represents the median length of human CDRH3 sequences []. Each experiment consisted of 10 000 signal (positive) sequences paired with a specific quantity of noise sequences. Resulting in datasets of up to 100 000 sequences for the smallest signal ratio. We demonstrated that 10 000 sequences are adequate for achieving robust model performance by training deep generative models on iteratively smaller subsets of a complete dataset (). We generated new datasets based on three primary variables: signal ratio, signal positions, and signal logic. The signal position parameter also included motifs of varying sizes to examine the signal recovery rate of our model in relation to motif size. Synthetic antibody datasets generated with the Absolut! framework We evaluated GAMA on four antibody datasets created with the Absolut! simulation framework. Each dataset contains CDRH3 sequences that bind a single antigen. The four antigen-dataset pairs are: 5KN5 (2 047 binders), 5CZV (4 823 binders), 4K24 (2 173 binders) and 3Q3G (1 013 binders). Each dataset was generated by first identifying an epitope hotspot on an antigen, followed by selecting the top 1% of antibody binders to form the dataset. We conducted a comprehensive search of all possible epitopes within the Absolut! database and selected the four epitopes that yielded datasets with the lowest entropy (i.e., highest information content), while ensuring that each dataset contained a minimum of 1,000 sequences. Experimental dataset The original dataset from Mason et al. [], comprising HER-2 binders. The positive binder sequences were filtered to include only sequences with a minimum of two reads. This filtering step was undertaken to eliminate false negatives that may have arisen due to erroneous reads or contaminations (). Computational methods and data processing LSTM training and sampling We trained an LSTM (Long Short-Term Memory) (1) model to learn a dataset in a generative manner, enabling the sampling of novel sequences. Each sequence in a dataset was encoded using one-hot encoding while adding start and stop tokens. All sequences shared the same length within one dataset. The LSTM model is trained to predict the identity function, which allows the model to predict the subsequent amino acid in the sequence, a technique known as teacher forcing []. This allows the sampling of novel sequences from the same distribution. We sampled new sequences from the model by initiating the trained model with a start token, then progressively sampling one amino acid at a time until an end token was reached. The LSTM model has a hidden state size of 1024. Training was conducted using teacher forcing and mini-batches, with the ADAM optimizer [] from PyTorch 1.8.1, utilizing default parameters except for a learning rate set to lr = 0.00001. This learning rate was determined through multiple test runs to ensure stable convergence (see and ). The weights were initialized with a uniform distribution according to [pcbi.1014805.e004.webp]U(−k,k) where [pcbi.1014805.e005.webp]k=1size_hidden_state. To ensure reproducibility of our stochastic experiments, the random seed value was set to zero. illustrates the prediction step of an LSTM, with input values shown in blue and output values in black, and gradients in black. Integrated gradient analysis Integrated Gradients (IG) [] is a method for attributing the contribution of input features to the output of a machine learning model. Given an input, a model, and its respective output, IG quantifies the importance of each feature by measuring the degree of change in the output when that particular input feature is altered, as defined by the gradient. The importance is defined by the magnitude of change in the output resulting from a change in the feature. Applying IG to generative models is generally non-trivial, as most attribution methods focus on explaining changes in predictions with respect to inputs, while deep generative models often lack a supervised objective that directly corresponds to the sample output. However, in the case of LSTMs, which are trained in a supervised manner to learn the identity function, IG can be computed with respect to the inputs for a single output. IG were computed on the input sequence [pcbi.1014805.e006.webp]x (encoded as a tensor with dimensions 22 x sequence length) and a zero input baseline [pcbi.1014805.e007.webp]x′. The zero input baseline mirrors the dimensions of the one-hot encoded input sequence but sets all values to zero, despite this not constituting a valid one-hot encoding and does not refer to a specific token. We generated 1000 equally spaced points along a linear path between the input sequence and the input baseline, with linear interpolations performed in the one-hot encoding space. Each input interpolation is described as [pcbi.1014805.e008.webp]x′+α*(x−x′) for 1000 equally spaced [pcbi.1014805.e009.webp]α∈[0,1]. For each of the 1000 input interpolations, we used the LSTM model to predict outputs and subsequently computed the gradients with respect to the corresponding inputs. By averaging over all 1000 gradients, we obtained the IG for a given sequence. These gradients were computed using automatic differentiation provided by PyTorch 1.8.1. Calculation of integrated gradients and structuring into a 4D Tensor The previous section described how IGs are computed. For this study, the IG are computed for each output token of a sequence, structuring the results into a four-dimensional tensor (1. Input encoding, 2. Input sequence token, 3. Output probability vector, 4. Output sequence token), which we hereafter refer to as the “IG-tensor” (refer to ). Given that IG can only be computed for a single output of the LSTM at a time, the LSTM, trained on a positively labeled dataset, was queried for the IG of an input sequence from the same dataset. This computation results in a two-dimensional tensor for a single output, representing the IG with respect to all LSTM inputs. The dimensions of this tensor are 22 (20 amino acids + start token + stop token) by the sequence length. We performed this IG computation for all possible outputs. Each output comprises the softmax output encoding of size 21 (20 amino acids + stop token) and a prediction for each token in the input sequence of size matching the sequence length. Consequently, this results in a four-dimensional IG-tensor with dimensions [22 (number of input tokens), sequence length, 21 (number of predicted outputs), sequence length]. Following the enumeration of all possible IG computations, we reduce and summarize the 4D IG-tensor to a more manageable two-dimensional tensor in the subsequent section. First, the 4D-IG-tensor was reduced to a 2D tensor by eliminating the first dimension (input encoding) and the third dimension (the output dimension for which the gradient is computed). The first dimension of the IG tensor was reduced by only selecting scalar values that correspond to the encoding position of the input amino acid as other positions are zero. The third dimension was reduced by averaging over all elements as we found them to be similar (). The IG baseline is the mean of the absolute IG tensor across dimensions one, three, and four. This results in a one-dimensional positionwise importance score, where each sequence position t is assigned the mean absolute IG magnitude aggregated over all input token types, all output token types, and all output positions. We use this quantity as the “IG baseline” throughout the Results section. The IG baseline uses the same IG values used to compute GAMA. GAMA computation algorithm We summarize the computation of GAMA in Algorithm 1. Algorithm 1. Calculation of GAMA: each sequence is passed into the trained LSTM model and the IG is computed for all outputs of the LSTM. This is done both for the reference model and the trained model. Subsequently, the difference is quantified using the GAMA function. Packages used For the creation of graphics, we employed several libraries. Logo plots were generated using Logomaker 0.8 [], while all other figures were created using Seaborn 0.11.1 []. Our computational work relied on the following libraries: PyTorch 1.8.1 [], NumPy 1.23.1 [], pandas 1.2.4 [], and SciPy 1.14.1 []. These libraries were integral to the implementation and analysis of our methods.