ClinIAN: Clinically Informed Attention Network Authors: Florian König, Jonas C Ditz, Elham Shamsara, Pontus Hedberg, Iuri Fanti, Pontus Nauclér, Luca Carioti, Andreas Walker, Milosz Parczewski, Francis Drobniewski, Francesca Ceccherini-Silberstein, Maurizio Zazzi, Björn-Erik Ole Jensen, Anders Sönnerborg, Nico Pfeifer Published: 2026 ABSTRACT AbstractMotivationArtificial Neural Networks (ANNs) hold promise in predicting disease severity from viral protein sequences. To gain clinical insights, the models must adhere to two key factors: first, they need to be interpretable, as using black-box models within a healthcare setting poses risks, and second, they should integrate viral and clinical features to correct for biases within the data. We propose ClinIAN (Clinically Informed Attention Network), an inherently interpretable end-to-end learning model that effectively combines clinical and sequence parameters. We evaluate ClinIAN within the context of the challenging task of predicting the severity of SARS-CoV-2 infection, but it could be applied to other medical and biological contexts, with sequence and tabular features, such as bacterial infections or cancer research.ResultsWe evaluate the performance capabilities of ClinIAN in predicting SARS-CoV-2 severity using a subset of the EuCARE hospitalized cohort. We show ClinIAN’s ability to capture biologically relevant features across multiple layers of resolution while retaining stable state-of-the-art performance.Availability and implementationThe code is available on Zenodo and on GitHub. 1 Introduction The impact of SARS-CoV-2 and the viral infection COVID-19 it causes continues to affect healthcare systems worldwide, even after more than five years since it was first identified in December 2019. As of 10 November 2024, the World Health Organization (WHO) reported more than 776.8 million confirmed cases and more than 7 million deaths attributed to the disease (WHO 2024). Although the pandemic has transitioned to an endemic phase, this situation presents a valuable opportunity to use data to improve preparedness for future health emergencies and mitigate medical and socio-economic challenges (Seabra et al. 2024). Several projects and initiatives have gathered and analyzed data about the pandemic. Among these efforts is the European Cohorts of Patients and Schools to Advance Response to Epidemics (EuCARE) (https://eucareresearch.eu) project, which has amassed substantial data to support pandemic response while also examining its effects on educational institutions and post-acute sequelae of SARS-CoV-2 (Nicola et al. 2020, Bai et al. 2024, Bellerba et al. 2024). These extensive data repositories can help understand the mechanisms of viral infections, which is essential for effective protection, particularly in identifying markers that predict disease severity. Various factors, including host characteristics (age, sex, comorbidities, lifestyle, genetic predispositions) (Hedberg et al. 2024) and pathogenic elements (viral variant virulence) (Abdullah et al. 2022), as well as interventions (Mohammed et al. 2022) and behavioral influences, must be considered. However, assessing all of these influencing factors in a laboratory setting can be very expensive and often unfeasible. Computational methods can support these efforts. In this space, artificial neural networks (ANN) have become crucial tools across a variety of applications due to their ability to identify nuanced relationships in large and complex datasets. ANNs can perform end-to-end learning for feature extraction and prediction, reducing the need for labor-intensive data preprocessing. They have also become a staple in biological and medical settings. From image classification and segmentation in biological imaging (Sarvamangala and Kulkarni 2022), protein structure prediction (Jumper et al. 2021), epidemiology (Xiang et al. 2021), to personalized medicine (MacEachern and Forkert 2021). Nonetheless, the application of ANNs to medicine requires a clear understanding of the inherent decision-making processes known as interpretable AI. This requirement poses a significant challenge, as ANNs are typically considered black-box models that exhibit strong predictive performances but lack transparency in their decision-making processes. To address this challenge, several methods have been proposed to understand the processes without altering the model structure, including the so-called post-hoc interpretation methods SHAP (Lundberg and Lee 2017) and DeepLIFT (Shrikumar et al. 2017). These methods are advantageous because they can be applied without modifications to the training regimen, allowing them to be used with any trained predictor. However, these methods have shown inconsistencies across applications, leading to uncertainty about the validity of the interpretations (Rudin 2019, Bordt et al. 2022). Several classical machine learning models, such as linear regression and decision tree methods, are inherently interpretable without the need for post-hoc methods. This inherent interpretability has also been successfully applied to ANNs, often leveraging attention mechanisms to enable model transparency (Balcı et al. 2023, Ditz et al. 2023a, 2023b). Within biological and medical applications, several approaches have been proposed to predict sequence-to-function relationships from sequencing data, including those available for SARS-CoV-2 (Dey et al. 2020). However, existing strategies often focus solely on the spike protein (S), neglecting the influence of other viral proteins. While the spike protein plays a key role in virus-host interactions, other proteins contribute significantly to viral infection dynamics and virulence (Kehrer et al. 2023, Walia et al. 2024). Additionally, many approaches oversimplify disease progression by focusing solely on viral sequences, thereby overlooking the impact of clinical and epidemiological factors. By excluding these critical features, there is a risk of introducing biases that skew the model towards mutations that could be better understood through clinical data, such as those that do not have a strong effect on mortality but are simply present in a successful variant. Integrating clinical and sequential features promises to enhance insights into the decision-making processes of the model. However, clinical data often present challenges due to its inherent heterogeneity and tabular structure. While ANNs excel in contexts with homogeneous data, they tend to struggle with heterogeneous tabular data. Conversely, decision tree methods, such as Random Forests and XGBoost, have been staples in tabular data analysis, but lack the capacity to learn embeddings within a greater modeling context. Recent advancements have led to ANN approaches capable of effectively handling heterogeneous tabular data while performing on par with or outperforming tree-based approaches (McElfresh et al. 2023, Borisov et al. 2024). A notable approach is TabNet (Arik and Pfister 2021), a state-of-the-art architecture that generates interpretation masks and feature embeddings, allowing insights into the relative importance of individual clinical attributes. In this work, we introduce ClinIAN (Clinically Informed Attention Network), a novel solution that effectively integrates clinical tabular and viral amino acid (AA) sequence data using artificial neural networks (ANNs) within an inherently interpretable, end-to-end learnable framework. We design ClinIAN to predict disease severity, leveraging both data modalities. ClinIAN applies TabNet to generate interpretation masks for clinical features and embeddings in later network layers. We incorporate AA sequence data by applying multiple-instance learning (MIL) based attention (Ilse et al. 2018) to enable interpretability both within and across proteins. We benchmark ClinIAN against established baseline methods using data from the hospitalized cohort of the EuCARE project. The EuCARE hospitalized study (Hedberg et al. 2023) provides a rich and nuanced dataset that includes both clinical and viral features for a subset of patients, offering an excellent platform for assessing the models’ capabilities. Our analysis reveals the performance and interpretability of ClinIAN across multiple layers of resolution, including patient-level predictors, protein-level contributions, and the influence of specific mutational positions on the predictions. By adopting a multimodal approach to predict medical severity in an interpretable manner, our framework promises to enhance decision-making processes in clinical settings. 2 Material and methods 2.1 EuCARE whole genome sequencing cohort We assessed the integration of clinical and viral features within a unified end-to-end trainable model utilizing data from the EuCARE hospitalized study (Hedberg et al. 2023). Our analysis is based on data collected as of 9 July 2024. The dataset from the hospitalized study encompasses a broad range of information, including laboratory parameters, patient comorbidities, and clinical patient metadata. For our analysis, we streamlined the parameter set to focus on 15 key factors: age, sex, infecting variant (variant of concern, VOC), year of hospitalization, and comorbidities (cancer, cardiac and cerebrovascular diseases, chronic kidney diseases, chronic liver diseases, chronic lung diseases, diabetes, hypertension, immunosuppression, neurologic diseases including dementia, obesity, and count of vaccinations). We made this decision due to missing data points in other features, ensuring we focus only on factors available to all patients. The primary outcome of interest is defined as mortality (within 30-days after infection). In the following, we refer to this variable selection as metadata. Further details regarding the distribution of features, labels, and case counts are provided in the supplementary material (Figs SF1 and SF2, available at supplementary material Bioinformatics Advances online). 2.2 Data preparation The subset within the hospitalized cohort that comprises both viral and clinical features, as described above, comprises 6982 samples. The viral sequences are represented as ribonucleic acid (RNA) sequences. To reduce dimensionality in the input space, we translated the sequences into amino acid (AA) sequences, focusing on identifying mutation positions within each viral protein. We applied Nextclade (Aksamentov et al. 2021) to align the sequences with the reference Wuhan sequence (Wuhan-Hu-1; Accession: MN908947.3) and identify amino acid mutations within each protein. Subsequently, we refined the set of mutations, retaining only those that occur in at least 0.5% of sequences, thereby reducing potential noise within the data (Table 1). Table 1This table displays the different mutation counts in the AA sequences of annotated and unannotated open reading frames (ORF) after applying quality control measures.ProteinsLengthIns.Del.Sub.#positions S 12733116976 ORF1a 44010145366 ORF1b 2695003333 N 419042425 ORF9b 9704912 ORF3a 275001211 ORF8 1210335 ORF7a 1210055 M 2220054 ORF7b 430022 ORF6 610022 E 750011Note that the sum of individual mutations might differ for each #position, as some sequence positions can exhibit multiple mutations (e.g. position 3 in M: D3G and D3N). S, spike protein; E, envelope protein; M, membrane protein; N, nucleocapsid protein.Ins., Insertion.Del., Deletion.Sub., Substitution. This table displays the different mutation counts in the AA sequences of annotated and unannotated open reading frames (ORF) after applying quality control measures. Note that the sum of individual mutations might differ for each #position, as some sequence positions can exhibit multiple mutations (e.g. position 3 in M: D3G and D3N). S, spike protein; E, envelope protein; M, membrane protein; N, nucleocapsid protein. Ins., Insertion. Del., Deletion. Sub., Substitution. We applied quality control measures to reduce the dataset and retain only high-quality samples. These measures included the removal of samples with a high percentage of ambiguous residues (“X”) in at least one protein (>10%). We also excluded samples that were substantially shorter than the reference sequence (<80%). Sequences containing mutations that led to a stop codon were also discarded. Acknowledging that multiple samples from individual patients may exist over time, we opted to exclude all subsequent sequences for this analysis, as our aim was not to model temporal variations, but rather to investigate the factors contributing to severe hospitalization in the first place. Additionally, we ensured that all clinical features were complete. Detailed steps of the preprocessing are outlined in the supplementary materials (Fig. SF10, available at supplementary material Bioinformatics Advances online). Following these procedures, the final dataset comprised 3328 samples, including 2999 samples from patients categorized as “survival” and 329 samples from patients labeled as “mortality.” We refer to this AA mutation set as mutations and to the entire dataset with metadata as the EuCARE whole-genome sequencing (WGS) cohort. To encode the identified mutations, we employed one-hot encoding. However, it is important to note that the ClinIAN model can be adapted to incorporate positional encodings derived from protein language models (pLMs), as demonstrated in our evaluations. To maintain dimensionality, we substituted positions with the corresponding amino acids from the reference sequences, unless a mutation was present. This encoding for a given protein p is denoted as ηp. We adjusted for insertions by expanding the mutation set to include a vector of zeros at the positions of the reference sequence and in all sequences lacking a specific insertion. For the analysis using pLM embeddings, we replace these positional one-hot-encoded vectors with vectors from the pLM. We create positional embeddings for the entire sequences and select the embeddings at the specific positions used in our analysis as input vectors for the model. While features such as sex and comorbidities are provided categorically, we adhered to the encoding framework proposed by Hedberg et al. (2024) for age and vaccinations. In particular, age was stratified into 3 distinct categories (18–49, 50–69, 70+), and vaccination status was classified into four categories (0, 1, 2, and 3 or more doses). The infecting variant was determined using Nextclade (Aksamentov et al. 2021), with the assigned Pango lineage (Rambaut et al. 2020, O’Toole et al. 2021) for each sample mapped to the closest variant within a predefined set that includes the Wuhan strain, as well as the Alpha, Beta, Delta, Gamma, and Omicron variants (BA.1, BA.2, BA.4, BA.5, BA.5.3.1, BN.1, BQ.1.1). For the baseline models, we employed one-hot encoding similarly to the sequence data. Our proposed model, ClinIAN, was designed to effectively handle categorical data without prior encoding by leveraging the capabilities of TabNet. 2.3 ClinIAN—Clinically Informed Attention Network In the following section, we introduce the modeling background of the ClinIAN network structure. We build on two ideas: learning from tabular data for clinical features and using MIL for the mutational features. 2.3.1 ClinIANseq—utilizing mutation data The first step towards ClinIAN was to develop an inherently interpretable, attention-based model that uses sequence information to predict COVID-19 disease severity. The model consists of two layers, providing different levels of information about its prediction decisions. The right part of Fig. 1 shows a schematic visualization of the model structure. A complete schematic is provided in the supplements (Fig. SF6). Similar hierarchical structures have been introduced before for other tasks (Do and Lähdesmäki 2025). Visualization of the proposed ClinIAN model. ClinIAN consists of a TabNet that embeds clinical features and provides interpretations, as well as a multilevel mutation-based attention model (ClinIANseq). We concatenate the TabNet embedding vector for each mutation position, enabling the model to consider this information for each position. For the proposed purely sequence based model ClinIANseq, we create an embedding matrix for each protein by encoding every mutation position as a one-hot encoded vector. These embeddings are forwarded through a MIL attention layer to produce local attention weights for each mutation in each protein. The second MIL attention layer produces interpretations of the embeddings from the previous attention layers, indicating which protein is most relevant to the final severity prediction. Our model provides full inherent interpretability by applying attention-based MIL with a gating mechanism (Ilse et al. 2018). In general, MIL treats each sample as a bag of instances, with a single label for the bag (Dietterich et al. 1997). In MIL, two approaches exist: instance-level MIL and bag-level MIL, with instance-level MIL providing interpretability within models and bag-level MIL achieving stronger performance (Carbonneau et al. 2018, Wang et al. 2018). The extension of MIL with attention mechanisms has enabled interpretability comparable to that of instance-level MIL while retaining the performance of bag-level MIL (Ilse et al. 2018). Within our model, we applied MIL on two separate layers. First, we considered each mutation position as an instance. Therefore, there is a bag for each protein. Each MIL layer provides an embedding, which we then use in a second bag, where each protein embedding is considered as an instance. The first level of the model applies attention-based MIL for each protein individually. We applied MIL layers for each protein as follows. We considered each protein p and its embedding as a bag of instances Hp={ηp,1,…,ηp,Np} where Np is the number of mutation positions within p. We computed the calculated embedding of the bag with It was shown that adding a gating approach increases the expressiveness of the layer (Ilse et al. 2018). We therefore adapted the calculation of ap,n as follows: The variables w∈RL×1, V∈RL×M, and U∈RL×M are parameters of the attention layer and sigm(·) is a sigmoid non-linearity. Applying the MIL attention in this way enables the observation of the feature importance of each mutation within the sequence for the model’s final prediction. The learned embeddings zp are then concatenated to form a new bag of instances Z. The second level of the model utilizes the newly created embedding matrix Z as a new bag of instances and applies an attention-based MIL layer again. The embedding for the bag Z={zS,…,zORF7a} is calculated in a manner analogous to the prior layer (see Supplement, EQ. SE1). This enables the feature importance of each complete protein in the prediction task to be evaluated. Our model, therefore, provides multiple layers of resolution within our interpretability scheme to uncover the important features. In our training, we optimized the embedding vector dimension. The activation of the last layer is passed through a linear layer, which classifies the severity of COVID-19 disease. 2.3.2 ClinIAN—incorporating clinical information We extended the proposed model structure by integrating TabNet (Arik and Pfister 2021) as an upstream module, then combining its outputs within our architecture. TabNet can make predictions using heterogeneous tabular data while providing interpretations of the feature importances. The network consists of multiple sequential attention steps that combine feature selection and prediction, mimicking the approach of tree-based methods. Each step uses a learned attention mask that emphasizes the most salient features for the prediction task. By aggregating these masks across steps, we derive a comprehensive feature-importance map for the clinical metadata. Additionally, we use TabNet to learn an embedding of the clinical metadata within the greater context of our model. The dimensionality of this embedding vector is systematically optimized within our parameter grid. We concatenated this embedding vector with the mutation at each position across all proteins, thereby enriching the attention-based MIL layers with both sequence and clinical information. This strategic integration served multiple purposes. By appending the embedding vector to each position, we utilize this clinical information to mitigate biased attention toward mutation positions that may be more easily explained by clinical parameters. This adjustment applies to the calculation of attention weights, which are computed instance-wise, thereby ensuring that no inherent bias is introduced toward specific mutations due to variations in clinical vectors. This approach mimics the correction for population structure in linear mixed models (Hoffman 2013, Hake et al. 2023). Alternatively, one might consider implementing an additional layer to project the TabNet embeddings onto protein-specific embeddings. However, this approach risks introducing inconsistencies among proteins, attributable to the differing lengths of each protein’s embedding (for instance, the envelope protein (E) has an embedding dimension of one, while the spike protein (S) has an embedding dimension of 76, as detailed in Table 1). Subsequently, these concatenated features are processed through an analogous model architecture to that utilized in the sequence-based approach. Figure 1 displays the full ClinIAN model structure. The entire model is designed for end-to-end training, providing interpretability for clinical features through TabNets’ structure and the viral sequence interpretations described above. We optimized ClinIAN using the additional sparsity loss term suggested for TabNet (Arik and Pfister 2021) with L=Lprediction−λLsparsity where λ controls the strength of the sparsity regularization (see supplement EQ. SE3). This strategic integration of tabular confounders into a hierarchical model differentiates ClinIAN from other multimodal approaches that first encode modalities independently, such as MOMA (Moon and Lee 2022). The structure allows for a principled approach: first analyzing the influence of whole proteins, then enabling deeper analysis of the correlation between mutations and the outcome. 3 Results We trained the models using the EuCARE WGS cohort, which includes viral sequences, to predict COVID-19 mortality. Our approach’s performance was compared against established baseline methods, specifically Support Vector Machine (SVM) and Random Forests. We further incorporated the state-of-the-art method MOMA (Moon and Lee 2022) in our comparison. To ensure a fair evaluation in performance reporting, we used a 10x10 Nested Cross-Validation (CV) strategy in our training pipeline, optimizing hyperparameters for both the baseline and proposed models via grid search. Detailed information about the grid parameters can be found in the supplementary materials (Sec. S3, available at supplementary material Bioinformatics Advances online). To maintain consistency across the observed data at each iteration, we provided an index file containing the indices for the inner and outer folds. This ensured that all models were trained on the same data during each cross-validation fold, and the same test sets were used for evaluation. Additionally, we corrected for class imbalances within training folds with our evaluated parameter grid either with random over-sampling or by applying the class-balance loss (Cui et al. 2019) for the PyTorch models and with class-weights for the sklearn models. The comparison method MOMA was trained analogously with the same training loop and optimized according to the authors suggestions (see Sec. S3, available at supplementary material Bioinformatics Advances online). We evaluated model performance with the Matthews correlation coefficient (MCC), as it has been shown to be a better evaluation metric for binary classification tasks, especially when dealing with class imbalances (Chicco and Jurman 2020). We also report the area under the precision-recall curve (AUPRC), further evaluation metrics are listed in the supplement (Table ST1, available at supplementary material Bioinformatics Advances online). Table 2 details the median performances across all folds of both the sequence-based predictions (Sequences) and the combined approaches (Sequences + Features). For ClinIAN, we pre-trained TabNet within each CV fold on the prediction task and initialized ClinIAN with these learned weights. Table 2Comparison of the sequence-based mutation model (ClinIANseq) and ClinIAN against RFa, LinSVMb, PolySVMc, and the state-of-the-art method MOMA (Moon and Lee 2022).MCC [CI]AUPRC [CI] Mutations RF 0.14 [0.07, 0.48] 0.13 [0.11, 0.38] LinSVM 0.14 [0.07, 0.49] 0.13 [0.11, 0.38] PolySVM 0.14 [0.07, 0.49] 0.13 [0.11, 0.38] ClinIANseq 0.14 [0.07, 0.46]0.12 [0.11, 0.35] ClinIANseq; prott5 0.05 [-0.02, 0.29]0.11 [0.05, 0.26] ClinIAN –– MOMA –– ClinIANprott5 –– Mutations + Metadata RF 0.16 [0.09, 0.48]0.14 [0.11, 0.37] LinSVM 0.14 [0.05, 0.38]0.13 [0.09, 0.29] PolySVM 0.14 [0.05, 0.37]0.13 [0.09, 0.28] ClinIANseq –– ClinIANseq; prott5 –– ClinIAN 0.18 [0.10, 0.28]0.14 [0.12, 0.20] MOMA 0.05 [0.02, 0.14]0.11 [0.10, 0.13] ClinIANprott5 0.20 [0.09, 0.27] 0.15 [0.12, 0.18]We further evaluated ClinIAN’s ability to incorporate more complex pLM embeddings, using embeddings from prott5-xl-uniref50 (Elnaggar et al. 2021). Performances are reported as the median AUPRCd and MCCe, with confidence intervals (CIs), across the 10 × 10 nested CV iterations. ClinIAN and MOMA were only applied to the combined data, and ClinIANseq was only applied to mutations.aRandom Forest.bLinear SVM.cPolynomial SVM.dArea under the precision-recall curve.eMatthews correlation coefficient. Comparison of the sequence-based mutation model (ClinIANseq) and ClinIAN against RFa, LinSVMb, PolySVMc, and the state-of-the-art method MOMA (Moon and Lee 2022). We further evaluated ClinIAN’s ability to incorporate more complex pLM embeddings, using embeddings from prott5-xl-uniref50 (Elnaggar et al. 2021). Performances are reported as the median AUPRCd and MCCe, with confidence intervals (CIs), across the 10 × 10 nested CV iterations. ClinIAN and MOMA were only applied to the combined data, and ClinIANseq was only applied to mutations. Random Forest. Linear SVM. Polynomial SVM. Area under the precision-recall curve. Matthews correlation coefficient. 3.1 Implementation We implemented ClinIAN in Python using Pytorch (Paszke et al. 2019) (v2.5.1). The baseline models and pipeline utilities were implemented in Scikit-learn (v1.7.1) (Pedregosa et al. 2011). All models used the same training loop, implemented with skorch (Tietz et al. 2017), which provides an interface between PyTorch and Scikit-learn. We trained the PyTorch models with early stopping and a maximum of 200 epochs using the Adam optimizer (Kingma 2014). We used Pango Aliasor (v0.3.0) for the variant mapping. The code was executed on a single Nvidia GeForce GTX 1080 Ti GPU. 3.2 Quantifying classification performances First, we benchmarked ClinIANseq and the baselines using only sequences. These results are shown in the upper part of Table 2. We observed difficulties in predicting mortality from the sequences without additional information across all models. The median performances were low, with considerable confidence intervals (CI). We report the median, which is more robust against extreme outliers than the mean. These performances confirm the assumption that many other factors contribute to the progression of COVID-19. Therefore, it is preferable to combine the viral mutations with clinical parameters. In the lower part of Table 2 are the performance measures of ClinIAN against the baseline models on the combined features set. We observed that simply combining features did not improve the performance of SVM. While RF performed better with the combined features, ClinIAN outperformed the baselines in MCC and AUPRC. Additionally, ClinIAN exhibited greater performance stability than the other models, except for the state-of-the-art method MOMA, indicated by the smaller confidence interval. While the state-of-the-art method MOMA exhibited greater stability, it did not perform as expected. This suggests that in a prediction setting such as COVID-19, where numerous factors influence disease outcomes, it is beneficial to integrate both clinical metadata and mutations. ClinIAN’s performance could be further improved by utilizing the protein language model (pLM) embeddings from prott5-xl-uniref50 (Elnaggar et al. 2021) (ClinIANprott5). The observed increase in performance, while not statistically significant (Sec. S3.1, available at supplementary material Bioinformatics Advances online), suggests a more stable learning process. 3.3 Identifying predictive features The above comparative analysis demonstrated that ClinIAN exhibits greater performance stability than baseline approaches when addressing this challenging predictive task. Our approach outperforms the state-of-the-art method MOMA, although MOMA shows slightly more stability in terms of SD. A strong advantage of ClinIAN is its ability to derive insights directly from its predictions, thereby circumventing the need for post hoc explanations by integrating MIL attention layers. We visualized the feature importances (FI) by plotting the distance between each feature’s observed (a) and expected attention (E[a]), with FI=a−E[a]. We computed the expected attention for each sample assuming a uniform distribution, then averaged it across the entire dataset. Consequently, mutation positions and clinical features that were more frequently observed were allocated higher expected attention values (see supplement Sec. S3.2, available at supplementary material Bioinformatics Advances online). Mutation positions in proteins with more mutations than other proteins therefore received a lower expected attention due to the assumption of a uniform distribution. Additionally, we analyzed the FIs for both protein and clinical parameters. These FI maps facilitate a systematic exploration of our model, starting from the output layer and working backward to the individual features. We can use these FI maps to identify protein and mutation-level associations with the outcome. Figure 2A displays FI maps from ClinIANseq and ClinIAN highlighting the disparities in the proteins that are deemed significant by each model. We categorize the correctly classified samples into two groups: “survival” and “mortality.” ClinIANseq places considerable emphasis on the envelope protein (E) for both groups. Additionally, it assigns the membrane protein M great power in predicting survival. ClinIAN similarly assigns the M protein greater power in predicting survival, while also underscoring the significance of ORF3a and ORF6 in classifying survivability. Notably, ClinIAN places greater emphasis on ORF6 for patients who did not survive, diverging from ClinIANseq’s focus. Visualization of FI masks of ClinIAN and ClinIANseq across the subset of correctly classified samples. (A) Compares the identified FI across proteins. ClinIAN identifies more relevant proteins for host survival. (B) Shows the comparison of both model interpretations (left: ClinIANseq; right: ClinIAN) on the envelope (E) and membrane (M) protein. (C) Visualizes ORF3a and ORF6 attentions of ClinIAN. (D) shows identified clinical features of ClinIAN, emphasizing the number of vaccinations as an important predictive feature. For both models, we could dive deeper into decision-making by evaluating the FIs assigned to each mutation position. Figure 2B provides a comparative visualization of E and M for ClinIAN (right) and ClinIANseq (left), focusing on patients accurately classified as survivors. The envelope protein E only exhibits one mutation, T9I. We observed a consistency within FI, with the original amino acid T being important for predicting patients who passed away, and the substitution to I being weighted highly for surviving patients. The mutation T9I occurred between November 2021 and January 2022 and is indicative of Omicron variants. This specific variant has been associated to lower virulence (Xia et al. 2022). In both models, the membrane protein M receives significant FIs among surviving patients. In contrast, it is only weighted highly in patients who died by ClinIANseq. Both mutations Q19E and A63T are defining mutations for the Omicron variants (Hossain et al. 2022). I82T, with the original amino acid I being associated with host survival, has been identified prior as a minor mutation in the Delta variant, while the membrane protein, due to its low variability, has been suggested as a possible target for antibody development (Zhang et al. 2022). Figure 2C displays additional interpretations of ClinIAN for the identified proteins ORF3a and ORF6. The mutation T223I is associated with host survival, while the original amino acid seems to be associated with mortality. T223I has occured in Omicron variants, starting with BA.2 (Wang et al. 2023). ClinIAN also pinpoints the mutation at position 61 in ORF6 as important for identifying patients who passed away, with the original amino acid D predictive of mortality. The mutation D61L has been identified in the Omicron variants BA.2 and BA.4 and subvariants, as well as some recombinants (BN.1) (Hossain et al. 2022). Figure 2D displays the TabNet interpretations of ClinIAN for the same two cohorts. Our analysis indicates that receiving three or more vaccinations (“Vaccine 3 or more”) is a strong predictor of host survival. Conversely, individuals with zero vaccinations (“Vaccine 0”) are associated with increased mortality. Additionally, the variants D614G and Delta, alongside the years of patient’admission to the hospital 2020 and 2021, seem to be highly predictive of mortality. This is consistent, as both variants occurred in 2020 and 2021. Surprisingly, the absence of comorbidities, such as “no obesity” and “no diabetes,” is predictive of mortality within our cohort. Indeed, within our data, patients without these comorbidities tended to be less vaccinated than patients with these comorbidities. Along with prioritization in the vaccine distribution, one explanation could be a false sense of security among people without these comorbidities, leading them not to take the vaccine. This finding is consistent with the mean vaccination level among patients who passed away, which was 0.41 for patients without diabetes and 0.63 for patients with diabetes. Similar values were observed for obese patients. In addition to these findings, we validated the significance of the identified mutations on viral sequences shared via GISAID, the global data science initiative (Khare et al. 2021). All mutations discussed above show a highly significant association with the outcome, which validates the interpretative value of our proposed model. Details can be found in the supplementary materials Sec. S3.7, available at supplementary material Bioinformatics Advances online. We further validated that these correlations are indeed positive associations with the corresponding outcome label (see supplement Sec. S3.3, available at supplementary material Bioinformatics Advances online). In summary, our analysis identified several non-spike mutations that emerged in late 2021 and early 2022 and are typical of certain Omicron lineages and sublineages. Notably, these mutations are associated with survival in our models, corroborating existing knowledge of the lower virulence of Omicron variants. 4 Conclusion In this study, we introduce ClinIAN, an innovative and inherently interpretable end-to-end learning model that effectively combines clinical metadata and viral sequences. Our evaluation of ClinIAN underscores its potential in addressing the complex challenge of predicting COVID-19 disease severity by integrating viral sequence mutations with clinical features, an area typically associated with limited predictive performance. Our findings indicate that ClinIAN not only surpasses established baseline methods but also outperforms the contemporary state-of-the-art approach, MOMA. This demonstrates the model’s capacity to effectively leverage information from multiple data modalities, enhancing predictive accuracy. Moreover, the versatility of ClinIAN suggests its applicability extends beyond COVID-19, with potential utility in studying other viral diseases and parasitic conditions, such as malaria, as well as in cancer research. By integrating clinical insights with genetic data from pathogens or cancerous cells, ClinIAN opens new avenues for exploring disease mechanisms and outcomes. However, applicability in these other contexts would first require thorough evaluation and is only assumed, as the main limitations are the number of data points and the availability of tabular data with matching sequence data rather than the underlying data itself. Importantly, ClinIAN offers inherent interpretability of its decision processes across various hierarchical levels, highlighting the influence of specific proteins and mutation positions. The integration with TabNet adds a layer of interpretability and a way to correct for biases in mutation positions by utilizing expressive embeddings. This combination of modalities provides a deeper understanding of clinical features. Our decision-making analysis also confirmed the biological relevance of the interpretations provided by ClinIAN, thus highlighting it as a promising explorative tool in medical research. We acknowledge the limitations within our analysis. ClinIAN in its current form can not deal with missing data and will be further explored in the future. Excluding features with high missingness can limit the clinical applicability of our analysis. This was done to remove biases introduced by different imputation strategies, as our main objective was not to maximize performance but to provide a template for what an inherently interpretable model merging tabular clinical data and viral sequence data can look like while still delivering state-of-the-art performance. Introducing biases through imputation complicates this interpretation and requires a careful approach (Shadbahr et al. 2023). Nonetheless, we acknowledge that the assumption of no missing data does not necessarily hold true in clinical applications. Further, the complexity of predicting mortality, as reflected in the modest current absolute performance, makes its application in a clinical setting difficult. Still, combining tabular clinical features with viral sequences like this is a step in the right direction. Supplementary Material