Contrastive learning of adverse events to provide effective and interpretable vector representations for machine-assisted pharmacovigilance Authors: Olivér M Balogh, Mátyás Pétervári, Áron M Csernák, Eszter Puhl, András Horváth, Péter Ferdinandy, Bence Ágg Published: 2026 ABSTRACT AbstractPost-marketing surveillance is crucial for drug safety, yet the tools of pharmacovigilance rely solely on text-based data that may limit contemporary machine learning methodologies in the support of decision-making. With the recent surge of employing large language models (LLMs) for text-based tasks, there also arises an unmet need for a different approach which is not grounded in the linguistic patterns of unfiltered natural text, like LLMs, but rather based on real-world drug safety data. Here, we adapt contrastive learning algorithms to generate adverse event vector representations from spontaneous adverse event reports to serve as machine-readable (i.e. numerical) resources for downstream pharmacovigilance applications, such as drug–event association prediction for signal detection or causality assessment. We present comprehensive interpretability analyses of the resulting representations through density-based clustering, semantic evaluation, and comparison of multivariate dispersions, revealing patterns that reflect both functional and causal relations of the adverse events while also capturing drug-safety-related information better than existing medical terminologies and encoder-only LLMs. Furthermore, we demonstrate the applicability of our representations as input features in our downstream classifier model, outperforming the reporting odds ratio method, commonly used by regulatory agencies, and also LLM-generated representations (area under the receiver operating characteristic curve: 0.88 versus 0.76–0.83) on drug–event association prediction benchmarks. Therefore, we propose an interpretable adverse event vector representation, serving as a general resource that could enable the development of a wide array of machine learning applications to support decision-making in pharmacovigilance and facilitate patient safety.Keywords: drug safety, adverse drug reaction, embedding, causality assessment, interpretable ai Introduction Adverse events are any harmful medical occurrences experienced during the administration of a medicinal product. When suspicion is conveyed about a possible causal relationship between an adverse event and a product, the terms adverse effect, adverse reaction, or adverse drug reaction (ADR) are used instead [1]. The role of pharmacovigilance is to minimize the risk-to-benefit ratio of drugs via monitoring their adverse reactions throughout both the development and post-marketing phases. Several adverse reactions appear only during the latter, recorded in spontaneous adverse event reports (AERs). AERs are collected for analysis by various pharmacovigilance entities, such as the EudraVigilance system of the European Medicines Agency (EMA) [2], the Adverse Event Reporting System of the United States Food and Drug Administration (FAERS) [3], and the VigiBase database of the World Health Organization [4]. AERs suggest only the potential for causality between drugs and adverse events (or equivalently in this case, adverse reactions), and thus they require careful and time-consuming causality assessment by experts [5] to distinguish coincidental observations from genuine ADRs before the appropriate regulatory action can be issued. Given the sheer volume of AERs (e.g. more than 2 million received by FAERS in 2024 [6]), there is an increasing need for efficient automation methods to either improve the identification of new safety concerns from the reports (a process called signal detection) or to accelerate the causality assessment of already suspected drug–event pairs. Within our framework, we define this supporting task as “drug–event association prediction,” where a positive association corresponds to a relationship substantiated by clinical or regulatory evidence rather than strictly proven causal effects. Drug safety professionals have expressed their keenness to engage with artificial intelligence systems in order to shift their volume-based tasks to a value-based work [7–11], and so, our aim here is to provide an interpretable resource that could enable the application of a wide array of machine learning models for drug–event association prediction, facilitating machine-assisted pharmacovigilance. A core limitation of handling AERs is their inherent text-based structure, lacking any informative, non-linguistic descriptor of the suspect drugs and the accompanying adverse events beyond their names, rendering contemporary machine learning models unreliable in providing evidence and supporting the decision-making of regulatory experts. For drug substances, several chemical and biological descriptors are available from external databases, such as DrugBank [12] or PubChem [13], that can provide numeric features for machine learning. There is, however, no similarly rich, machine-readable resource for adverse events. Representation learning could serve as a substitute by producing information-preserving embedding vectors; yet, to the best of our knowledge, no such data specifically made for general pharmacovigilance applications are readily available. Representation learning of medical concepts is, however, not a new field of study. In terms of embedding style, we can identify: (i) methods with traditional word2vec-like approaches [14–22], (ii) methods where knowledge graphs or term hierarchies are also constructed, resembling graph embeddings [22–26], and (iii) methods that are based on pretrained encoder-only large language models (LLMs) [26–33]. Regarding the application of these models in pharmacovigilance, one of the most frequent obstacles is that they embed on the level of various Unified Medical Language System subsets [16, 17, 19, 20, 22, 26] with only a partial mapping to the preferred terms (PTs) of the Medical Dictionary for Regulatory Activities (MedDRA®), which is the most commonly used terminology standard in pharmacovigilance [34]. For static embeddings (i.e. word2vec-like and graph-based ones), this obstacle introduces either a complicated task of needing to translate between terminologies, or significantly limits the number of embedding vectors readily available. For contextual embeddings produced by encoder-only LLMs, this problem is bypassed through subword- or character-level tokenization, and the use of attention. However, LLMs are rarely utilized to provide term embeddings in a static fashion, as the methods to do so (e.g. context aggregation, de-contextualization) have their own challenges and limitations, especially in terms of interpretability [26, 35, 36]. The second recurring aspect hindering the reuse of medical concept embeddings is that they are often learned through a very specific, non-generalizable task and are part of a larger end-to-end prediction model, with no description of their applicability in other downstream models. For example, encoder-only LLMs fine-tuned for language tasks, such as document classification, natural language inference, and named entity recognition, all belong to this category [27–33], as well as some of the word2vec-like models [18] and graph-based embeddings [23, 25]. The third notable limitation of reusing medical concept embeddings in pharmacovigilance is that most of them utilize natural text as their input, ranging from PubMed abstracts and full text articles [16, 17, 20, 22, 28–32] to clinical notes and electronic health records [16, 17, 27, 30, 31] to even insurance claims and social media posts [16, 17, 19, 29], making AERs an underutilized resource for representation learning, despite their relevance in pharmacovigilance. Lastly, availability is also an issue as some embeddings have simply not been made public [15, 21]. Motivated by these limitations of previous studies, we created our own representation learning framework that is based on a data processing module (Fig. 1a), handling raw FAERS reports, and an embedding module (Fig. 1b) responsible for learning the vector representation, i.e. embedding, of adverse events (from here on referred to as reactions) on the level of MedDRA® PTs. We developed two models: one based on normalized temperature-scaled cross entropy [37] (here referred to as NTX), and one based on the negative sampling skip-gram version of word2vec [38] (here referred to as NSG). In both cases, the embeddings are learned via noise-contrastive estimation [39] for which the reactions are first paired up with their context terms as observed in the AERs. We defined three different context-sampling approaches based on what is considered as context for a given “target” reaction: only drugs it was reported with (reaction2drug), only the other reactions it was reported with (reaction2reaction), or both (reaction2all), resulting in six different embedding sets overall. Comparing this framework to learning embeddings from natural text, the reports are treated similarly to the sentences of a corpus, and our context-sampling approaches function as skip-gram sampling methods with “dynamic” context window, operating within only the current sentence. As such, in our framework the distribution of context terms is influenced by the drug-safety-related behavior, i.e. reporting patterns, of the given reaction, rather than its occurrences in written language. Consequently, the resulting embeddings encode the pharmacovigilance profile of reactions, instead of their linguistic characteristics, which makes these embeddings not only effective descriptors for downstream applications, such as for machine learning models performing drug–event association prediction, but also semantically interpretable through various analysis methods. Schematic visualization of the main framework; (a) data processing: we accessed spontaneous AERs from the FAERS database, and used a combination of various naming dictionaries to identify the suspect drugs and reactions, mapping them into canonic DrugBank substance names and MedDRA® PTs, respectively; this step also acts as a quality filter, excluding poorly made reports; we performed three different context-sampling methods on the resulting report “corpus,” based on what is considered as context for a given reaction: the drugs it was reported with (reaction2drug), the other reactions it was reported with (reaction2reaction), or both (reaction2all); (b) embedding: the three types of target–context pairs were utilized as input data for our embedding models, which learn the vector representation, i.e. embedding, of the reactions through two different contrastive learning algorithms; the NTX model is based on the normalized temperature-scaled cross entropy loss, and performs multiclass classification on batches of target–context pairs, where each target is expected to be classified as belonging only to its corresponding context and vice-versa; the NSG model uses negative sampling, and performs binary classification on the target–context pairs (positive class) that are complemented with randomly sampled negative pairs (negative class); in both cases, the embedding models are discarded after training, and only their embedding matrices are kept for further tasks and analysis. Materials and methods For more details on the following subjects, see the extended methods description in Section 1.1–1.12 of the Supplementary. Data sources As outlined in Fig. 1a, raw AERs were accessed in XML format from the publicly available quarterly data extracts of the FAERS database, from the interval 2012/Q4—2023/Q1, totaling ˃15 M AERs [6]. Given that drug names are not standardized in FAERS, multiple dictionaries (DrugBank [12], RxNorm [40], and the Anatomical, Therapeutic and Chemical classification (ATC) [41]) were employed to identify the canonic DrugBank substance name of each drug, when possible, enabling us to later trivially link them to their molecular attributes in DrugBank [12] and PubChem [13] for the downstream drug–event association prediction model. As for reaction names, we used MedDRA®, which is an extensive, international medical terminology for the support of regulatory communication and data analysis in pharmacovigilance [34]. Validated (positive only) drug–ADR associations for dispersion analysis and training data were collected from three sources: The Side Effect Resource (SIDER) [42], the “PROTECT” project of the Innovative Medicines Initiative (IMI-PROTECT) [43], and the British National Formulary (BNF) [44]. Benchmark (positive and negative) drug–ADR associations for performance testing of the association prediction model were collected from another three sources: Exploring and Understanding Adverse Drug Reactions by integrative mining of clinical records and biomedical knowledge (EU-ADR) [45], Observational Medical Outcomes Partnership (OMOP) [46], and FDA revisions from the MedWatch website (FDA-rev) [47]. Data preprocessing We selected AERs only with the report type of “spontaneous,” which corresponds to those made by healthcare professionals, patients, and pharmaceutical companies, but excludes studies. The deduplication and the exclusion of AERs with no identifiable drugs/reactions resulted in about 10 M reports remaining in the report “corpus.” For the embedding models, AERs were reduced to simple lists of target–context pairs, where the target term is always a reaction and the associated context term is determined by the type of context-sampling (Fig. 1a): only drugs in the same AER (reaction2drug), only other reactions in the same AER (reaction2reaction), or both (reaction2all). The three validated drug–ADR datasets and the three benchmark sets were subjected to the same drug and reaction identification processes as the AERs, then merged separately, eliminating duplicate entries, to serve as the basis for a training and a test dataset for association prediction, respectively. Embedding Contrastive learning can be considered as a self-supervised approach, where the labels are not provided by an external source of positive–negative ground truth samples but rather determined in the training data internally, by a chosen association, like the spatial proximity or semantic similarity of the unlabeled samples [48]. Both our models (Fig. 1b) are a form of noise-contrastive estimation, where the associated word pairs (positive label) are contrasted with noisy random pairs (negative label). In particular, one of the models is inspired by the negative sampling skip-gram version of word2vec from natural language processing [38], here referred to as NSG, which treats co-occurring words as the positive samples and randomly selected non-co-occurring ones as negative samples, leading to a binary classification problem in practice. The other model is based on the normalized temperature-scaled cross entropy loss from computer vision [37], here referred to as NTX, which handles co-occurrence as a multi-class classification problem, expecting each word to be classified as belonging to only its current context out of all the other words present in the same batch. The NSG model training was done for 30 epochs (embeddings extracted after every fifth), using Root Mean Square Propagation (RMSProp) with learning rate η =0.01, smoothing parameter α =0.75, subsampling parameter t=0.00001, number of sampled negatives k=7, and eight positive target–context pairs per batch, thus a total batch size of 64. The NTX model training was done for 1000 epochs (embeddings extracted after every 100th), using RMSProp with learning rate η =0.01, temperature parameter τ =0.3, and a batch size of 16. For performance comparison, reaction embeddings were also created via various pretrained BERT-based models [30–33] by providing them as input the same list of reaction terms as used for the reaction2all embeddings (largest coverage). Each term was embedded separately, like a sentence, and mean pooling was applied to generate single static embeddings, which is a method known as de-contextualization [35]. It should be noted that the other option for creating static embeddings with encoder-only LLMs, namely context aggregation, could possibly lead to a stronger baseline by exploiting richer contexts. However, this method was shown to underperform as scaling the amount of context used can quickly saturate the embeddings and eliminate all preserved information [36]. Consequently, a rigorous fine-tuning would be required to determine how and which reports should be utilized for providing the best possible context for each LLM with each reaction term, which we leave for a possible future study. Interpretability analyses To characterize the interpretability of the reaction embeddings, we conducted three different analyses, consisting of visualizations (Fig. 2a), semantic analyses (Fig. 2b), and statistical evaluations (Fig. 2c). Summary of our evaluation methods and the data sources used to characterize the interpretability and applicability of the reaction embeddings; (a) we applied t-SNE for dimensionality reduction and HDBSCAN clustering to support the visualization by discovering potentially meaningful clusters of reactions; (b) by calculating various distance and similarity measures in the embedding space, we evaluated the semantic similarities of top 10 closest neighbors of a selected reaction across the representation spaces; (c) using validated drug–ADR data, we analyzed the multivariate dispersion of embedding vectors for reactions that belong to the same drug; dispersions of validated groups were statistically tested against randomly selected groups, and also compared to the dispersions measured in the MedDRA® hierarchical terminology network; (d) we constructed a deep neural network classifier to utilize the embedding vectors as input features, along with drug features from DrugBank and PubChem, for drug–event association prediction; the model was trained on the validated drug–ADR sources with various negative sampling approaches applied, and its performance was evaluated on established benchmark sets, comparing it with using embeddings generated by LLMs and the ROR statistical disproportionality analysis method. Visualizing the embeddings enables us to gain a general intuition about the underlying data structures and so, with the inherent limitations kept in mind, we applied t-distributed stochastic neighbor embedding (t-SNE) [49] to reduce the original representations into 2D. Supporting our visualization are the clusters deemed most prominent in this 2D space by the Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) [50], which we chose for its robust performance on variable densities and ability to prune off noisy elements. By performing clustering after dimension reduction, small patterns might emerge that are mere artifacts of the t-SNE optimization process; however, larger, well-separated clusters are still provably and consistently retrieved [51]. In our case, the resulting clusters were further examined according to their ratios of highest level MedDRA® categories, called System Organ Classes (SOCs), which are specific terms assigned to the reaction PTs (one primary SOC term to each). Semantic analysis was conducted by manually assessing the pathophysiological connections and clinical relevance of the closest neighboring reactions of a selected MedDRA® PT, “Myocardial Ischemia,” based on several international medical guidelines, categorizing them into one of four distinct semantic groups with MedDRA® SOC categorization as reference. Group 1 consists of synonyms, such as “Ischemia,” and the various signs/symptoms considered during the diagnosis of “Myocardial Ischemia” itself, like “Angina Pectoris” (chest pain) and “Electrocardiogram Abnormal.” Group 2 contains the most neighbors, corresponding to various pathophysiological risk factors, like “Arteriosclerosis Coronary Artery” (partial or complete obstruction in the coronary arteries) or “Sinus Tachycardia/Bradycardia” (heart beats faster/slower than normal), and possible complications of “Myocardial Ischemia,” such as “Arrhythmia” (rhythm abnormalities) or “Cardiac Failure” (impaired pump function). For Group 3, we selected reactions that share their pathophysiological cause with “Myocardial Ischemia,” and have identifiable common risk factors other than old age. For example, both “Peripheral Artery Disease” and “Myocardial Ischemia” can be caused by arteriosclerosis, with diabetes mellitus, hypertension, and dyslipidemia (abnormal levels of lipids in the blood) being well-known risk factors. Finally, Group 4 consists of reactions with semantic connections that are too indirect to be relevant. See Supplementary Table S8 for further details on the sources used for these categorizations. At the core of the statistical evaluation was a permutation test for homogeneity of multivariate dispersions that enables the use of arbitrary distance measures, and calculates group dispersion by considering the distances of samples from the estimated group center [52]. As described by Gijbels and Omelka [53], the test can be modified to use pairwise interpoint distances instead. This version of the test was not only used directly, but it also served as an inspiration for employing another dispersion difference measure [54], here simply termed “gain norm.” Calculation of the gain norm involves the difference of -normalized average pairwise interpoint distances, similar to the modified homogeneity test. With this measure, we evaluated how well reactions that are caused by the same drug separate from the background noise in the embedding space, as well as in the MedDRA hierarchical terminology network, enumerating the preserved drug-safety-related information. Association prediction model To demonstrate the real-life applicability of the reaction embeddings, we introduced a drug–event association prediction model, with a relatively simple deep neural classifier architecture (Fig. 2d). The input of this model consists of the embedding vector of the given reaction as well as the so-called chemical and biological feature vectors of the given drug [55]. The output, that is produced by sigmoid activation and optimized against the expected label through binary cross entropy loss, can be considered as the probability of the given drug and reaction being associated, either causally, functionally, or on a regulatory basis. Consequently, we emphasize that in the context of pharmacovigilance data, this association may also reflect strong co-reporting patterns and correlating regulatory warnings rather than strictly proven mechanistic causality, which is often subject to confounding and reporting biases. For the training data, we employed the validated drug–ADR associations, treating them as positive samples, and created negative pairs via various negative sampling (i.e. random recombination) techniques. Namely, we defined six methods according to which constituent of the positive drug–ADR pairs to keep (drug, reaction, none) and whether to sample the other constituent uniformly from the available terms or follow their frequency distribution as observed in the positive pairs. Hyperparameter search was performed in a 0.7–0.3 train-test split fashion. After retraining the final model on all data, performance testing was conducted using the withheld benchmark drug–ADR associations, averaged across 10 trainings. Performance was measured via area under the receiver operating characteristic curve (AUROC), as well as accuracy, and area under the precision–recall curve (AUPRC). Model training was done for 50 epochs, using stochastic gradient descent with learning rate η =0.001 and a batch size of 512. Software and hardware Data preprocessing was implemented in C++ 17 standard, while model training and result analysis were done using Python (version 3.10). The contrastive embedding and classifier models were implemented using Tensorflow/Keras (version 2.11) [56]. We used the scikit-learn (version 1.3) [57] implementations of t-SNE and HDBSCAN for dimensionality reduction and clustering analysis, respectively, and plotly (version 5.19) [58] for visualization. BERT-based pretrained LLMs were accessed from Hugging Face through the transformers package (version 4.48) [59], and used to generate embeddings (inference only) with Python (version 3.12) and PyTorch (version 2.6) [60]. Model trainings were performed on an NVIDIA A100 Tensor Core GPU, taking 1–3 h for the NTX models and 46–221 h for the NSG models, depending on the context-sampling method used. This marks a notable advantage in our NTX model as compared to the NSG model, in terms of efficiency. Classifier training took 1–2 h per run, depending on the negative sampling methods used. Results Visualization and clustering For a deeper analysis, we selected the NSG-reaction2all embedding, which was the best performing representation in the downstream drug–event association prediction task (Fig. 7). See Supplementary Figs S1–S5 for the others. The reaction2all-based embeddings have a perspicuous explanation, i.e. vectors will be close to each other for reactions that were reported with the same or similar drugs and reactions. For the NSG models specifically, a logarithmic frequency-based coloring (Fig. 3a) reveals a further, yet evident property: during training, the model encountered the often reported reactions much more frequently, even with subsampling applied, thus learning embedding vectors for them that are a lot more refined and pulled farther away from the randomly initialized values. We can observe the most frequently reported reactions congregating in the right side of the plot and on the right-facing edges of many of the visually well-separated clusters. These are general terms, such as “Drug Ineffective,” “Fatigue,” “Nausea,” “Pain,” “Headache,” and “Off Label Use.” At the left and left-center side of Fig. 3a, we can also observe a large, elongated group with low frequency, lacking a clear set of frequent reactions to gravitate toward. This group holds embedding vectors that had such a low occurrence during training that they barely moved from the noise initialization. Here, we have a diverse repertoire of niche reactions, such as “Vaccine Breakthrough Infection,” “Benign Rolandic Epilepsy,” “Nicotinamide Decreased,” “Mechanic’s Hand,” “Perthes Disease,” and “Coccidioides Encephalitis.” Drug–event association prediction results; performance of the classifier was tested by training it with no embeddings (learning them in-place), with using the presented six contrastive embeddings, and with using the four LLM generated embeddings; as an additional baseline, the performance of the ROR signal detection method was also calculated, and for further comparison, the results of two more end-to-end models are included, as reported by Joopudi et al. [25] (see the Discussion section); highest values of the present study are highlighted in bold, with one result being outperformed by previous studies, denoted by an asterisk; three different sources of benchmark data were utilized (here referred to as EU-ADR [45], OMOP [46], and FDA-rev [47]), from which a merged set was created, as the individual datasets had duplicate entries and varying sizes; Acc: accuracy; AUROC: area under the receiver operating characteristic curve; AUPRC: area under the precision–recall curve. T-SNE visualization of the embedding space generated by the NSG model with reaction2all context-sampling; (a) frequency-based coloring of the t-SNE plot, using occurrence counts from the processed AERs with a logarithmic scale for smoother color transitions; example reactions discussed in section Visualization and clustering have their positions indicated; (b) HDBSCAN-based coloring of the t-SNE plot, with noise removed for a better visual separation; (c) ratio of the top three MedDRA® SOCs in each selected cluster; selected clusters are numbered in descending order of their highest SOC ratio; SOC names are shortened to fit the figure by leaving out the words “disorders” and “conditions,” as well as the list of neoplasm types. Small-scale density differences are not preserved by t-SNE, but with the elimination of noise, HDBSCAN still proved successful in highlighting well-separated clusters in the dimension-reduced space for further analysis (Fig. 3b). For example, we can examine the composition of highest level MedDRA® categories, i.e. SOCs, in the resulting clusters to get an approximate idea of how closely the embedding space resembles them (Fig. 3c). Given that there are 27 distinct SOC terms, we selected only the top three with the highest ratios (with respect to the size of the cluster) in each cluster, and chose 15 clusters out of 161 in total, focusing on some of the larger and most representative ones. The first and second clusters are almost homogeneous, filled with reactions related to tumors (neoplasms) and blood tests, respectively, while the other clusters are more heterogeneous, with a pronounced plurality class. This is unsurprising, as reactions related to each other on the basis of prescribed therapies (and evidently, anatomical considerations) are often delegated into different SOCs, which is a phenomenon most prevalently caused by the umbrella SOCs, like those relating to investigations, injuries, infections, and procedures. For example, “Corneal Erosion” is categorized by MedDRA® SOCs as being primarily an eye disorder, while “Injury Corneal” is an injury and “Corneal Transplant” is a surgical procedure. Yet, the strong relation of these terms is reflected by the embedding space, with all three appearing close to each other in the sixth cluster, despite the embedding model lacking word and subword information (e.g. corneal) due to the term-level “tokenization.” This shows that the embedding space has a clear, interpretable structure that deviates from MedDRA® categories, but various anatomically and functionally consistent groups still emerge on the macro-scale. From the analysis of the NSG-reaction2drug embedding, which showed the highest gain norm values during the statistical evaluation (Fig. 5), almost identical conclusions can be drawn (Supplementary Fig. S1). Representation of how the “gain norm” was calculated and used; (a) the concept of (unnormalized) gain: for each drug, we collected their validated reactions from known drug–ADR association data (validated group), and randomly sampled an equal number of other reactions (random group); the within-group average interpoint measures are plotted for both group types, as a function of group size, where the scale of the Euclidean, Bray–Curtis, and Canberra distances are negated so that a higher value on the y-axis means more closely related groups on all plots; gain is calculated as the average difference between the validated and the random groups; (b) changes in the gain norm (normalized gain) as seen throughout the training of the NSG-reaction2drug embedding; we utilized the gain norm as a heuristic method for selecting the best epoch for each embedding; highlighted here is the epoch selected for NSG-reaction2drug, from which data is visualized in panel (a) as well; (c) summary of the highest scoring embedding results and their embedding-based gain norm values; highlighted are the results belonging to the NSG-reaction2drug embedding, used as an example in panel (a) and (b); MedDRA-based gain norms were calculated by using hierarchical similarity measures on the MedDRA® terminology network, which is independent of the embeddings, and so the results are summarized separately. Top 10 closest neighbors Here we present a brief semantic analysis through the example of a MedDRA® PT, “Myocardial Ischemia,” and its 10 closest neighbors across the representations to provide a closer insight into the local patterns of the embedding spaces without resorting to dimensionality reduction (Fig. 4 and Supplementary Tables S1–S7). We selected “Myocardial Ischemia” as the focus of this evaluation because of its relevance to global health [61, 62], rich literature [63], and connection to our previous studies [64, 65]. Lists of the top 10 closest neighbors of the MedDRA® PT myocardial ischemia across the six reaction embeddings, measured by cosine similarity in the original vector space, colored (a) by semantic evaluation based on expert knowledge and literature, and (b) by primary MedDRA® SOC. We can observe several different tendencies between the embeddings of the two models. NSG is consistent in emphasizing closely connected concepts across the three context-sampling methods, with distinguishable preferences among them: reaction2drug prioritizes the most well-known complications and symptoms, reaction2reaction leans toward terms describing impaired pump function and heart rhythm problems, and reaction2all results in a balanced set of the two. In contrast, NTX embeddings resemble such patterns only with reaction2reaction, indicative of its downstream performance (Section Association prediction), while the other two sampling methods lead to several terms that share pathophysiological causes with “Myocardial Ischemia,” like “Polydipsia” (excessive thirst) or “Diabetic Ketoacidosis” (acidification of blood due to lack of insulin), which are typical symptoms of untreated diabetes mellitus. Furthermore, many indirect entries appear as well, suggesting patient-specific connections that are less based on pathophysiology and more on circumstantial factors. For example, “Subdural Hematoma” (bleeding in the skull) might occur from falling or collapsing caused by hemodynamic instability, and “Aspiration” is common in patients who require intensive care due to an acute exacerbation of “Coronary Artery Disease.” Overall, this analysis demonstrates how pathophysiological and clinical properties of various diseases (reactions) can be captured purely from pharmacovigilance data by our proposed representation learning framework, encoding them on the micro-scale, in local neighborhoods. Dispersion of related reactions So far, we observed that the embedding vectors tend to cluster differently than MedDRA® SOCs (Fig. 3), yet their immediate neighbors display strong semantic relations (Fig. 4). For pharmacovigilance purposes, we would expect the representations to also preserve drug-safety-related information, which we measure by using curated datasets of drug–ADR associations [42–44], considering them as pairs in which a strong positive relationship is validated (from here on referred to as validated data). A fair assumption would be that reactions caused by the same drug (a “validated group” of reactions) share more similarities than unrelated reactions (a “random group”), thus should be more closely packed together (i.e. less dispersed) in the embedding space. The statistical test for the homogeneity of dispersions between these validated and random groups showed that above 95% of the validated groups are significantly less dispersed than their corresponding random groups, in almost all embeddings (Supplementary Table S9), which motivated us to switch to a finer measure of separation, here referred to as “gain norm.” We expect the gain norm to be large for embedding spaces that well preserve drug-safety-related information (i.e. a large separation of validated groups from background noise), therefore we also utilized it as a heuristic measure for tuning the hyperparameters of our embedding models, most importantly the number of epochs for selecting the final checkpoints (Fig. 5b and Supplementary Figs S6–S10). Through node-based, hierarchical semantic similarity measures [66], we can calculate the gain norm for the MedDRA® terminology network as well. Comparison to the embedding-based gain norms (Fig. 5c) reveals a stark difference in the distinguishability of these validated groups from the background noise. While the embedding-based values consistently reach 0.25–0.3 across all measures (except Euclidean), going as high as 0.46, the MedDRA-based ones are all in the range of 0.02–0.04. Distributions plotted in Fig. 5a reveal that the MedDRA-based measures are hindered by the larger validated groups, which blend in with the random groups. An exception is the Jiang–Conrath measure that shows a clear separation, albeit with a very small margin. As such, this evaluation demonstrates that even though the embedding vectors are organized into a different structure as compared to the MedDRA® terminology network, drug-safety-related information is better reflected by their emerging patterns, correlating with real-life pharmacovigilance data, which could provide a more suitable resource in practice. Association prediction As the focal point of our work, here we present an application of the embedding vectors as input features for a deep neural network model (Fig. 2d) performing drug–event association prediction. The previously used validated drug–ADR datasets [42–44] provide us only with positively labeled associations, which are insufficient for supervised training. Although there are datasets with negative pairs [45–47], they are substantially smaller in scale and utilized mostly as benchmarks during performance evaluations. We followed this trend, resorting to synthetic negative data for training, which can be difficult to create, as easy negatives will lead to an overestimated model performance during validation but poor generalization when tested on real samples. Consequently, the data generated by different negative sampling methods cannot be used reliably in a train-validation split to determine the best-performing methods themselves, without using external testing data. Due to the aforementioned benchmark datasets being already heavily limited, we chose not to allocate parts of them for this purpose, and instead we evaluated the final model with all six negative sampling methods (Fig. 6). The average training loss revealed that the most difficult training set was the one that also led to the best performance (highlighted with a blue frame in Fig. 6), demonstrating how robust generalization is best achieved with hard negatives. Additional trainings for the following comparisons were done only with this negative sampling method. See the Discussion and Section 8 of the Supplementary for further assessment on the performance-influencing effects of the various types of negative sampling approaches. Performance of the drug–event association prediction model with the six different embeddings as reaction features, with the six different negative samplings, tested on benchmark datasets from three sources (here referred to as EU-ADR [45], OMOP [46], and FDA-rev [47]); highest performance metrics for each dataset are highlighted in bold, and the overall best performing negative sampling method in highlighted with a blue frame; Acc: accuracy; AUROC: area under the receiver operating characteristic curve; AUPRC: area under the precision–recall curve. As a simple ablation study, we trained a version of the classifier that receives only the one-hot vector representation of reactions and learns the embeddings “in-place,” during training, so that the added benefit of using separately created representations can be observed. The results (Fig. 7) clearly indicate one of the primary detriments of this approach, i.e. that many of the reactions in the benchmark datasets may not appear in the training data, making it impossible for the model to generalize to unseen samples, leading to a random classifier (AUROC ~0.5) in practice. This necessitates the use of separately learned reaction embeddings. To position our work in the larger scope of the modern representation learning landscape, we also trained versions of the same classifier using embedding vectors that are generated by various state-of-the-art BERT-based models, fine-tuned for clinical and biological language tasks (PubMedBERT [32], BioMed-RoBERTa [33], Clinical ModernBERT [31], and BioClinical ModernBERT [30]). Although encoder-only LLMs are rarely employed to create static embeddings (as discussed in the Introduction section), their usage here demonstrates how natural-language-informed reaction embeddings compare to those trained on structured pharmacovigilance data. As the overall “Merged” results show (Fig. 7), the NSG-variant contrastive embeddings all outperformed the LLM-based ones, while the NTX-variants outperformed one of them. However, dispersion analysis of these LLM-based representation spaces, as done in Section Dispersion of related reactions, reveals a very poor separation of validated groups from the background noise (Supplementary Table S9), suggesting that they are unable to capture interpretable drug-safety-related information and encode patterns that are harder to decipher. As the final baseline, we calculated the performance of the reporting odds ratio (ROR), which is a statistical disproportionality measure and the cornerstone of the signal detection method prominently used and recommended by EMA [67]. With only two exceptions (AUPRC for NTX-reaction2drug and NTX-reaction2all), all six contrastive embeddings resulted in the classifier outperforming the ROR method when evaluated on the merged benchmark set (Fig. 7), which demonstrates the ability of the embeddings to serve as a general resource in the support of decision-making, improving the causality assessment process done by experts. Comparison between our embeddings themselves shows the performance-influencing effect of the various context sampling methods and model choices. The reaction2reaction sampling performed better for the NTX model than both reaction2drug and reaction2all, matching their semantic evaluation trends (Fig. 4a). From this, we can conclude that reaction2reaction is the most suitable sampling method for the NTX model, possibly because it resembles the original use case of such architectures the most, where target–context pairs consist of augmented versions of the same images. On the other hand, the reaction2all sampling method was not only the best-performing NSG-variant but it also resulted in the overall best classification results across all methods (Fig. 7). We can observe a clear increase in the performance metrics going from reaction2drug to reaction2reaction to reaction2all for the NSG model, which might be explained by the available information (number of context terms) also increasing in this order. In general, embeddings made by NSG performed better than their NTX counterparts, which suggests that the N-pairs-based noise contrastive technique employed by NTX is less suited for learning effective embeddings from AERs as compared to the negative sampling noise contrastive technique of NSG. As a final analysis, we examined the temporal robustness of the best-performing embedding (NSG-reaction2all) by training it with incrementally more AERs, based on date, and testing the resulting new embeddings with our drug–event association prediction model the same way as with the other embeddings. Results showed (Supplementary Fig. S11) an already high baseline with the smallest of splits and also a steady increase with more data. See Section 9 of the Supplementary for our further assessment. Discussion Here, we present a framework to generate vector representations of adverse events from spontaneous AERs to offer an interpretable and effective resource for general machine learning applications in pharmacovigilance. A comprehensive interpretability analysis of the results was provided, revealing numerous mathematical and semantic patterns, which imply high levels of preserved, drug-safety-related information, with anatomical and pathophysiological patterns emerging. To showcase a possible application of the embedding vectors and demonstrate their effectiveness as input features, we presented a deep neural classifier performing drug–event association prediction. We trained the model on data that was independent of AERs used for the embeddings, as not to reuse them and to avoid some of the complications arising if we labeled parts of the reported pairs as positives and others as negatives via some mathematical or other unfounded consideration. An example with this issue would be the Transformer-based Hierarchy-aware Adverse Reaction Embeddings (T-HARE) model introduced by Joopudi et al. [25], which is a type of representation learning we already discussed in the Introduction section, one that learns and uses the embeddings for a specific end-to-end prediction task instead of general purpose. For its training, a bipartite graph from FAERS data was constructed, and an edge-sampling method was utilized to select positive and negative pairs, based on their drug-normalized reporting frequency. Similar pharmacovigilance networks can indeed be used to perform signal detection, as shown by Pétervári et al. [68] and Barbieri et al. [69]. However, the edge weights that are based on report counts need to be properly normalized via network topological features even for outperforming the ROR method [68], suggesting that simpler normalization techniques might be inadequate for creating ground truth training data for machine learning. This limitation was indeed observed by Joopudi et al. during the training of T-HARE, which they remedied by scaling the loss on negative samples by their normalized pointwise mutual information. Another notable case is the aer2vec model introduced by Portanova et al. [18], which is a similar negative sampling skip-gram version of word2vec as our NSG model; however, it was utilized to perform association prediction directly, after training it on FAERS data. The first issue with this approach is that such shallow, noise contrastive embedding models are not meant to perfectly optimize their training objective, making them a suboptimal choice for this purpose. The other possible issue is the same as with T-HARE, that given its training regimen, the model might learn that all frequently reported pairs are positive, while only the rarely or never reported noisy pairs are negative, which might fail to be of any use in practice. Instead of relying on AERs for the training data to our drug–event association prediction model, we utilized data where the drug–event pairs are validated to have strong positive associations [42–44]. Although these same three validated datasets were used for the dispersion analysis of related reactions, they presented a fundamentally different objective there, i.e. a heuristic measurement of the interpretable drug-safety-related information. This difference is best showcased by the LLM-derived embeddings achieving relatively comparable results to our contrastive embeddings on the association prediction task (Fig. 7) but orders of magnitude worse gain norm values during dispersion analysis (Supplementary Table S9). Further deviating from this task, we generated additional negative associations for the supervised training of our classifier by randomly recombining the established positive pairs. Here, we also encountered the problem of easy negatives. We observed that the main contributor to the varying performance of the classifier was whether we sampled the other constituent of the negative pair uniformly or proportionally to its occurrence frequency (Fig. 6). Sampling uniformly made it too easy for the model to separate the uniform negatives from the heavily skewed positives, which resulted in high performance metrics during training but low values during testing. Matching the frequency of the constituents created a more balanced dataset and an overall harder training task (as shown by the loss values in Supplementary Table S10), leading to better generalization capabilities. The best performance coincided with the hardest negative sampling approach, where by keeping the reactions and sampling the drugs by their frequency, the distributions of reactions on the positive and negative sides remain the exact same, enforcing the model to pay more attention to how the reaction embedding vectors and the drug feature vectors interact. The results demonstrate the effect of strong, hard negative samples on model generalizability and thus how much they are required for real-world deployment. As outlined in the recently published 2025 Artificial Intelligence in Pharmacovigilance report, by the Council for International Organizations of Medical Sciences [70], real-world application is highly context-dependent and the impact of dataset limitations heavily relies on the specific objective. For instance, detecting low-prevalence adverse events requires fundamentally different benchmark statistics than training a mechanistic pharmacological model, where low-prevalence values might actually be excluded. Thus, future validation steps must extend beyond small-scale, general benchmarking, and focus on task-specific, expert-curated regulatory data distributions that can properly reflect the challenges of pharmacovigilance. This leads us to the most important limitation of studies dealing with machine-assisted pharmacovigilance, one that plagues countless other biomedical research areas as well: the lack of proper gold standard benchmark data. Even with our diverse selection of sources (EU-ADR [45], OMOP [46], and FDA-rev [47]), we faced the issue of having a very small test dataset, which imposes serious limitations on estimating real-life performance for association prediction models. As there are no comprehensive, large-scale benchmark data with curated positive and negative drug–event associations, and many of the existing smaller sets are either unavailable [71] or heavily restricted in scope [72, 73], we resorted to some of the more commonly used datasets, enabling comparison with results from other studies (Fig. 7). However, a closer look into these datasets reveals that they also contain varying degrees of easy negatives, which is also reflected in the performance of the discussed methods. For example, in the EU-ADR dataset, negatives are defined as pairs without MEDLINE citations and AERs in VigiBase, making it the easiest dataset to predict, as also seen in the associated AUROC values (ROR: 0.9094, aer2vec: 0.9120, T-HARE: 0.9230, and NSG-reaction2all: 0.9213). The OMOP dataset, on the other hand, was sourced via rigorous literature review and specifically aimed at providing a broad set of positive and negative controls. This is well reflected in the AUROC values (ROR: 0.7096, aer2vec: 0.8210, T-HARE: 0.8430, and NSG-reaction2all: 0.9098) in which our model achieved a significant improvement compared to the other methods, potentially due to its training data being independent from AERs. Finally, in the FDA-rev dataset, negative controls were the result of a nearly identical negative sampling approach to what we used for generating our training data: aiming at a balanced distribution of drugs and reactions on both the positive and negative side while adhering to known associations. This led to the hardest negative samples and arguably the most difficult benchmark dataset, on which even our best embedding was slightly outperformed by the ROR method (AUROC: 0.7544 versus 0.7592). Low performance might also be attributed to the FDA-rev positive pairs containing not just validated reactions but “warnings and precautions” as well, which are significant safety hazards with or without established causality (see Supplementary Table S11 for our assessment). There are no performance measures available on FDA-rev data for the other methods. Conclusions In conclusion, we advance machine-assisted pharmacovigilance by offering a versatile and interpretable vector representation of adverse events as a resource that could enable the development of a wide array of machine learning applications to support decision-making and improve patient safety. Key Points Assessing the potential association between drugs and adverse events is a central, yet laborious task of drug safety. However, supporting the decision-making process of experts by machine learning during this causality assessment is hindered by the text-based nature of the adverse event reports (AERs) and the unreliability of artificial intelligence models that operate purely by linguistic information, such as large language models (LLMs). Assessing the potential association between drugs and adverse events is a central, yet laborious task of drug safety. However, supporting the decision-making process of experts by machine learning during this causality assessment is hindered by the text-based nature of the adverse event reports (AERs) and the unreliability of artificial intelligence models that operate purely by linguistic information, such as large language models (LLMs). Here, we present a representation learning framework utilizing contrastive learning algorithms that generate adverse event embeddings from structured AERs, reflecting the real-world drug safety profile of the adverse events rather than their linguistic characteristics as inferred by LLMs from natural text. Here, we present a representation learning framework utilizing contrastive learning algorithms that generate adverse event embeddings from structured AERs, reflecting the real-world drug safety profile of the adverse events rather than their linguistic characteristics as inferred by LLMs from natural text. Through various interpretability analysis methods, like visualization, clustering, semantic analysis, and statistical evaluations, we demonstrate the preserved drug-safety-related information and emerging pathophysiological patterns of the adverse event embeddings. By employing the embeddings as input features in a drug–event association prediction model, we demonstrate their real-life applicability as a general resource for downstream tasks, outperforming LLM-based embeddings. Through various interpretability analysis methods, like visualization, clustering, semantic analysis, and statistical evaluations, we demonstrate the preserved drug-safety-related information and emerging pathophysiological patterns of the adverse event embeddings. By employing the embeddings as input features in a drug–event association prediction model, we demonstrate their real-life applicability as a general resource for downstream tasks, outperforming LLM-based embeddings. We offer publicly available code and the accompanying data, both input and output, of our representation learning framework at deepvigilace.com. We offer publicly available code and the accompanying data, both input and output, of our representation learning framework at deepvigilace.com. Supplementary Material Acknowledgments We thank Dóra Bihary for providing valuable feedback about the grammatical correctness and language of the manuscript. Contributor Information Olivér M Balogh, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; HUN-REN-SU System Pharmacology Research Group, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., H-1089 Budapest, Hungary; Pharmahungary Group, Szeged, Hajnóczy utca 6., H-6722 Szeged, Hungary. Mátyás Pétervári, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; HUN-REN-SU System Pharmacology Research Group, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., H-1089 Budapest, Hungary; Sanovigado Kft, Somfa köz 11., Somfa köz 11., H-1107 Budapest, Hungary. Áron M Csernák, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary. Eszter Puhl, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary. András Horváth, Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter utca 50/A., H-1083 Budapest, Hungary. Péter Ferdinandy, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; HUN-REN-SU System Pharmacology Research Group, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., H-1089 Budapest, Hungary; Pharmahungary Group, Szeged, Hajnóczy utca 6., H-6722 Szeged, Hungary. Bence Ágg, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; Center for Pharmacology and Drug Research and Development, Semmelweis University, Üllői út 26., Üllői út 26., H-1085 Budapest, Hungary; HUN-REN-SU System Pharmacology Research Group, Department of Pharmacology and Pharmacotherapy, Semmelweis University, Üllői út 26., H-1089 Budapest, Hungary; Pharmahungary Group, Szeged, Hajnóczy utca 6., H-6722 Szeged, Hungary. Author contributions Olivér M. Balogh (Conceptualization [equal], Methodology design [equal], Coding [equal], Model implementation, Data analysis [equal], Visualization [equal], Paper preparation—original draft [equal]), Mátyás Pétervári (Conceptualization [equal], Data analysis [equal], Domain expertise [equal], Paper preparation—revision [equal]), Áron M. Csernák (Coding [equal], Data analysis [equal], Domain expertise [equal], Visualization [equal], Paper preparation—original draft [equal]), Eszter Puhl (Data analysis [equal], Domain expertise [equal], Paper preparation—revision [equal]), András Horváth (Methodology design [equal], Paper preparation—revision [equal]), Péter Ferdinandy (Project supervision [equal], Domain expertise [equal], Paper preparation—revision [equal]), and Bence Ágg (Project supervision [equal], Conceptualization [equal], Methodology design [equal], Paper preparation—revision [equal]) Conflicts of interest P.F. is the founder and CEO of, and B.Á. and O.B.M. are employed by Pharmahungary Group, a group of R&D companies. M.P. is the founder and CEO of Sanovigado Kft, a pharmaceutical consultancy and R&D company. Funding Project No. RRF-2.3.1-21-2022-00003 has been implemented with the support provided by the European Commission, European Union. This project has received funding from the HUN-REN Hungarian Research Network and the National Research, Development and Innovation Office of Hungary: 2020-1.1.6-JÖVŐ-2021-00013. The research was financed by the Thematic Excellence Program (2020-4.1.1.-TKP2020) of the Ministry for Innovation and Technology in Hungary, within the framework of the Therapeutic Development and Bioimaging thematic programs of the Semmelweis University. This study was supported by the Semmelweis 250+ Excellence Fellowship (O.M.B. and E.P.). This study was supported by the 2024-1.2.3.-HU-RIZONT-2024-00026 (P.F.), the 2024-2.1.1-EKÖP-2024-00023, and the 2025-2.1.1-EKÖP-2025-00014 New National Excellence Program (O.M.B.) of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovation Fund. This project has been implemented with the support provided by the European Union and co-financed by the Hungarian Government under the Digital Renewal Operational Program Plus (DROP Plus), project ID: DIMOP_PLUSZ-1.1.2/B-24-2025-00130. Data availability All the data files for training our embedding models and the drug–event association prediction model, as well as for performing clustering is available at deepvigilace.com, namely the preprocessed report corpus made from FAERS AERs, the derived drug and reaction frequencies, the preprocessed validated and benchmark drug–ADR data, the collected drug feature vectors, the presented reaction embedding vectors, and their 2D t-SNE coordinates. In addition, the complete results of the HDBSCAN clustering, the raw top-neighbor tables for “Myocardial Ischemia,” and the prediction results made by the ROR-based signal detection method and our best classifier on the benchmark sets were also made available, along with the best performing classifier model itself. Interactive HTML visualizations are provided for the t-SNE and clustering results, enabling manual data analysis and browsing. Due to licensing, MedDRA® terminology data are not provided, which are needed for top 10 closest neighbor and dispersion re-calculations with the hierarchical semantic similarity measures. Other supporting data files are available upon reasonable request to the authors. Code availability All code necessary for replicating the results of this study from the provided preprocessed adverse event reporting data, i.e. Python code for generating, analyzing, and using the reaction embeddings, is available under the MIT license at github.com/semmelweis-pharmacology/deepvigilace. References Associated Data Supplementary Materials Data Availability Statement All the data files for training our embedding models and the drug–event association prediction model, as well as for performing clustering is available at deepvigilace.com, namely the preprocessed report corpus made from FAERS AERs, the derived drug and reaction frequencies, the preprocessed validated and benchmark drug–ADR data, the collected drug feature vectors, the presented reaction embedding vectors, and their 2D t-SNE coordinates. In addition, the complete results of the HDBSCAN clustering, the raw top-neighbor tables for “Myocardial Ischemia,” and the prediction results made by the ROR-based signal detection method and our best classifier on the benchmark sets were also made available, along with the best performing classifier model itself. Interactive HTML visualizations are provided for the t-SNE and clustering results, enabling manual data analysis and browsing. Due to licensing, MedDRA® terminology data are not provided, which are needed for top 10 closest neighbor and dispersion re-calculations with the hierarchical semantic similarity measures. Other supporting data files are available upon reasonable request to the authors.