Semantic Similarity-Informed Bayesian Borrowing for Quantitative Signal Detection of Adverse Events

François Haguinet, Jeffery L. Painter, Gregory E. Powell, Andrea Callegaro, Andrew Bate · 2025

Read Semantic Similarity-Informed Bayesian Borrowing for Quantitative Signal Detection of Adverse Events by François Haguinet, Jeffery L. Painter, Gregory E. Powell, Andrea Callegaro, Andrew Bate (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12052.txt.

Introduction

Francois Haguinet, Jeffery Painter, Gregory Powell, Andrea Callegaro, and Andrew Bate sharpen drug harm spotting in report banks in a 2025 paper. Spontaneous reports flood in after a drug launches, and teams must flag true harms fast. The standard count check treats each harm term on its own, which wastes what kin terms can teach.

Their method lends strength across terms by how close their sense sits, not just by fixed group lists. On United States report data from 2015 to 2019, the method caught more true harms than the standard check. It flagged them about five months sooner on average as well. Gains ran best in the early months past launch.

Catalog record

CREATORFrançois Haguinet, Jeffery L. Painter, Gregory E. Powell, Andrea Callegaro, Andrew Bate
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12052

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semantic Similarity-Informed Bayesian Borrowing for Quantitative Signal Detection of Adverse Events François Haguinet, Jeffery L. Painter, Gregory E. Powell, Andrea Callegaro, Andrew Bate Abstract We present a Bayesian dynamic borrowing (BDB) approach to enhance the quantitative identification of adverse events (AEs) in spontaneous reporting systems (SRSs). The method embeds a robust meta-analytic predictive (MAP) prior within a Bayesian hierarchical model and incorporates semantic similarity measures (SSMs) to enable weighted information sharing from clinically similar MedDRA Preferred Terms (PTs) to the target PT. This continuous similarity-based borrowing overcomes limitations of rigid hierarchical grouping in current disproportionality analysis (DPA). Using data from the FDA Adverse Event Reporting System (FAERS) between 2015 and 2019, we evaluate our approach - termed IC SSM - against traditional Information Component (IC) analysis and IC with borrowing at the MedDRA high-level group term level (IC HLGT). A reference set (PVLens), derived from FDA product label updates, enabled prospective evaluation of method performance in identifying AEs prior to official labeling. The IC SSM approach demonstrated higher sensitivity (1332/2337=0.570, Youden’s J=0.246) than traditional IC (Se=0.501, J=0.250) and IC HLGT (Se=0.556, J=0.225), consistently identifying more true positives and doing so on average 5 months sooner than traditional IC. Despite a marginally lower aggregate F1-score and Youden’s index, IC SSM showed higher performance in early post-marketing periods or when the detection threshold was raised, providing more stable and relevant alerts than IC HLGT and traditional IC. These findings support the use of SSM-informed Bayesian borrowing as a scalable and context-aware enhancement to traditional DPA methods, with potential for validation across other datasets and exploration of additional similarity metrics and Bayesian strategies using case-level data. keywords Pharmacovigilance, Semantic Similarity, Bayesian borrowing, Signal Detection, Drug Safety 1 Introduction Post-marketing safety surveillance of medicines and vaccines during routine healthcare delivery is essential for identifying emerging safety concerns and evaluating risks previously identified or hypothesized during development Croteau et al. (2022) . This process ensures that marketed products continue to reflect an accurate benefit-risk profile. Spontaneous reporting of Individual Case Safety Reports (ICSRs) has long served as the primary data source for detecting new safety signals CIOMS (2008) . While clinical review remains fundamental, the increasing volume and complexity of safety data have elevated the role of quantitative methods in pharmacovigilance (PV). Quantitative analysis of ICSRs was first proposed in the late 1960s and 1970s Finney (1974) ; Mandel et al. (1976) . Although early applications were sporadic and often limited to specific product issues Stricker and Tijssen (1992) ; Moore et al. (1997) , the late 1990s marked a turning point, with routine use of these methods driven by increasing data availability and advances in computation Noren et al. (2013) . While some quantitative signal detection approaches incorporate external data such as sales or prescription volumes to construct background rates Norwood and Sampson (1988) ; Praus et al. (1993) , the most widely used methods rely solely on ICSR data due to their practicality and timeliness Bate and Evans (2009) ; Noren et al. (2006) ; Norén et al. (2008) ; Chakraborty et al. (2022) ; Ding et al. (2020) ; Hu et al. (2015) ; Ahmed et al. (2009) 1 1 1 It should be noted that sales and prescription data are routinely used in other contexts, such as to establish denominators in PSURs and PBRERs; the distinction here is that they are less commonly used directly in disproportionality-based signal detection. . The standard methods—introduced in the late 1990s and early 2000s—are based on two-by-two contingency tables and remain the foundation of quantitative signal detection Bate et al. (1998) ; DuMouchel (1999) ; Evans et al. (2001) ; van Puijenbroek, E. P. et al. (2002) , despite their simplicity Dijkstra et al. (2020) . Although more complex techniques have since been proposed, they have not consistently outperformed these foundational methods in real-world settings Caster et al. (2010) . Notably, a multi-database comparison from the IMI PROTECT project showed that variation in signal detection performance across methods was often more influenced by the chosen detection threshold than by the specific algorithm employed Seabroke et al. (2016) . Nevertheless, disproportionality analysis (DPA) using these scores has notable limitations. The underlying data are noisy and subject to non-random missingness, and the independent treatment of adverse event (AE) terms limits their ability to capture clinically similar cases that may share an underlying mechanism Bate et al. (2012) . Addressing this limitation requires a more effective way to cluster related AEs for signal detection. The Medical Dictionary for Regulatory Activities (MedDRA) MedDRA (2020) is an international hierarchical terminology for coding AEs in clinical trials and post-marketing ICSRs. It facilitates consistency in AE classification and enables grouping of related events via its tree structure, which includes levels of specificity from Preferred Terms (PTs) to higher-level groupings. While useful, MedDRA’s rigid hierarchy often fails to capture nuanced clinical relationships between terms Brown (2003) . For instance, PT Eyelid oedema and PT Periorbital oedema are located far away from each other in the hierarchy. Another example is PT Pneumonitis NOS and PT Pneumonia NOS. Standard disproportionality methods treat MedDRA PTs independently, unless they are manually grouped under more general terms. This approach can lead to missed signals when related but distinct PTs are analyzed in isolation or to diluted signals when overly broad terms group unrelated AEs. Although general groupings like high-level group terms (HLGTs) or standardized MedDRA queries (SMQs) are sometimes used during post hoc clinical review, there is little evidence that higher-level aggregation improves algorithmic performance Hill et al. (2012) . As a result, most automated analyses rely on PT or Lower Level Term (LLT) granularity, both of which exist at the same hierarchical level within MedDRA Merrill (2008) . Bayesian borrowing offers a potential solution to this limitation Berry and Berry (2004) . Previous applications to ICSRs, particularly for vaccines, have demonstrated benefits when borrowing information across PTs within the same System Organ Class (SOC) Crooks et al. (2012) . However, these methods often rely on binary definitions of similarity: PTs are either allowed to borrow from each other or not. This binary framing fails to reflect the continuum of clinical relatedness among AEs. In reality, AEs vary in their informativeness for one another, and a continuous weighting scheme would allow more nuanced borrowing. Semantic similarity provides a mechanism for introducing such gradation, enabling context-sensitive borrowing across PTs. Although its application in PV remains limited, some related work has explored data-driven clustering of terms that frequently co-occur Orre et al. (2005) ; Erlanson et al. (2025) ; Chandler et al. (2017) . Bayesian dynamic borrowing (BDB) has gained traction in clinical trials for integrating historical control data based on similarity to current study data. Pocock Pocock (1976) pioneered this approach, which has since evolved into several frameworks, including the power prior Ibrahim and Chen (2000) , modified power prior Duan et al. (2006) ; Neuenschwander et al. (2009) , the Meta-Analytic Predictive (MAP) method Neuenschwander et al. (2010) , and the commensurate prior Hobbs et al. (2011) . Schmidli et al. Schmidli et al. (2014) enhanced the MAP method by introducing a robust MAP prior, incorporating a vague component to mitigate the impact of discordant historical data. This

Read the complete text