Application of the U.S. Food and Drug Administration’s Sentinel Routine Querying Tools to the Taiwan Sentinel Data Model-formatted National Health Insurance Research Database Authors: Fang-Ju Lin, Ling-Ya Huang, Chi-Chuan Wang, Sengwee Toh Published: 2023 ABSTRACT AbstractThe U.S. Food and Drug Administration’s Sentinel System is a leading distributed data network for drug safety surveillance in the world. The National Health Insurance Research Database (NHIRD) in Taiwan was converted into the Taiwan Sentinel Data Model (TSDM) based on the Sentinel Common Data Model (SCDM) version 6.0.2. The goal of this study was to investigate the feasibility of applying the same study designs, analytic choices, and analytic tools as used by the U.S. Sentinel System to examine the same drug–outcome associations in the TSDM-formatted NHIRD. Four known drug–outcome associations previously examined by the U.S. Sentinel System were selected as the use cases: (1) use of angiotensin-converting enzyme inhibitors (ACEIs) and risk of angioedema, (2) use of warfarin and risk of gastrointestinal bleeding, (3) use of oral clindamycin and risk of Clostridioides difficile infection (CDI), and (4) use of glyburide and risk of serious hypoglycemia. We followed the same study designs and analytic choices used by the U.S. Sentinel System and applied the Sentinel Routine Querying Tools to answer the same study questions within the TSDM-formatted NHIRD. The results showed that ACEIs were associated with a non-significant increase in risk of angioedema compared to beta-blockers (hazard ratio [HR]: 1.21; 95% confidence interval [CI]: 0.89–1.64); warfarin was associated with a higher risk of gastrointestinal bleeding compared to statins (HR: 1.72; 1.50–1.98); glyburide was associated with an increased risk of hypoglycemia compared to glipizide (HR: 1.61, 1.30–2.00). We were unable to evaluate the association between oral clindamycin and risk of CDI due to the low event number. Our study demonstrated that it was feasible to directly apply the publicly available Sentinel Routine Querying Tools within the TSDM-formatted NHIRD. However, sources of heterogeneity other than design and analytic differences should be carefully considered when comparing the results between the two systems.Keywords: Pharmacovigilance, Routine Querying Tools, Sentinel Initiative, Taiwan Sentinel Data Model 1. Introduction In 2016, the U.S. Food and Drug Administration (FDA) officially launched the Sentinel System, an active surveillance system to monitor the post-market safety of medical products [1–3]. The Sentinel System is a distributed data network of more than a dozen data partners, all of whom have converted their source data into the Sentinel Common Data Model (SCDM) to allow large-scale assessment of medical product safety across multiple databases [4]. To facilitate rapid query and analysis across the databases, the Sentinel System has developed a suite of Routine Querying Tools that can be run against the SCDM [5]. The Routine Querying Tools include semi-automated analytic programs that can be customized according to specific study designs, exposures, outcomes, and covariates. These tools can be used for cohort identification, descriptive analysis, and complex confounding-adjusted inferential analysis. Currently, there are four levels of analyses available: signal identification, descriptive analysis, retrospective inferential analysis, and prospective sequential inferential analysis [6]. To test the validity of the Routine Querying Tools, four known positive drug exposure-outcome associations were selected as use cases within Mini-Sentinel (the pilot program of the Sentinel System established in 2009): (1) use of angiotensin-converting enzyme inhibitors (ACEIs) and risk of angioedema [7], (2) use of warfarin and risk of gastrointestinal (GI) bleeding [8], (3) use of oral clindamycin and risk of Clostridioides difficile infection (CDI) (C. difficile was formerly known as Clostridium difficile) [9], and (4) use of glyburide and risk of serious hypoglycemia [10]. The Cohort Identification and Descriptive Analysis (CIDA) and the Propensity Score Analysis (PSA) modules from the Routine Querying Tools were tested in these four use cases. Following the SCDM structure, we previously built the Taiwan Sentinel Data Model (TSDM) using the longitudinal data from the National Health Insurance Research Database (NHIRD) [11]. This is a U.S. Sentinel System-compatible platform which allows us to directly adapt the U.S. Sentinel Routine Querying Tools and run them against the data in Taiwan. The goal of this study was to investigate the feasibility of applying the same study designs, analytic choices, and analytic tools as used by the U.S. Sentinel System to examine the same drug exposure-outcome associations in the TSDM-formatted NHIRD. All four known positive drug exposure-outcome associations tested in the Mini-Sentinel program were tested in the present study. 2. Methods 2.1. Data source The TSDM-formatted NHIRD between 1 January 2011 and 31 December 2017 served as the data source for this study. Details of the TSDM are described elsewhere [11]. Briefly, the TSDM contains 11 tables: Enrollment, Demographic, Dispensing, Encounter, Diagnosis, Procedure, Death, Cause of Death, Laboratory Result, Vital Signs, and Inpatient Pharmacy. Laboratory results, vital signs, and inpatient pharmacy data are available only in a subset of population with linked electronic health records. 2.2. Use cases Because data for specific time periods used in the original Sentinel analyses were not available in the NHIRD at the time we conducted the study, we modified the study period for each use case. The first use case (ACEI exposure and risk of angioedema) was conducted with data from 1 January 2011 to 31 December 2017 in the NHIRD (versus 1 January 2001 to 31 December 2011 in the Sentinel study [7]). We used data from 1 January 2012 to 31 December 2015 for the second use case (warfarin exposure and risk of GI bleeding), which was roughly consistent with the study period of the original Sentinel analysis (1 January 2012 to 30 September 2015) [8]. The third use case (use of oral clindamycin and risk of CDI) was conducted with data from 1 January 2012 to 31 December 2015 (versus 1 January 2006 to 31 December 2013 in the Sentinel analysis [9]), and the fourth use case (use of glyburide and risk of hypoglycemia) was conducted with data from 1 January 2012 to 31 December 2015 (versus 1 January 2008 to 30 September 2014 in the Sentinel study [10]). Details of these four use cases have been published elsewhere [7–10,12–15]. We followed the same designs of the four use cases used by the U.S. Sentinel System [7–10]. In brief, these four cases were retrospective cohort studies with a new-user, active-comparator design. We applied the codes and definitions described in the publicly available Sentinel reports to identify exposure, outcome, and covariates, with modifications as appropriate. For example, instead of using the National Drug Codes, we used Taiwan’s National Health Insurance (NHI) Drug Codes to identify medications in the TSDM-formatted NHIRD. Because the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes have been used for reimbursement in Taiwan since 1 January 2016, both the Ninth Revision of the ICD Clinical Modification codes (ICD-9-CM) and the ICD-10-CM codes were used to define diseases and conditions in the first use case. We only used ICD-9-CM codes for the other three use cases as the study period ended in or before 2015. We also followed the outcome definitions provided by the U.S. Sentinel System. Angioedema was defined by an inpatient or outpatient diagnosis at any position; GI bleeding was defined by an inpatient diagnosis at the primary position; CDI was defined as a hospitalization with a diagnosis of CDI in any position; hypoglycemia events were defined by emergency department visits with a diagnosis of hypoglycemia at any position or admissions with a primary diagnosis of hypoglycemia. The complete lists of codes can be found in the Sentinel reports and publications [7–10,12–15]. Codes that were mapped or modified based on the Sentinel reports are provided in Tables S1–S4 (https://www.jfda-online.com/journal/vol31/iss4/13/). Propensity score (PS) methods were applied to all four use cases to balance baseline characteristics. PS stratification (by quantile) was applied in the first use case, while 1:1 PS matching was applied in the second, third, and fourth use cases. Following the analytical approach in the Sentinel study, we also performed 1: up to 10 variable matching in the second use case. Covariate balance was assessed by absolute standardized differences (aSD), where a value <0.1 generally indicated no considerable difference between the two groups. 2.3. Sentinel Routine Querying Tools Two modules from the Sentinel Routine Querying Tools were used in this project: the CIDA module and the PSA module, which consisted of SAS macros that allowed users to identify the study cohort and pre-specify the statistical analysis [16]. We were able to apply the Sentinel tools directly to TSDM, with one major modification. While the column length for drug code is 11 digits in the SCDM version 6.0.2, the Sentinel routine querying macro program “ms_extractdrugs.sas” was set to read drug codes with either 9 or 11 digits. Although the column length of the drug codes was set to be 11 digits in the TSDM (per SCDM version 6.0.2 specification), the NHI Drug Codes only contained 10 digits. We therefore modified the SAS macro “ms_extractdrugs.sas” to read 10-digit drug codes in the TSDM. To mimic the distributed data network of the Sentinel System and improve computational efficiency, we randomly divided the TSDM-formatted NHIRD into eight subsamples (i.e., “sites”) for analysis. For the first use case, the pooled result was generated from a Cox proportional hazards model stratified by PS quantile and site. For the second, third, and fourth use cases, the pooled results were generated from Cox models with two analytical approaches. We first performed the analysis stratified solely by site for samples with 1:1 PS matching. We next stratified the Cox models by both site and matched pair for 1: up to 10 variable matching in the second use case and for 1:1 matching in the third and fourth use cases. All statistical procedures were performed in the Health and Welfare Data Science Center using SAS software version 9.4 (SAS Institute, Cary, North Carolina, USA). 3. Results 3.1. Use of ACEIs and risk of angioedema In the first use case, we identified 533,237 ACEI initiators and 2,181,246 beta-blocker initiators after applying the same inclusion and exclusion criteria used in the U.S. Sentinel System analysis [7]. Table 1 shows the baseline characteristics of the ACEI and beta-blocker initiators. ACEI initiators tended to be older than beta-blocker initiators (mean age: 62.1 versus 52.1 years). Approximately 46% of the ACEI initiators were female, compared to 59% in the beta-blocker initiators. A higher proportion of ACEI initiators had diabetes (24% versus 10%), heart failure (5% versus 2%) and ischemic heart disease (11% versus 7%) than beta-blocker initiators. Table 1Baseline characteristics for angiotensin-converting enzyme inhibitors and beta-blocker initiators.N (%)ACEI initiatorsBeta-blocker initiatorsAbsolute standardized difference N = 533,237N = 2,181,246Age, yearMean (std)62.05 (21.69)52.12 (15.66) 0.525 18–4466,782 (12.52)760,279 (34.86) 0.584 45–54104,698 (19.63)465,061 (21.32) 55–64139,097 (26.09)452,078 (20.73) 65+222,660 (41.76)503,828 (23.10)Female243,212 (45.61)1,297,133 (59.47) 0.280 Diagnosis of: Allergic reactions74,660 (14.00)343,123 (15.73)0.049 Diabetes125,981 (23.63)226,473 (10.38) 0.358 Heart failure27,410 (5.14)47,748 (2.19) 0.158 Ischemic heart disease57,725 (10.83)152,088 (6.97) 0.136 Use of prescription NSAIDs264,525 (49.61)1,099,989 (50.43)0.016ACEI: angiotensin-converting enzyme inhibitor; NSAIDs: non-steroidal anti-inflammatory drugs; std: standard deviation. Baseline characteristics for angiotensin-converting enzyme inhibitors and beta-blocker initiators. ACEI: angiotensin-converting enzyme inhibitor; NSAIDs: non-steroidal anti-inflammatory drugs; std: standard deviation. The incidence rate of angioedema was 70.7 per 100,000 person-years for ACEI initiators and 74.9 per 100,000 person-years for beta-blocker initiators, with an adjusted HR of 1.21 (95% confidence interval [CI]: 0.89–1.64) (Table 2). Table 2Crude and adjusted results in angiotensin-converting enzyme inhibitors and beta-blocker initiators, with a maximum follow-up of 365 days.Number of eventsNumber of initiatorsPerson-yearsRisk per 100,000 initiatorsIncidence rate per 100,000 person-years (95%CI)Crude HR (95% CI)Adjusted HRa (95% CI)ACEI initiators65533,23791,94612.1970.69 (55.44–90.15)0.98 (0.75–1.29)1.21 (0.89–1.64)Beta-blocker initiators2452,181,246326,91711.2374.94 (66.12–84.94)ReferenceACEI: angiotensin-converting enzyme inhibitor; HR: hazard ratio; CI: confidence interval.aAdjusted hazard ratio was generated from a site- and propensity score-stratified Cox model. Crude and adjusted results in angiotensin-converting enzyme inhibitors and beta-blocker initiators, with a maximum follow-up of 365 days. ACEI: angiotensin-converting enzyme inhibitor; HR: hazard ratio; CI: confidence interval. Adjusted hazard ratio was generated from a site- and propensity score-stratified Cox model. 3.2. Use of warfarin and risk of GI bleeding In the second use case, 58,788 warfarin initiators and 1,398,317 statin initiators were included in the final sample (Table 3). Compared to statin initiators, warfarin initiators were older (mean age: 67.6 versus 60.5 years) and more likely to have comorbid conditions and receive other classes of medications. Warfarin initiators tended to have a higher mean number of emergency room and inpatient hospital encounters but a lower mean number of ambulatory encounters, number of generics, and unique drug classes than statin initiators. Most of the baseline characteristics were well-balanced after PS matching, except for baseline antiplatelet and aspirin use. We therefore included baseline antiplatelet and aspirin use as covariates in the final Cox proportional hazards model after PS matching. Note that race is 100% Asian in Taiwan, which was correctly reflected in Table 3, and therefore race was not adjusted in the PS model. Table 3Baseline characteristics of warfarin and statin initiators.Demographics, n (%)Unmatched1:1 Propensity score matching1:10 Variable propensity score matching Warfarin initiatorsN = 58,788Statin initiatorsN = 1,398,317Absolute standardized differenceWarfarin initiatorsN = 58,624Statin initiatorsN = 58,624Absolute standardized differenceWarfarin initiatorsN = 58,624Statin initiatorsN = 407,254Absolute standardized differenceAge, year mean (std)67.58 (20.29)60.47 (10.08) 0.444 67.57 (13.90)68.68 (17.11)0.07267.57 (13.90)68.72 (12.68)0.032 18–6426,554 (37.90)1,045,718 (62.77) 0.611 22,199 (37.87)20,914 (35.67)0.06822,199 (37.87)145,118 (35.63)0.068 65–7417,982 (25.66)392,371 (23.55)15,075 (25.71)17,189 (29.32)15,075 (25.71)118,736 (29.16) 75+25,536 (36.44)227,899 (13.68)21,350 (36.42)20,521 (35.00)21,350 (36.42)143,400 (35.21)Sex Male36,376 (51.91)799,271 (47.98)0.07930,456 (51.95)31,156 (53.15)0.02430,456 (51.95)216,207 (53.09)0.023 Female33,696 (48.09)866,717 (52.02)28,168 (48.05)27,468 (48.85)28,168 (48.05)191,047 (46.91)Race (Asian)70,072 (100.0)1,665,988 (100.0)58,624 (100.00)58,624 (100.00)–58,624 (100.00)407,254 (100.00)–Recorded history of: Advanced liver disease277 (0.40)1,015 (0.06)0.070238 (0.41)222 (0.38)0.004238 (0.41)1,476 (0.36)0.007 Alcohol abuse or dependence667 (0.95)9,952 (0.60)0.040564 (0.96)621 (1.06)0.010564 (0.96)4,103 (1.01)0.005 End stage renal disease0 (0.00)0 (0.00)0 (0.00)0 (0.00)–0 (0.00)0 (0.00)– Gastritis or gastroenteritis12,730 (18.17)287,560 (17.26)0.02410,759 (18.35)10,721 (18.29)0.00210,759 (18.35)74,339 (18.25)0.003 Helicobacter pylori infection280 (0.40)6,460 (0.39)0.002226 (0.39)206 (0.35)0.006226 (0.39)1,557 (0.38)0.001 Peptic ulcer219 (0.31)1,460 (0.09)0.050180 (0.31)198 (0.34)0.005180 (0.31)1,292 (0.32)0.002 Inflammatory bowel disease372 (0.53)7,600 (0.46)0.011318 (0.54)289 (0.49)0.007318 (0.54)2,230 (0.55)0.001 Intestinal infections1,794 (2.56)37,424 (2.25)0.0211,545 (2.64)1,495 (2.55)0.0051,545 (2.64)10,570 (2.60)0.003 Obesity302 (0.43)10,568 (0.63)0.028251 (0.43)232 (0.40)0.005251 (0.43)1,680 (0.41)0.002 Other GI diverticula239 (0.34)2,355 (0.14)0.041196 (0.33)207 (0.35)0.003196 (0.33)1,335 (0.33)0.001 Other GI ulcer disease9,624 (13.73)158,097 (9.49) 0.133 8,107 (13.83)8,204 (13.99)0.0058,107 (13.83)56,957 (13.99)0.005 Other non-GI bleeding2,638 (3.76)38,867 (2.33)0.0832,230 (3.80)2,216 (3.78)0.0012,230 (3.80)15,361 (3.77)0.002 Renal disease4,115 (5.87)55,810 (3.35) 0.121 2,497 (5.97)3,545 (6.05)0.0032,497 (5.97)25,008 (6.14)0.007 Sepsis3,600 (5.14)11,916 (0.72) 0.265 2,999 (5.12)2,647 (4.52)0.0282,999 (5.12)18,413 (4.52)0.028 Shock1,141 (1.63)3,525 (0.21) 0.149 963 (1.64)856 (1.46)0.015963 (1.64)6,025 (1.48)0.013 Tobacco use394 (0.56)11,520 (0.69)0.016351 (0.60)400 (0.68)0.011351 (0.60)2,605 (0.64)0.005History of use of: Antiplatelets and aspirin35,136 (50.14)469,248 (28.17) 0.462 29,528 (50.37)33,042 (56.36) 0.120 29,528 (50.37)228,383 (56.08) 0.115 COX-2 inhibitors8,121 (11.59)93,662 (5.62) 0.214 6,853 (11.69)7,509 (12.81)0.0346,853 (11.69)52,471 (12.88)0.036 H2RAs and sucralfate15,708 (22.42)285,587 (17.14) 0.133 13,044 (22.25)13,148 (22.43)0.00413,044 (22.25)91,466 (22.46)0.005 Heparins4,892 (6.98)15,022 (0.90) 0.316 4,106 (7.00)3,901 (6.65)0.0144,106 (7.00)27,091 (6.65)0.014 Methotrexate425 (0.61)4,447 (0.27)0.052361 (0.62)421 (0.72)0.013361 (0.62)2,789 (0.68)0.009 NSAIDs36,764 (52.47)834,385 (50.08)0.04830,770 (52.49)31,137 (53.11)0.01330,770 (52.49)215,275 (52.86)0.008 Opioids10,356 (14.78)81,471 (4.89) 0.337 8,601 (14.67)8,629 (14.72)0.0018,601 (14.67)59,829 (14.69)0.001 Oral glucocorticoids14,710 (20.99)213,066 (12.79) 0.220 12,308 (20.99)12,372 (21.10)0.00312,308 (20.99)86,025 (21.12)0.003 Proton pump inhibitors6,439 (9.19)84,195 (5.05) 0.161 5,346 (9.12)5,229 (8.92)0.0075,346 (9.12)35,823 (8.80)0.011 SSRIs or SNRIs2,763 (3.94)59,247 (3.56)0.0202,333 (3.98)2,391 (4.08)0.0052,333 (3.98)16,732 (4.11)0.007 Respiratory opioids18,623 (26.58)359,126 (21.56) 0.118 15,510 (26.46)15,528 (26.49)0.00115,510 (26.46)108,376 (26.61)0.004Health service utilization intensity, mean (std) Ambulatory encountersa13.09 (5.64)10.13 (12.60) 0.303 13.09 (10.07)13.47 (13.48)0.03113.09 (10.07)13.44 (10.03)0.017 Other ambulatory encountersa0.44 (1.83)0.51 (1.09)0.0470.43 (0.78)0.42 (1.06)0.0110.43 (0.78)0.42 (0.92)0.005 Emergency room encounters0.85 (1.16)0.23 (0.47) 0.700 0.84 (1.49)0.79 (1.14)0.0380.84 (1.49)0.79 (2.08)0.023 Inpatient hospital encounters0.81 (0.77)0.14 (0.29) 1.143 0.80 (1.44)0.72 (0.92)0.0650.80 (1.44)0.72 (1.20)0.037 Non-acute institutional encounters0.00 (0.02)0.00 (0.02)0.015<0.01 (0.03)<0.01 (0.04)0.006<0.01 (0.03)<0.01 (0.04)0.003 Prescription fills37.44 (25.59)24.86 (33.73) 0.420 37.38 (27.30)38.55 (42.07)0.03337.38 (27.30)38.44 (32.41)0.020 Genericsb14.96 (6.92)10.31 (10.53) 0.521 14.93 (7.45)15.26 (10.60)0.03614.93 (7.45)15.24 (10.34)0.018 Unique drug classes14.91 (6.91)10.28 (10.62) 0.517 13.58 (6.31)13.88 (9.04)0.04013.58 (6.31)13.87 (8.74)0.020GI: gastrointestinal; H2RAs: H2 receptor antagonists; NSAIDs: non-steroidal anti-inflammatory drugs; SSRI: selective serotonin reuptake inhibitors; SNRI: Serotonin and norepinephrine reuptake inhibitors; std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1.aAmbulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits.bThe number of drug generic names (i.e., the number of active pharmaceutical ingredients). Baseline characteristics of warfarin and statin initiators. GI: gastrointestinal; H2RAs: H2 receptor antagonists; NSAIDs: non-steroidal anti-inflammatory drugs; SSRI: selective serotonin reuptake inhibitors; SNRI: Serotonin and norepinephrine reuptake inhibitors; std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1. Ambulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits. The number of drug generic names (i.e., the number of active pharmaceutical ingredients). Table 4 shows the incidence rate and HRs of GI bleeding between warfarin and statin initiators. The crude incidence rate of GI bleeding was 25.7 per 1,000 person-years for warfarin initiators and 5.5 per 1,000 person-years for statin initiators, which corresponded to a crude HR of 4.70 (95% CI: 4.29–5.15). The incidence rate in the 1:1 PS-matched cohort was 25.6 per 1,000 person-years for warfarin initiators and 15.2 per 1,000 person-years for statin initiators, with a HR of 1.72 (95% CI: 1.50–1.98). Similar results were observed in the 1:10 variable PS-matched analysis. The incidence rate was 25.6 per 1,000 person-years for warfarin initiators and 10.0 per 1,000 person-years for statin initiators, with a HR of 2.56 (95% CI: 2.32–2.82). Table 4Effect estimates for gastrointestinal bleeding between warfarin and statin initiators.Medical productNumber of initiatorsPerson-yearAverage person-daysNumber of eventsIncidence rate per 1,000 person-yearIncidence rate difference per 1,000 person-yearsHR (95% CI) Unmatched analysis (site adjusted only) Warfarin58,78821,729.30135.0055825.68 (23.64–27.90)20.174.70 (4.29–5.15) Statins1,398,317490,137.08128.032,6985.51 (5.30–5.72)Reference 1:1 propensity score-matched analysis (Cox model NOT stratified by matched pair and adjusted for site and antiplatelet and aspirin use) a Warfarin58,62421,687.59135.12255525.59 (23.55–27.81)10.351.72 (1.50–1.98) Statins58,62421,723.33135.34533115.24 (13.68–16.97)Reference 1:10 propensity score-matched analysis (Cox model stratified by matched pair and adjusted for site and antiplatelet and aspirin use) a Warfarin58,62421,687.59135.12255525.59 (23.55–27.81)15.592.56 (2.32–2.82) Statins407,254151,401.86135.7861,51410.00 (9.51–10.52)ReferenceHR: hazard ratio; CI: confidence interval.aBaseline antiplatelet and aspirin exposure was included in the final model because it remained imbalanced after matching. Effect estimates for gastrointestinal bleeding between warfarin and statin initiators. HR: hazard ratio; CI: confidence interval. Baseline antiplatelet and aspirin exposure was included in the final model because it remained imbalanced after matching. 3.3. Use of oral clindamycin and risk of C. difficile infection The baseline characteristics of oral clindamycin and oral penicillin initiators are presented in Table 5. There were 1,570,359 oral clindamycin initiators and 9,411,950 oral penicillin initiators. Compared to oral penicillin initiators, a higher proportion of oral clindamycin initiators were female (57.8% versus 51.0%), received first-generation cephalosporins (18.2% versus 14.4%) and tetracyclines (4.9% versus 2.6%), but a lower proportion of clindamycin initiators received macrolides (1.3% versus 4.0%) and proton pump inhibitors (1.7% versus 3.8%). A slightly lower number of inpatient hospital encounters and prescription refills was observed among those who initiated oral clindamycin than those who initiated oral penicillins. No meaningful differences were found after PS matching. Table 5Baseline characteristics of oral clindamycin and penicillin initiators.Demographics, n (%)Unmatched1:1 Propensity score matching Oral clindamycin initiatorsN = 1,570,359Oral penicillin initiatorsN = 9,411,950Absolute standardized differenceOral clindamycin initiatorsN = 1,569,188Oral penicillin initiatorsN = 1,569,188Absolute standardized differenceSex, female907,104 (57.76)4,797,309 (50.97) 0.137 906,280 (57.75)903,298 (57.56)0.004Mean age (std)37.97 (18.58)38.78 (9.04)0.05537.97 (18.60)38.72 (20.40)0.009Recorded use of: Aminoglycosides13,468 (0.86)90,755 (0.96)0.01113,090 (0.83)12,795 (0.82)0.002 Beta Lactam inhibitors331 (0.02)3,475 (0.04)0.009327 (0.02)352 (0.02)0.001 Cephalosporins-1st generation285,492 (18.18)1,350,413 (14.35) 0.104 285,056 (18.17)286,155 (18.24)0.002 Cephalosporins-2nd generation22,650 (1.44)128,438 (1.36)0.00722,573 (1.44)22,959 (1.46)0.002 Cephalosporins-3rd generation5,397 (0.34)42,291 (0.45)0.0175,376 (0.34)5,418 (0.35)0.001 Cephalosporins-4th and 5th generation62 (0.00)889 (0.01)0.00762 (0.00)74 (0.00)0.001 Fluoroquinolones22,220 (1.41)128,996 (1.37)0.00422,152 (1.41)22,598 (1.44)0.002 H2Ras208,643 (13.29)953,304 (10.13)0.098208,445 (13.28)215,053 (13.70)0.012 Injectable clindamycin3,325 (0.21)2,184 (0.02)0.0552,154 (0.14)2,010 (0.13)0.003 Injectable penicillin1,004 (0.06)27,282 (0.29)0.0541,004 (0.06)1,030 (0.07)0.001 Macrolides19,928 (1.27)377,418 (4.01) 0.172 19,928 (1.27)19,861 (1.27)0.000 Other antimicrobials3,473 (0.22)23,788 (0.25)0.0073,469 (0.22)3,434 (0.22)0.001 Proton pump inhibitors26,188 (1.67)358,171 (3.81) 0.131 26,175 (1.67)25,073 (1.60)0.006 Sulfonamides31,635 (2.01)181,036 (1.92)0.00731,597 (2.01)31,700 (2.02)0.001 Tetracyclines76,288 (4.86)247,673 (2.63) 0.118 75,944 (4.84)71,199 (4.54)0.014Recorded history of: Chemotherapy administration3,133 (0.20)28,460 (0.30)0.0213,130 (0.20)3,196 (0.20)0.001 Diabetes80,868 (5.15)686,221 (7.29)0.08980,783 (5.15)80,155 (5.11)0.002 Dialysis3,514 (0.22)34,792 (0.37)0.0273,509 (0.22)3,609 (0.23)0.001 Immunocompromised state22,027 (1.40)179,116 (1.90)0.03922,006 (1.40)22,449 (1.43)0.002 Inflammatory bowel disease11,357 (0.72)69,466 (0.74)0.00211,350 (0.72)11,384 (0.73)<0.001Health service utilization intensity, mean (std) Ambulatory encountersa6.09 (7.47)6.57 (5.47)0.0736.09 (7.37)6.02 (6.29)0.008 Other ambulatory encountersa0.26 (0.75)0.29 (1.06)0.0240.26 (0.76)0.26 (0.78)0.004 Emergency room encounters0.14 (0.59)0.19 (0.86)0.0650.14 (0.58)0.14 (0.79)0.001 Inpatient hospital encounters0.06 (0.18)0.08 (0.21) 0.131 0.06 (0.18)0.06 (0.36)0.002 Non-acute institutional encounters0.00 (0.02)0.00 (0.04)0.0020.00 (0.02)0.00 (0.02)0.001 Prescription fills11.25 (14.26)13.04 (17.68) 0.111 11.25 (14.02)11.22 (15.96)0.002 Genericsb5.93 (5.80)6.19 (8.07)0.0375.93 (5.68)5.89 (5.60)0.003H2Ras: H2 receptor antagonists; std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1.aAmbulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits.bThe number of drug generic names (i.e., the number of active pharmaceutical ingredients). Baseline characteristics of oral clindamycin and penicillin initiators. H2Ras: H2 receptor antagonists; std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1. Ambulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits. The number of drug generic names (i.e., the number of active pharmaceutical ingredients). Table 6 shows the incidence rates of CDI among the oral clindamycin and oral penicillin initiators. In the unmatched analysis, 94 events were found in the penicillin initiators, but less than three events were found in the clindamycin initiators. Due to the low event numbers, we were unable to obtain a reliable effect estimate from the matched analysis. Table 6Effect estimates for inpatient admission and emergency visits of Clostridioides difficile infection.ExposureNumber of InitiatorsPerson-yearsNumber of eventsIncidence rate per 1,000 person-yearsIncidence rate difference per 1,000 person-years Unmatched analysis (site adjusted only) Oral clindamycin1,570,35977,288.75NA0.03−0.17 Oral penicillins9,411,950484,387.82940.19NA: not available. Per the Health and Welfare Data Science Center’s rule, statistical tables with variable classifications yielding fewer than three units or numbers cannot be exported. Due to the low number of events, we were unable to obtain a reliable effect estimate from the matched analysis. Effect estimates for inpatient admission and emergency visits of Clostridioides difficile infection. NA: not available. Per the Health and Welfare Data Science Center’s rule, statistical tables with variable classifications yielding fewer than three units or numbers cannot be exported. Due to the low number of events, we were unable to obtain a reliable effect estimate from the matched analysis. 3.4. Use of glyburide and risk of serious hypoglycemia We identified 56,216 glyburide initiators and 52,073 glipizide initiators in the fourth use case (Table 7). Compared to glipizide users, glyburide users were younger (mean age: 59.9 versus 62.4 years), less likely to have chronic kidney disease, less likely to use insulin or non-secretagogue antidiabetic drugs, and had a slightly lower comorbidity score at baseline. A slightly higher mean number of emergency room encounters was observed in glyburide users compared to glipizide users, but a slightly lower health services utilization intensity was observed in glyburide users for inpatient encounters, prescription fills, and number of generic drugs used. All covariates were well-balanced after PS matching. Table 7Baseline characteristics of glyburide and glipizide users.Unmatched1:1 Propensity score matching Glyburide usersN = 56,216Glipizide usersN = 52,073Absolute standardized differenceGlyburide usersN = 44,685Glipizide usersN = 44,685Absolute standardized differenceSex, female, n (%)24,999 (44.47)23,336 (44.81)0.00719,992 (44.74)19,959 (44.67)0.002Mean age (std)59.91 (14.60)62.40 (18.86) 0.148 61.27 (17.03)61.19 (14.71)0.005Recorded history of, n (%) Chronic kidney disease2,725 (4.85)6,588 (12.65) 0.279 2,724 (6.10)2,724 (6.10)0.005 Serious hypoglycemia968 (1.72)1418 (2.72)0.068892 (2.00)892 (2.00)0.002 Insulin5,087 (9.05)7,655 (14.70) 0.175 4,944 (11.06)4,944 (11.06)0.001 Metformin37,724 (67.11)33,978 (62.25)0.03930,500 (68.26)30,500 (68.26)0.006 Non-secretagogue antidiabetic drugs13,005 (23.13)15,549 (29.86) 0.153 12,175 (27.25)12,175 (27.25)0.012 Combined Charlson/ Elixhauser comorbidity score (std)0.28 (1.06)0.65 (2.12) 0.224 0.38 (1.01)0.38 (0.72)<0.001Health service utilization intensity, mean (std) Ambulatory encountersa9.95 (4.27)10.62 (13.28)0.06910.11 (12.31)10.11 (12.31)0.001 Other ambulatory encountersa0.46 (1.41)0.48 (0.91)0.0180.46 (0.92)0.46 (0.92)0.007 Emergency room encounters0.46 (1.64)0.31 (0.87) 0.114 0.35 (1.08)0.35 (1.08)0.007 Inpatient hospital encounters0.23 (0.87)0.39 (1.07) 0.169 0.27 (0.86)0.27 (0.86)0.009 Non-acute institutional encounters0.00 (0.03)0.00 (0.02)0.0050.00 (0.02)0.00 (0.02)0.008 Prescription fills25.43 (20.81)29.66 (32.26) 0.156 27.06 (26.36)27.06 (26.36)0.003 Genericsb10.24 (8.43)11.69 (10.81) 0.149 10.78 (8.81)10.78 (8.81)0.004std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1.aAmbulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits.bThe number of drug generic names (i.e., the number of active pharmaceutical ingredients). Baseline characteristics of glyburide and glipizide users. std: standard deviation. Bold numbers indicate the absolute standardized difference between the two groups >0.1. Ambulatory encounters included visits at outpatient clinics, urgent care visits, and other same-day ambulatory hospital encounters, but exclude emergency department encounters; other ambulatory encounters included other non-overnight ambulatory encounters such as home health visits, rehabilitations and nursing facility visits. The number of drug generic names (i.e., the number of active pharmaceutical ingredients). Incidence rates and HRs of hypoglycemia are presented in Table 8. The crude incidence rate of hypoglycemia was 11.4 per 1,000 person-years for glyburide users and 9.9 per 1,000 person-years for glipizide users, which corresponded to a crude HR of 1.15 (95% CI: 0.95–1.39). Similar incidence rates were observed after we applied 1:1 PS matching without stratifying on matched pair, with a HR of 1.61 (95% CI: 1.30–2.00). Slightly higher incidence rates were observed in the analysis stratified on matched pair (22.8 per 1,000 person-years for glyburide users and 12.7 per 1,000 person-years for glipizide users), but the HR remained the same as in the unstratified analysis (HR: 1.61; 95% CI: 1.30–2.00). Table 8Effect estimates for inpatient admission and emergency department visit for hypoglycemia.ExposureNumber of initiatorsPerson-yearsNumber of eventsIncidence rate per 1,000 person-yearsIncidence rate difference per 1,000 person-yearsHazard ratio (95% CI) Unmatched analysis (site adjusted only) Glyburide56,21620,638.8423611.431.491.15 (0.95, 1.39) Glipizide52,07319,415.751939.94Reference 1:1 propensity score-matched analysis (Cox model NOT stratified by matched pair and adjusted for site) Glyburide44,68516,448.1821613.134.981.61 (1.30, 2.00) Glipizide44,68516,674.901368.16Reference 1:1 propensity score-matched analysis (Cox model stratified by matched pair and adjusted for site) Glyburide44,6856,542.9714922.7710.091.61 (1.30, 2.00) Glipizide44,6856,542.978312.69Reference Effect estimates for inpatient admission and emergency department visit for hypoglycemia. 4. Discussion To our knowledge, this study is the first direct application of the U.S. Sentinel Routine Querying Tools on the NHIRD in Taiwan. We demonstrated the feasibility of applying the same study designs, analytic choices, and analytic tools as used by the U.S. Sentinel System to examine the same drug exposure-outcome associations in the TSDM-formatted NHIRD. We observed two known associations: (1) warfarin use and risk of GI bleeding, and (2) glyburide use and risk of serious hypoglycemia in the TSDM-formatted NHIRD. An elevated risk of angioedema associated with ACEI use was observed in our analysis, but the 95% CI included the null. We did not have enough event numbers to examine the association between oral clindamycin and risk of CDI. It is therefore important to note that being able to apply the same design and analytic approaches does not mean the results would be comparable between data sources. Instead, it allows us to remove modifiable sources of heterogeneity and focus on other more meaningful differences, as we discuss below. There were some notable differences in study samples between our analyses and the U.S. Sentinel System analyses. Our samples tended to be older than the samples from the U.S. Sentinel System studies. The reason for age difference could be the inclusion of primarily privately insured populations in the U.S. Sentinel System studies, in which approximately two-thirds of individuals were between 18 and 65 years old [12]. A considerably higher utilization of ambulatory services and prescription drugs was observed in our samples compared to the Sentinel samples in all four use cases. The high utilization rate was mainly attributed to the easy access and low costs of ambulatory services and prescription drugs under the NHI program in Taiwan. A lower number of ACEI initiators was identified in our study than that in the prior reports from the Sentinel System [12]. The low utilization rate of ACEIs in Taiwan may reflect the concern about ACEI-induced cough [17]. In addition to the differences in the baseline characteristics, the incidence rates of angioedema were lower in the present study compared to the results from the U.S. Sentinel System analysis [12]. The magnitude of the relative risk of angioedema was also different, with a lower and non-significant HR found in our study compared to the HRs reported from the Sentinel System. One potential explanation for these differences is that the risk of angioedema may differ by race. Compared to beta-blocker initiators, lower incidence rate and risk of angioedema was reported in ACEI initiators of Asians compared to those of Whites in a U.S. study [18]. We identified less than 100 CDI cases in our third use case, which prevented us from further evaluating the association between oral clindamycin and risk of CDI. Several factors may have contributed to the low event number. First, the prevalence of CDI was low (estimated 4%) in Taiwan [19]. In addition, because the diagnosis of CDI requires patient symptoms, positive stool culture for C. difficile and positive test for toxin A or toxin B, CDI may be underdiagnosed during our study period as the test for C. difficile toxin A and B was not reimbursed by the NHI until 1 March 2017. Finally, as there are only three diagnoses available for outpatient claims and five diagnoses available for inpatient claims in the NHI, less severe CDI events might not have been recorded. The crude and adjusted HRs from the present study were comparable to the HRs from the U.S. Sentinel study in the second use case (HRTSDM: 1.72–2.56 and HRSCDM: 2.22–3.10) and the fourth use case (HRTSDM: 1.61 and HRSCDM: 1.35–1.36). The lower magnitude of GI bleeding risk found in the TSDM may be explained by the fact that physicians tended to maintain an international normalized ratio within a lower range for warfarin users in Taiwan due to the concerns about bleeding [20,21]. Regardless of the difference in magnitude of risk, the results from the TSDM and SCDM support the two known associations between warfarin exposure and risk of GI bleeding and between glyburide exposure and risk of hypoglycemia; therefore, these results would likely lead to the same conclusion for regulatory decision making [22]. Although we showed the feasibility of direct application of the Sentinel Routine Querying Tools within the TSDM-formatted NHIRD, there are several issues that need to be considered. First, direct application of certain study specifications may not be practicable given the potential differences in clinical practice and healthcare systems. For example, we did not identify any patients with a history of end-stage renal disease (ESRD) in the present study. This is mainly because the diagnosis codes used to identify ESRD (ICD-9-CM 585.6 and 585.6x) are not commonly used in Taiwan. Most physicians only use the code 585 for chronic kidney disease without specifying the stage, making it difficult to identify ESRD using only diagnosis codes. Additional information from the Registry of Catastrophic Illness is often required to identify ESRD in the NHIRD. Different coding systems or access to medical products among the countries is another issue. An example is the mapping between the Current Procedural Terminology/Healthcare Common Procedure Coding System codes and the NHI procedure codes. Given that procedure codes are often designed for reimbursement purposes, the structure and granularity of procedure codes may differ between different healthcare systems. For instance, a parenterally administered chemotherapy could be captured as a procedure in one country but as a dispensing in another. Moreover, as information captured in claims databases is heavily influenced by insurance coverage policies, we might not be able to capture treatments that are covered in the U.S. but not in Taiwan. For example, none of the patients in our study had bariatric surgery because bariatric surgery was not covered by the NHI program before 2020. New medical products may also enter the U.S. and Taiwanese markets at different times, which could affect the practicability of running parallel analyses between the two countries. Given these challenges, caution is required when considering direct application of the study designs and analytic choices used in the U.S. Sentinel System to the TSDM-formatted NHIRD. A thorough investigation of the definitions and codes before direct application of the Sentinel analytic package is recommended. In conclusion, the TSDM provides a platform that allows direct application of the Sentinel Routine Querying Tools with minor modifications. However, sources of heterogeneity other than design and analytic differences should be carefully considered when interpreting and comparing the results between the two systems. Supplementary Information Acknowledgments This study is supported by National Science and Technology Council of Taiwan (MOST-109-2636-B-002-002). The sponsors had no role in the preparation of the manuscript or analysis of the study results. No payment was received for writing this article. Funding Statement This study is supported by National Science and Technology Council of Taiwan (MOST-109-2636-B-002-002). Footnotes Conflicts of interest: None. References Associated Data Supplementary Materials