Efficacy of COVID-19 Treatments in Intensive Care Unit: A Systematic Review and Meta-Analysis of Randomized Controlled Trials Authors: Mahmoud Alwakeel, Francois Abi Fadel, Abdelrahman Nanah, Yan Wang, Mohamed K. A. Awad, Fatima Abdeljaleel, Mohammed Obeidat, Talha Saleem, Saira Afzal, Dina Alayan, Mary Pat Harnegie, Xiaofeng Wang, Abhijit Duggal, Peng Zhang Published: 2024 ABSTRACT Objectives: Examining the cumulative evidence from randomized controlled trials (RCTs), evaluating the use of pharmacological agents for the treatment of COVID-19 infections in patients with critical illness. Data Sources: Databases Medline, Embase, Web of Science, Scopus, CINAHL, and Cochrane.Study Selection: Inclusion criteria were RCTs that enrolled patients with confirmed or suspected COVID-19 infection who are critically ill. Only RCTs that examined therapeutic agents against one another or no intervention, placebo, or standard of care, were included. Data Extraction: Pairs of reviewers extracted data independently. Outcomes of interest included the overall reported mortality defined as either the ICU mortality, hospital mortality, mortality within 28 days or mortality within 90 days. Data Synthesis: A total of 40 studies (11,613 patients) evaluated 50 therapeutic intervention arms divided into five main therapy categories; steroids, antiviral medications, immunomodulators, plasma therapies [intravenous immunoglobulins (IVIG), convalescent plasma and/or, therapeutic plasma exchange], and therapeutic anticoagulation. Immunomodulators was the only group with possible mortality benefit, risk ratio (RR) 0.83 (95% CI 0.73; 0.95), with nonsignificant heterogeneity (I2 = 8%, p=0.36). In contrast, the other therapy groups showed no significant impact on mortality, as indicated by their respective pooled RRs: steroids [RR 0.91 (95% CI 0.82; 1.01), I2 = 31%], antiviral medications [RR 1.11 (95% CI 0.82; 1.49), I2 = 57%], plasma therapies [RR 0.77 (95% CI 0.58; 1.01), I2 = 36%], and anticoagulation [RR 1.06 (95% CI 0.95; 1.18), I2 = 0%]. Conclusions: This meta-analysis highlights both the heterogeneity and a lack of benefit from therapies evaluated during the COVID-19 pandemic. Many of the RCTs were developed based on limited observational data. Future RCTs investigating pharmaceutical interventions in critically ill patients during pandemics need to be designed based on better evidence. 1. Introduction Since the beginning of the COVID-19 pandemic more than 8000 randomized controlled trials (RCTs), in different stages of progression and completion have been conducted to identify a therapy to achieve survival benefit in COVID-19-associated infection [1]. Over the last 3 years more than 500 meta-analyses have been published to summarize and develop an evidence base for the efficacy of pharmacologic agents for the treatment of COVID-19 infections [2]. The desire for effective therapy is even more heightened in the intensive care unit (ICU) settings where the majority of mortality from the pandemic occurred [3]. Unfortunately, most therapeutic RCTs for COVID-19 have enrolled patients with varying degrees of severity of disease and need for oxygen support, and consequently have had to grapple with significant heterogeneity in the included populations [4]. Evidence that looked exclusively in the ICU is limited to early case series from single centers or from studies with inherent biases, suboptimal study designs and inappropriate statistical analysis [5]. These initial studies were used to inform treatment in many countries, despite these significant limitations [6]. Subsequent well-designed studies developed by research consortia globally (REMAP [7] and ACTIV [8]) mostly reported outcomes in ICU patients as subgroups. Even though these results were compelling, the issue of type I errors in those groups remains a concern [9, 10]. For that reason, a systematic review and meta-analysis was performed examining the cumulative evidence from RCTs, which have studied pharmacologic agents exclusively in the ICU setting and how those results would differ when compared to the larger studies performed in all hospitalized adults, which informed the majority of treatment decisions for patients with critical illness in the later phases of the pandemic. 2. Methods This systematic review and meta-analysis follow the preferred reporting items for systematic reviews and meta-analyses (PRISMA) checklist (Supporting Figure 11) [11]. The protocol was registered with the International prospective register of systematic reviews (PROSPERO# CRD42021250583https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=250583) [12]. 2.1. Eligibility Criteria 2.1.1. Studies and Participants Studies included were RCTs that included patients with confirmed or suspected COVID-19 infection who were critically ill. Critical illness was defined as any patient admitted to an adult ICU or area of the hospital where critically ill patients routinely received treatment, patients receiving invasive or noninvasive mechanical ventilation (MV), patients receiving continuous intravenous vasoactive medications or other criteria with justification presented in the individual study to designate patients as critically ill. Only trials published in English language were included. 2.1.2. Interventions and Comparisons Only RCTs that examined therapeutic agents against one another or no intervention, placebo, or standard of care were included. Exclusion criteria were RCTs evaluating vaccinations, nutrition, vitamins, herbal, or alternative medicines, and nonmedication-related supportive interventions such as proning. 2.2. Outcomes The outcome of interest for this study was the overall reported mortality defined as either the ICU mortality, hospital mortality, mortality within 28 days, or mortality within 90 days. 2.3. Information Source A systematic search of existing, relevant literature was performed by the authors, including an experienced medical information specialist, in the databases: Medline (via Ovid SP), Embase (via Ovid SP), Web of Science (via Clarivate Analytics), Scopus (via Elsevier), CINAHL (via EBSCOhost), and Cochrane (via Wiley). The databases were searched from Jan 1st, 2019, till April 21st, 2021 and updated twice on Sept 30th, 2021, and April 28th, 2023. Keywords: COVID, coronavirus, coronavirus infection, SARS, critical care, intensive care, artificial respiration, MV, vasoconstrictor agent, and RCTs. A comprehensive description of the search strategy is included in the supporting information file. The articles were imported from Endnote (version X8.2 for Windows; 2018 Clarivate, Philadelphia, USA) to Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia. Available at https://www.covidence.org) and checked for duplicates. 2.4. Study Selection Using the systematic review software, Covidence, pairs of reviewers, following training and standardization exercises, independently screened all titles and abstracts, followed by full texts of RCT that were identified as potentially eligible. If a conflict existed, a third independent reviewer resolved it. 2.5. Data Collection For each eligible trial, pairs of reviewers extracted data independently using a standardized data collection form on Microsoft Excel 2013. Reviewers collected information on trial characteristics (trial PubMed reference number, registration number, study country, sample size), patient characteristics (age, sex, and the number of patients on MV), and mortality outcomes of interest; the number of participants analyzed and the number of participants who experienced an event for dichotomous outcomes. Discrepancies were resolved by an independent third reviewer. 2.6. Risk of Bias Within Individual Studies and Publication Bias Assessment Two independent reviewers assessed the risk of bias for each study based on the criteria outlined in Version two of the Cochrane risk-of-bias tool for randomized trials [13]. These criteria included random sequence generation, blinding of participants and personnel, blinding of outcome assessment, allocation concealment, incomplete outcome data, selective reporting, and other potential sources of bias. For each criterion, the risk of bias was categorized as low, high, or unclear. Analyses of the data and figures were computed using the RevMan 5.4.1 software. Disagreements were solved by discussion between the reviewers; if necessary, a consensus was sought through consulting a third reviewer. To assess the potential presence of publication bias in the analysis, a funnel plot was utilized, a standard graphical tool for visualizing the symmetry of effect size estimates. To further test the symmetry of the funnel plot, in addition to the visual examination, Egger's regression test was conducted. 2.7. Data Synthesis and Statistical Analysis In the meta-analysis, each node represents a treatment medication, regardless of the dose or duration of administration. For studies involving more than a therapeutic intervention arm like different dosages from the drug or different combination, each arm was treated as a separate group compared to the same number of the control arm. The studies were divided into a total of five treatment groups depending on the intervention drug category; steroids, antiviral medications, plasma therapies (IVIG, convalescent plasma and/or therapeutic plasma exchange), immunomodulators, and agents targeting therapeutic anticoagulation. This classification was developed based on the author's consensus. Treatment effects were expressed as risk ratios (RRs) for mortality. Pooled RR was calculated using random effects models, as clinical heterogeneity was expected. Statistical heterogeneity among the studies' effects was investigated using the I2 statistic. All statistical analyses were performed using R programming language version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/). 3. Results After removing duplicates, 20,581 title/abstracts were screened. Subsequently, 565 studies were assessed for full-text eligibility. 40 studies (11,613 patients) with a total of 50 therapeutic intervention arms met inclusion criteria (Figure 1). An overview of the characteristics of the included studies is provided in Table 1. Eight studies (REMAP-CAP steroid 2020 [14]; CoDEX 2020 [15]; Dequin et al. 2020 [16]; COVID STEROID 1 2021 [17], COVID STEROID 2 2021 [18], Maskin et al. 2021 [19], COVIDICUS 2022 [20], Soliman et al. 2022 [21]) with total 9 therapeutic arms (Hydrocortisone n = 4, Dexamethasone n = 4, Methylprednisolone n = 1) assessed the use of Steroids exclusively in critically ill patients. Antivirals were studied in 4 studies (Davoudi-Monfared et al. 2020 [22]; REMAP-CAP antiviral 2021 [23]; Réa-Neto et al. 2021 [24]; Alavi Darazam et al. 2021 [25]) with a total 6 therapeutic arms (Lopinavir-Ritonavir n = 1, Hydroxychloroquine, or Chloroquine n = 2, Interferon-β1a n = 1, Lopinavir-Ritonavir + Hydroxychloroquine n = 1, Lopinavir/Ritonavir + Hydroxychloroquine + Interferon-β1a + Umifenovir n = 1). Fourteen studies evaluated immunomodulators in critically ill patients (PANAMO 2020 [26]; Fakharian et al. 2021 [27]; Rashad et al. 2021 [28]; REMAP-CAP immunomodulator 2021 [29], COV-BARRIER 2021 [30], Lescure et al. 2021 [31], PANAMO 2022 [32], Atmowihardjo et al. 2022 [33], CORIMUNO-19 TOCI-2 2022 [34], CORIMUNO-19 SARI-2 2022 [34], ESCAPE 2022 [35], Kharazmi et al. 2022 [36], Rein et al. 2022 [37], Annane et al. 2023 [16]) with a total of 17 therapeutic arms (Tocilizumab n = 3, Sarilumab n = 4, Adalimumab n = 1, Vilobelimab n = 2, Ravulizumab n = 1, Imatinib n = 1, Baricitinib n = 1, Anakinra n = 2, Ruxolitinib n = 2). Plasma-related interventions were studied in 6 RCTs (Rasheed et al. 2020 [38]; Faqihi et al. 2021 [39]; Ali et al. 2021 [39]; Bennett-Guerrero et al. 2021 [40], REMAP-CAP 2021 [41], ICAR 2022 [42]) with a total of 9 therapeutic arms (Therapeutic plasma exchange (TPE) n = 2, convalescent plasma n = 7). Eight studies evaluated anticoagulation exclusively in critically ill patients (HESACOVID 2020 [43]; INSPIRATION 2021 [44]; REMAP-CAP 2021 [45], Barrett et al. 2021 [46], Rashidi et al. 2022 [47], Swiss COVID-HEP 2022 [48], COVID-PACT 2022 [49], Garofalo et al. 2022 [50]) with total 9 therapeutic arms (Therapeutic anticoagulation by heparin, enoxaparin or Bivalirudin n = 7, Alteplase n = 2). The control groups varied between placebo, the standard of care, and different medication regimens summarized in Table 1. All the included studies are published. COVID STEROID-1 [17] was terminated early because of evidence from other studies indicating the benefit of corticosteroids in severe COVID-19. Also, Maskin et al. [19] study terminated early due to low enrollment rate. Studies that incorporated two or more different therapy categories within the same intervention arm were excluded, as the efficacy could not be attributed specifically to one category [51–53]. Outcomes reported for the individual studies included in this analysis are reported in Table 2. Figures 2, 3, 4, 5, and 6 summarize the reported overall mortality outcomes and the estimated pooled effect. For a detailed presentation of the pooled effects for ICU, hospital, 28 days, and 90 days mortality, forest plots are provided in the Supporting Figures 6–10. 3.1. Steroid Among the included studies, 32.5% (443/1361) of the intervention cohorts and 35.8% (456/1272) of the control cohorts experienced mortality. The pooled estimated RR using a random-effects model was 0.91 (95% CI 0.82; 1.01), with nonsignificant heterogeneity (I2 = 31%, p=0.17). 3.2. Antiviral Approximately, one-third of participants in both cohorts died by the conclusion of the study period. In the intervention cohort, the mortality rate was 33.3% (159 out of 478), while it was 29.7% (364 out of 1227) in the control cohort. The pooled estimated RR using a random-effects model was 1.11 (95% CI 0.82; 1.49), and there was significant heterogeneity (I2 = 57%, p=0.04). 3.3. Immunomodulators Among the included studies, 28.6% (445/1554) of the intervention cohorts and 35.9% (589/1641) of the control cohorts experienced mortality. The pooled estimated RR using a random-effects model was 0.83 (95% CI 0.73; 0.95), with nonsignificant heterogeneity (I2 = 8%, p=0.36). 3.4. Plasma Therapies Among the included studies, 36.2% (473/1306) of the intervention cohorts and 39.1% (433/1107) of the control cohorts experienced mortality. The pooled estimated RR using a random-effects model was 0.77 (95% CI 0.58; 1.01), with nonsignificant heterogeneity (I2 = 36%, p=0.13). 3.5. Anticoagulation Among the included studies, 33.7% (387/1148) of the intervention cohorts and 32.3% (383/1187) of the control cohorts experienced mortality. The pooled estimated RR using a random-effects model was 1.06 (95% CI 0.95; 1.18), with nonsignificant heterogeneity (I2 = 0%, p=0.83). 3.6. Risk of Bias Of the RCTs, 85% (34 out of 40) reported adequate generation of the random sequence, 73% (29/40) of the RCTs reported adequate allocation sequence concealment, and 40% (16/40) of the included RCTs reported adequate blinding of the outcome assessors. A graph of the risk of bias and the assessment of each RCT are shown in Figures 7 and 8. Assessing publication bias posed challenges in this analysis, primarily due to the relatively low number of available publications. Upon visual inspection of the funnel plot, no apparent asymmetry was observed in the steroid and immunomodulators groups. The results of Egger's regression test confirmed this, yielding p values of 0.27 and 0.89, respectively. In contrast, noticeable asymmetry was detected in the antiviral, plasma therapies, and anticoagulations groups. However, Egger's regression test reached statistical significance only in the plasma therapies group (p < 0.01), while remaining nonsignificant in the antiviral (p=0.27) and anticoagulation groups (p=0.48). Funnel plot diagrams and Egger's regression test results were summarized in Supporting Diagrams 1–5. 4. Discussion This systematic review and meta-analysis of COVID-19-related therapeutic interventions exclusively in patients with critical illness revealed that many of the trials that focused on patients with critical illness were usually underpowered, from single centers or health care systems and were disproportionately from the early parts of the pandemic. Most of the large international trials did not focus adequately on outcomes related to patients with critical illness exclusively and only reported on this population as a subgroup. This systematic review also highlights that many unproven therapies including different antivirals and convalescent plasma were used in this vulnerable and highly tenuous population despite limited scientific rationale. These decisions were likely driven by fact that many health care systems globally were overwhelmed and a sentiment of using any potential therapy as a rescue intervention was justified by clinicians and researchers. This systematic review also highlights that in many cases this led to multiple interventions being introduced for many patients, making the analysis of any effectiveness studies for specific medications almost impossible. But perhaps most importantly this meta-analysis highlights the significant heterogeneity in the quality, included populations and the outcomes reported in these studies. This heterogeneity is potentially a contributing factor in the disparate results seen in many of these studies. 4.1. Steroids Similar to the findings reported by the WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group meta-analysis [54] (September 2020); using random-effects modeling [OR 0.70 (95% CI, 0.48–1.01; p=0.053)], this present study did not identify a survival advantage associated with the use of steroids. The REACT analysis included only two studies that exclusively focused on patients with critical illness, in contrast to the approach taken by the British Medical Journal's (BMJ) living network meta-analysis [55]. This study has reported a favorable pooled effect of systematic steroids on COVID-19 patient mortality regardless of the severity, (OR 0.80, 95% CI 0.65–0.94). Unfortunately, the BMJ study did not conduct subgroup analyses, making direct comparisons with present study results challenging. It is important to highlight that the landmark study, RECOVERY: COVID-19 Dexamethasone [56] was excluded from the analysis. We made this decision because the patients with critical illness were only reported as a subgroup of this study. While the RECOVERY study did report a reduced risk of 28-day mortality for patients on MV compared to those not on MV (with a relative risk of 0.64 and a 95% CI of 0.51–0.81), it's crucial to acknowledge that the definition of patients with critical illness extends beyond those solely on MV. Therefore, generalizing the results to all critically ill patients not on MV would be overly restrictive. 4.2. Antivirals Current study findings align with the limited evidence regarding the effectiveness of antiviral treatments in critically ill COVID-19 patients. Lee et al. [57] found no significant impact of Remdesivir on mortality for MV patients (RR 1.08, 95% CI, 0.88–1.31). Consistent results have been observed in a meta-analysis that included RCTs with critically ill patients only. In a recent Living Systematic Review and Meta-analysis conducted by Kaka et al. [58], (March 2022) no difference in mortality was observed (absolute risk difference −0.7% [95% CI, −2.4%–1.0%]. Also, in the living Network meta-analysis by BMJ [55] the OR of mortality was not significant in Remdesivir studies (OR 0.91, 0.73–1.11) and lopinavir-ritonavir studies (1.06, 0.88–1.28). Although none of the Remdesivir RCTs met this study inclusion, the existing evidence from the previous meta-analysis supports the nonsignificant effect for antiviral drugs in critically ill COVID patients. The DisCoVeRy trial [59] (September 2021), which examined the efficacy of Remdesivir could not find any mortality benefit at 28 days, OR 0.93 (95% CI, 0.57–1.52). Also, the largest trial that examined four different antiviral medications, the Solidarity trial [60] (February 2021), Remdesivir, Hydroxychloroquine, Lopinavir, and interferon beta-1a, had little or no effect on hospitalized patients with COVID-19, as indicated by overall mortality. Also, they calculated the pooled effect of the available trials that examined Remdesivir, Hydroxychloroquine, and Lopinavir and none of them were significant, RR 0.91 (95% CI, 0.79–1.05), 1.09 (95% CI, 0.98–1.21), 1.01 (95% CI, 0.91–1.13), respectively. RECOVERY [61] trial group reported a similar conclusion regarding Lopinavir-Ritonavir, 28-day mortality (RR 1.03, 95% CI, 0.91–1.17) that remained nonsignificant even after stratification for a critically ill patient. 4.3. Immunomodulators This study outcome closely mirror the findings of the WHO REACT [62] group, indicating that the use of IL-6 antagonists such as Tocilizumab and Sarilumab in COVID-19 patients may reduce 28-day mortality, with an OR of 0.86 and a 95% CI of 0.79–0.95 (p=0.003). A subgroup analysis failed to show a significant benefit for patients on advanced respiratory or cardiovascular support at the time of randomization. There is no available meta-analysis for Vilobelimab or anti-TNF in subgroups of patients with critical illness at the time of RCT enrollment. RCTs excluded due to subgroup analyses focusing on patients with critical illness did not demonstrate a significant mortality benefit. For instance, in the RECOVERY [63] examining Tocilizumab, the RR for 28-day mortality was as follows when stratified by levels of respiratory support: 0.81 (0.67–0.99) for no ventilator support, 0.86 (0.74–1.00) for noninvasive ventilation (NIV), and 0.93 (0.74–1.18) for MV. 4.4. Plasma Therapies Few meta-analyses examined the TPE in patients with critical COVID-19. Wardhani et al. [64] (Dec 2021) and Qin et al. [65] (Jun 2022) concluded that the TPE in patients with critical illness subgroup decreased all cause-mortality; OR 0.21 (95% CI: 0.05–0.85) and RR 0.41 (95% CI 0.24–0.69), respectively. Both studies included only one RCT in addition to five or fewer retrospective studies, which could affect the accuracy of the result and weaken the level of evidence. Two meta-analyses examined the efficacy of convalescent plasma on COVID-19 patients and conducted a subgroup analysis on patients with critical illness. Axfors et al. [66] collaborative meta-analyses (Nov 2021) reported that convalescent plasma has not added a mortality benefit for patients with critical illness (2 trials with 2315 patients); RR 0.97 (0.65–1.44), and the same for all severity patients (33 trials; 16,477 patients); 0.97 (95% CI [0.92–1.02]). The same thing was reported by Marcec et al. [67]; RR for all-cause mortality in patients with critical illness was not significant; RR 86 (0.45–1.57). Although this study results are similar to the previous two studies it is important to clarify that this meta-analysis combined convalescent plasma and TPE as one group as we thought it is more continent for categorization. Removing the TPE study from this analysis did not affect the statistical significance. This study excluded the RECOVERY [68] trial because the patients with critical illness were a subgroup. The study could not find a benefit for the convalescent plasma in COVID-19 patients regardless of the severity and on subgroup patients on MV; (RR 1.00, 95% CI 0.93–1.07) and 0.86 (0.60–1.25), respectively. 4.5. Anticoagulation This meta-analysis showed that escalated doses of anticoagulation in patients with critical COVID-19 did not have a significant benefit compared to a standard of care prophylaxis regimen, which is similar to that reported by Ortega-Paz et al. [69] (Sep 2022) meta-analysis. The author included Seven trials (5154 patients) Compared to standard-dose prophylactic anticoagulation, escalated-dose prophylactic anticoagulation was not associated with a reduction of all-cause mortality (17.8% vs. 18.6%; RR 0.96 [CI95%, 0.78–1.18]). Subgroup analysis for patients with critical illness (4 studies) was still nonsignificant; 1.03 (0.91–1.18). The author's study included the three big trials, which this study included in addition to the Perepu et al. [70] trial, which excluded in this study due to the patients with critical illness being a subgroup. Perepu et al. study had 107 patients admitted to an ICU at the time of enrollment and an intermediate dose of enoxaparin compared to a standard prophylactic dose did not lead to any improvement in all-cause mortality (HR 0.47; 95% CI, 0.19–1.15). This study has multiple strengths, including the narrow and tight inclusion criteria to help reach a more robust and precise estimate of the currently available evidence for patients with critical COVID-19 specifically and try to minimize the heterogeneity in the inclusion criteria between the included studies. But those strong points were a source of limitation for this study since they led to few included studies, so a pooled summary could not be synthesized from all the included studies for each outcome examined, due to heterogeneity in the studies' outcomes reporting. These all highlight the need for well-designed RCTs specifically including patients with critical illness only with relevant clinical outcomes to be examined. Furthermore, only studies published in English were included. This may have limited the scope of the analysis and prevented the consideration of important findings reported in non-English publications. However, it is believed that the majority of relevant studies on COVID-19 have been published in English, and efforts were made to ensure that the search strategy was comprehensive and included multiple databases to mitigate this limitation. In future studies, exploring the sources of heterogeneity through meta-regression and conducting a network meta-analysis could provide deeper insights. 5. Conclusion This meta-analysis highlights the heterogeneity in the available evidence coming from RCTs investigating different COVID-19 treatments in patients with critical illness. This systematic review and meta-analysis highlight the need to be a more robust and well-thought-out process to study patients with critical illness in future pandemics. Patients with critical illness are responsible for disproportionately high morbidity and mortality in viral pandemics, and decisions related to therapeutic interventions should not be based only on the analysis of subgroups, underpowered studies, and/or observational study designs. Critical care societies and researchers need to invest in more robust therapeutic options and appropriate study designs with adequate power for any future pandemic-related research.