Approaches to interpreting and choosing the best treatments in network meta-analyses.

Mbuagbaw L, Rochwerg B, Jaeschke R, Heels-Andsell D, Alhazzani W, Thabane L, Guyatt GH. · 2017

Read Approaches to interpreting and choosing the best treatments in network meta-analyses. by Mbuagbaw L, Rochwerg B, Jaeschke R, Heels-Andsell D, Alhazzani W, Thabane L, Guyatt GH. (2017) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5389085.txt.

Introduction

Lawrence Mbuagbaw and six co-authors explain how to read treatment rankings in a 2017 paper in Systematic Reviews. Their topic is the network meta-analysis, a method that pools many trials of rival treatments for one health problem. It can compare treatments that never met in a single trial by using shared rivals. Yet its ranked lists can be hard for doctors to grasp and easy to misuse.

The authors walk through ranking tools, charts, and one common score that sums a treatment's rank chances. They weigh what each tool shows well and where each can mislead. A worked example shows how a ranked result should turn into a care choice, with care for weak proof and close calls. The paper serves readers who must act on pooled trial results.

Catalog record

CREATORMbuagbaw L, Rochwerg B, Jaeschke R, Heels-Andsell D, Alhazzani W, Thabane L, Guyatt GH.
YEAR2017
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC5389085

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Approaches to interpreting and choosing the best treatments in network meta-analyses By L Mbuagbaw, B Rochwerg, R Jaeschke, D Heels-Andsell, W Alhazzani, L Thabane, Gordon H Guyatt Published in: Systematic Reviews DOI: 10.1186/s13643-017-0473-z ABSTRACT Abstract When randomized trials have addressed multiple interventions for the same health problem, network meta-analyses (NMAs) permit researchers to statistically pool data from individual studies including evidence from both direct and indirect comparisons. Grasping the significance of the results of NMAs may be very challenging. Authors may present the findings from such analyses in several numerical and graphical ways. In this paper, we discuss ranking strategies and visual depictions of rank, including the surface under the cumulative ranking (SUCRA) curve method. We present ranking approaches’ merits and limitations and provide an example of how to apply the results of a NMA to clinical practice. Keywords: Ranking, SUCRA, Network meta-analysis, Advantages, Limitations BACKGROUND Systematic reviews of randomized clinical trials (RCTs) provide crucial information for determining the effect of interventions in clinical practice [1]. Typically, investigators statistically combine treatment effect estimates (effect sizes) from individual clinical trials [2]. Traditional meta-analyses compare a single intervention to a single alternative (direct pair-wise comparisons) [3]. In many clinical contexts, clinicians consider more than two alternative treatments, each of which may have been compared to standard care, a placebo, or an alternative intervention. Because some interventions have never been compared to a placebo, or lack head-to-head direct comparisons, choosing between a number of alternatives creates challenges for determining their relative merit [4]. A solution to the multiple alternative problem that uses an entire body of evidence with all available direct and indirect comparisons—termed network meta-analysis (NMA) or multiple treatment comparison meta-analysis—is seeing increasing use [5]. In addition to providing information on the relative merits of interventions that have never been directly compared, NMAs may also increase the precision of effect estimates by combining both direct and indirect evidence. However, the results of NMAs may be complex and difficult to interpret for clinicians especially when there are many alternative strategies and outcomes to consider [6, 7]. Guidance on how to interpret findings from NMAs remains limited [8]. To address interpretation challenges, NMA authors can complement numerical data with graphical tools [9–11] and by ranking interventions. Indeed, some form of ranking is reported in two thirds of all published NMAs [7], and experts recommend ranking as a form of presentation [12]. Other discussions have addressed reporting options, including ranking approaches, often assuming that readers have a sophisticated knowledge of analytic methods [9, 10]. Our objective here is not to be technical or comprehensive, but rather to discuss the merits and limitations of ranking methods with a specific focus on surface under the cumulative ranking (SUCRA) curve, a popular ranking method. RANKING TREATMENTS Clinicians wish to offer patients a choice among the most desirable treatment options. Though a treatment that is certain to be the best in terms of the most important benefit outcome (e.g. a reduction in risk of stroke) would be a strong candidate for the treatment of choice, it might also carry more harms than other options (e.g. greatest risk of bleeding, or greatest burden). Moreover, results of studies are always associated with uncertainty and we will seldom, if ever, be sure a treatment is best. Rather, we can think of the likelihood that, for a particular outcome, a treatment is best, or near best. Of two treatments that are unlikely to be the best, the treatment with a higher likelihood of being second best would—all else being equal—be preferable to one with a lower likelihood of being second best. Ranks can be presented graphically and numerically. The graphical approaches involve examining the area under the curve indicating the probability of each drug to occupy a specific rank. These graphs are daunting to compare, especially when many treatments and outcomes are examined. The surface under the cumulative ranking curve (SUCRA) is a numeric presentation of the overall ranking and presents a single number associated with each treatment. SUCRA values range from 0 to 100%. The higher the SUCRA value, and the closer to 100%, the higher the likelihood that a therapy is in the top rank or one of the top ranks; the closer to 0 the SUCRA value, the more likely that a therapy is in the bottom rank, or one of the bottom ranks. APPLYING THESE METHODS TO A REAL-LIFE EXAMPLE An NMA studied the impact of alternative resuscitative fluids on mortality in adult patients with sepsis [13]. We present here the results from an analysis that divided the intervention into six categories: albumin, balanced crystalloid, saline, gelatin, heavy starch and light starch. Figure 1 depicts the rankings of these six treatments. [Figure: Graphical ranking of resuscitation fluids in six-node analysis] From Fig. 1, we can see that balanced crystalloids have the highest likelihood of being ranked first, followed by albumin, gelatin and heavy starch; the results suggest no possibility that light starch and saline lead to the lowest mortality. For the second rank, balanced crystalloids and albumin still appear most likely and light starch and saline least likely, but heavy starch now has a higher likelihood than gelatin. Gelatin, the two starches, and saline are more likely to be among the lower ranks (3 to 6), and albumin and balanced crystalloid far less likely to be among the lower ranks. Looking across the figures, you could make an intuitive estimate of the rankings, and the gradient in effect across treatments. Table 1 presents the SUCRA results that emerge from these data. The SUCRA rankings confirm that balanced crystalloid and albumin are most likely to result in the lowest mortality (with quite similar SUCRA scores) while light starch appears appreciably less attractive than the other alternatives. [Table: SUCRA rankings from six-node analysis] FIVE REASONS WHY THESE RANKINGS MAY MISLEAD IF NOT INTERPRETED CORRECTLY Taking these results at surface value, clinicians should now be resuscitating all their septic patients with a balanced crystalloid solution. There are, however, several reasons why clinicians should not routinely choose a treatment with the higher SUCRA ranking. First, the evidence on which the SUCRA rankings are based may be of very low quality (synonyms: low certainty or confidence) and therefore untrustworthy. Second, there are typically several relevant outcomes. A treatment that is best in one outcome (say, a benefit outcome) may be the worst in another outcome (for example, a harm outcome). Third, issues such as cost and a clinician’s familiarity with use of a particular treatment may also bear consideration. Fourth, in the process of calculation, SUCRA does not consider the magnitude of differences in effects between treatments (e.g. in a particular simulation the first ranked treatment may be only slightly, or a great deal better than the second ranked treatment). Fifth, chance may explain any apparent difference between treatments, and SUCRA does not capture that possibility. In this case, clinicians may easily misinterpret the apparently clear hierarchy in the efficacy of these fluids in reducing mortality. Table 2 presents a more detailed summary of the evidence, including the number of direct comparisons, the direct, indirect and network estimates and their associated credible intervals, and the certainty (quality, confidence) of the evidence. [Table: NMA results

Read the complete text