Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.

Jansen JP, Naci H. · 2013

Read Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. by Jansen JP, Naci H. (2013) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3707819.txt.

Introduction

Jansen and Naci explain, in plain terms, when a network meta-analysis can be trusted as much as a standard pairwise one. Network methods compare treatments indirectly, through trials that share a common comparator, which lets many options be ranked without head-to-head studies. Their validity worries many readers.

The hinge, the authors argue, is effect modifiers: patient or study traits that change how well a treatment works. If those traits are distributed unevenly between the different kinds of direct comparisons feeding the network, the indirect conclusions are biased. If the distributions can be assumed balanced, the network result stands on the same footing as the pairwise one, whatever the network's shape.

Catalog record

CREATORJansen JP, Naci H.
YEAR2013
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC3707819

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers By Jeroen P Jansen, Huseyin Naci Published in: BMC Medicine DOI: 10.1186/1741-7015-11-159 ABSTRACT Background In the last decade, network meta-analysis of randomized controlled trials has been introduced as an extension of pairwise meta-analysis. The advantage of network meta-analysis over standard pairwise meta-analysis is that it facilitates indirect comparisons of multiple interventions that have not been studied in a head-to-head fashion. Although assumptions underlying pairwise meta-analyses are well understood, those concerning network meta-analyses are perceived to be more complex and prone to misinterpretation. Discussion In this paper, we aim to provide a basic explanation when network meta-analysis is as valid as pairwise meta-analysis. We focus on the primary role of effect modifiers, which are study and patient characteristics associated with treatment effects. Because network meta-analysis includes different trials comparing different interventions, the distribution of effect modifiers cannot only vary across studies for a particular comparison (as with standard pairwise meta-analysis, causing heterogeneity), but also between comparisons (causing inconsistency). If there is an imbalance in the distribution of effect modifiers between different types of direct comparisons, the related indirect comparisons will be biased. If it can be assumed that this is not the case, network meta-analysis is as valid as pairwise meta-analysis. Summary The validity of network meta-analysis is based on the underlying assumption that there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons, regardless of the structure of the evidence network. BACKGROUND Randomized controlled trials (RCTs) are considered as the gold standard of whether a health intervention works and/or whether it is better than another. Although often placed at the top of evidence hierarchies, single RCTs rarely provide adequate information for addressing the evidence demands of patients, clinicians and policymakers. Instead, each trial provides a piece of evidence that, when taken together with others, addresses important questions for patients, clinicians, and other healthcare decision-makers [1]. Traditional pairwise meta-analyses of RCTs are increasingly used to synthesize the results of different trials evaluating the same intervention(s) to obtain an overall estimate of the treatment effect of one intervention relative to the control. In the last decade, network meta-analysis has been introduced as a generalization of pairwise meta-analysis. When the available RCTs of interest do not all compare the same interventions but each trial compares only a subset of the interventions of interest, it is possible to develop a network of RCTs where all trials have at least one intervention in common with another. Such a network allows for indirect comparisons of interventions not studied in a head-to-head fashion [2]. For example, the treatment effects from trials comparing treatments B relative to A (AB trials) and trials comparing treatments C relative to A (AC trials) can be pooled to obtain an indirect estimate for the comparison between treatments B and C [3-5]. Even when a trial comparing treatments C and B (BC trial) exists, combining the direct estimates with the results of indirect comparisons can result in refined estimates as a broader evidence base is considered [6,7]. In general, if the available evidence base consists of a network of interlinked multiple RCTs involving treatments compared directly, indirectly, or both, the entire body of evidence can be synthesized by means of network meta-analysis [8]. Although assumptions underlying standard pairwise meta-analyses of direct comparisons are well understood, those concerning network meta-analysis for both direct and indirect comparisons might be perceived to be more complex, and might be prone to misinterpretation [9-11]. In this paper, we aim to compare pairwise meta-analysis with network meta-analysis with a specific focus on the primary role of effect modifiers as the common cause of heterogeneity and bias. We discuss effect modification first within individual trials, then in standard pairwise meta-analyses of multiple randomized trials, and finally in network meta-analyses. DISCUSSION Effect modification and within-study variation of treatment effects As a result of randomly allocating patients to the intervention or control group, well-designed RCTs achieve high internal validity by balancing (un)known and (un)measured prognostic factors across intervention groups within a trial (Figure 1). By doing so, the difference in observed outcomes in randomized trials, defined as the treatment effect (or effect size), can be attributed to differences in interventions compared (assuming that there are no other systematic differences resulting in bias, such as issues in concealment of treatment allocation) [12]. [Figure: Treatment vs study effect in a randomized controlled trial, and the role of effect modifiers.] Within an RCT, different groups of participants can respond to treatments differently. Hence, it is possible to have subgroups of participants with different treatment effects. This variation in true treatment effects is called heterogeneity and is caused by differences in patient characteristics within a trial that are effect modifiers (Figure 1) [13]. When heterogeneity occurs within an individual trial, it is referred to as within-study heterogeneity. Within-study heterogeneity occurs particularly in trials without strict entry criteria. For instance, RCTs evaluating the efficacy of cholesterol-lowering statins often include a mixture of patients with and without a history of coronary artery disease. As these subgroups of patients respond to statin therapy differently (that is, individuals with a history of coronary artery disease tend to derive greater relative mortality reduction as patients without a history of coronary artery disease), disease history is an effect modifier and results in within-study heterogeneity of treatment effects. Effect modification and between-study variation of treatment effects With standard pairwise meta-analysis, treatment effects of different studies comparing the same interventions are combined using statistical techniques to obtain a ‘pooled’ treatment effect estimate [14]. Although randomization is maintained within trials, it does not hold across the set of trials included in the meta-analysis; patients are not randomized to different trials. As a result, there are situations where there are systematic differences in study characteristics or the distribution of patient characteristics across trials. If these characteristics are effect modifiers (that is, influence the treatment effects), then there are systematic differences in treatment effects across trials: between-study heterogeneity. Hence, in a standard meta-analysis there can be both within-study heterogeneity and between-study heterogeneity (Figure 2). If meta-analyses are based on the reported treatment effects for the overall trial population in publications, there is typically no information on the within-study heterogeneity (which would be available in individual patient-level data). [Figure: Standard pairwise meta-analysis of four AB trials (comparing treatment B relative to A). (a) No differences in the effect modifier ‘disease severity’ across studies and therefore no between-study heterogeneity. (b) Extreme differences in the effect modifier across studies and therefore between-study heterogeneity. (Not useful to pool the studies). (c) Differences in the distribution of the effect modifier across studies and therefore between-study heterogeneity. Given the inclusion of both severe and moderate patients in each of the studies there is also within-study heterogeneity, but this cannot be observed without access to subgroup or individual patient-level data.] Pooling different studies in the presence of extreme between-study heterogeneity does not introduce bias, but may render the results of the meta-analysis irrelevant [15]. When considering the scenario presented in Figure 2b, the pooled result is not applicable to a moderate-only population or a severe-only population. In this

Read the complete text