Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis.
van Valkenhoef G, Dias S, Ades AE, Welton NJ. · 2016
Read Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis. by van Valkenhoef G, Dias S, Ades AE, Welton NJ. (2016) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5057346.txt.
Introduction
Gert van Valkenhoef, Sofia Dias, A. E. Ades, and Nicky J. Welton automate a consistency check for network meta-analysis in Research Synthesis Methods in 2016. Network meta-analysis combines trials of many treatments at once, mixing direct head-to-head results with indirect ones inferred through shared comparators. If the two kinds of evidence disagree, the synthesis is suspect. Node-splitting tests each comparison separately, but building each model by hand is slow.
The authors give an unambiguous rule for choosing which comparisons to split, and prove it selects exactly the comparisons sitting in loops where inconsistency could arise, no more and no fewer. They also show that under their rule, writing each model becomes trivial, sidestepping multi-arm trial tangles. The result removes most of the manual labor, letting analysts spend time judging the findings instead of building models. The methods ship for routine use in evidence synthesis.
Catalog record
| CREATOR | van Valkenhoef G, Dias S, Ades AE, Welton NJ. |
|---|---|
| YEAR | 2016 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Automated generation of node‐splitting models for assessment of inconsistency in network meta‐analysis By Gert van Valkenhoef, Sofia Dias, A. E. Ades, Nicky J. Welton Published in: Research Synthesis Methods DOI: 10.1002/jrsm.1167 ABSTRACT Network meta‐analysis enables the simultaneous synthesis of a network of clinical trials comparing any number of treatments. Potential inconsistencies between estimates of relative treatment effects are an important concern, and several methods to detect inconsistency have been proposed. This paper is concerned with the node‐splitting approach, which is particularly attractive because of its straightforward interpretation, contrasting estimates from both direct and indirect evidence. However, node‐splitting analyses are labour‐intensive because each comparison of interest requires a separate model. It would be advantageous if node‐splitting models could be estimated automatically for all comparisons of interest. We present an unambiguous decision rule to choose which comparisons to split, and prove that it selects only comparisons in potentially inconsistent loops in the network, and that all potentially inconsistent loops in the network are investigated. Moreover, the decision rule circumvents problems with the parameterisation of multi‐arm trials, ensuring that model generation is trivial in all cases. Thus, our methods eliminate most of the manual work involved in using the node‐splitting approach, enabling the analyst to focus on interpreting the results. © 2015 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd. INTRODUCTION Network meta‐analysis (Caldwell et al., 2005; Lumley, 2002; Lu and Ades, 2004) is a general framework for the synthesis of evidence from clinical trials comparing any number of treatments. It includes pair‐wise meta‐analysis (Hedges and Olkin, 1985) and indirect‐comparison meta‐analysis (Bucher et al., 1997; Song et al., 2003) as special cases (Jansen et al., 2011; Dias et al., 2013a). The key assumption underlying any meta‐analysis is exchangeability of the included trials (Lu and Ades, 2009). Violations of the exchangeability assumption can manifest as heterogeneity (within‐comparison variability) or inconsistency (between‐comparison variability). Although the most important defense against such violations is the a priori evaluation of trial design and population characteristics, the (statistical) evaluation of both heterogeneity and inconsistency is also important to ensure valid results from a network meta‐analysis. A number of methods have been proposed to detect inconsistency (Lu and Ades, 2006; Dias et al., 2010; Lu et al., 2011; Higgins et al., 2012; Dias et al., 2013b), and they can be subdivided into three classes according to their approach to inconsistency. The ‘null’ approach, consisting only of the unrelated mean effects model, does not attempt to model inconsistency at all; it simply estimates each pair‐wise comparison separately. Inconsistency is then assessed by comparing the model fit and between‐study variance (heterogeneity) estimate of the pair‐wise comparisons against the results of the consistency model (Dias et al., 2013b). The ‘loop inconsistency’ approach proposes that inconsistency can only occur in closed loops in the evidence network and is exemplified by the inconsistency factors (Lu and Ades, 2006) and node splitting (Dias et al., 2010) models. The potential for loop inconsistency was first recognised in relation to indirect treatment comparisons (Bucher et al., 1997). These models increase the power with which inconsistency can be detected by limiting the degrees of freedom in the model. However, the presence of multi‐arm trials introduces ambiguities in how these models should be specified, especially for the inconsistency factors model. The ‘design inconsistency’ approach addresses this concern by introducing the concept of design inconsistency, in which ABC trials can be inconsistent with AB trials (Higgins et al., 2012). Essentially, the design inconsistency approach allocates additional degrees of freedom to resolve the ambiguity of loop inconsistency models. We view both the design‐by‐treatment‐interaction model (Higgins et al., 2012) and the two‐stage linear inference model (Lu et al., 2011) as belonging to this approach. The design inconsistency models also enable a global test for inconsistency across the network (Higgins et al., 2012), but the loop inconsistency models do not. On the other hand, the interpretation of individual parameters of the design‐by‐treatment‐interaction model is not straightforward because, in any multiparameter model, the meaning of each parameter depends on what other parameters are in the model. Conceptually, design inconsistencies are also hard to grasp: why would three‐arm trials result in systematically different results from two‐arm or four‐arm trials? Why would the included treatments be a better predictor of inconsistency than any other design or population characteristic? Therefore, although the design inconsistency approach offers advantages, specifically unambiguous model specification and the global test for inconsistency, there are also reasons to favour the loop inconsistency approach. These are the clearer conception of inconsistency occurring in loops and the easier interpretation of local inconsistencies. The node‐splitting approach is especially attractive because inconsistency is evaluated one comparison at a time by separating the direct evidence on that comparison from the network of indirect evidence. The discrepancy between the estimates of relative treatment effects from these two sets of trials indicates the level of (in)consistency. However, node‐splitting analyses can be labour‐intensive, because each comparison of interest requires a separate model. Moreover, the analyst must decide which comparisons should be investigated, which is not trivial in the presence of multi‐arm trials. Finally, there may be several possible node‐splitting models for one comparison when it has been included in multi‐arm trials. In this paper, we present a decision rule to determine which comparisons to split that also ensures that each of the alternative node‐splitting models is valid. We build upon previous work on automated model generation for network meta‐analysis (van Valkenhoef et al., 2012a) to automatically generate the node‐splitting models. Automation is not a substitute for proper understanding of the implemented statistical methods and their limitations. Rather, it reduces the effort that well‐versed users of the methods must expend, enabling them to focus on other issues. In addition, the statistical analysis of inconsistency is not a substitute for a thoughtful selection of trials prior to applying evidence synthesis. It is also unwise to investigate inconsistency alone while ignoring heterogeneity, as the two are closely related, and in one model, the heterogeneity parameter may absorb some of the variance that another model would classify as inconsistency. Finally, when significant inconsistency or excess heterogeneity is detected, the analyst faces the difficult question of how to address it. A careful analysis of the included trials and (local) discrepancies between their effect estimates is required to identify potential confounding factors. If a satisfactory explanation is found, the synthesis may be repaired, either by excluding the offending subset of trials or by correcting for the confounder through a meta‐regression analysis. Unexplained inconsistency or heterogeneity may mean that the meta‐analysis must be abandoned altogether, or at the very least must be interpreted with extreme caution. BACKGROUND In this paper, we consider network meta‐analysis in a Bayesian framework (Dias et al., 2013a) and limit the discussion to homogeneous‐variance random‐effects models (Lu and Ades, 2004). First, we briefly review the consistency model, which is a simple extension of the Bayesian formulation of pair‐wise meta‐analysis. Then, we introduce node‐splitting models and, finally, review previous work on automated model generation for network meta‐analysis. Consistency models A network of evidence consists of a set of studies S numbered 1, …, n, where each study S i has a number of arms that evaluate a set of treatments T(S i),