Making trials matter: pragmatic and explanatory trials and the problem of applicability.

Treweek S, Zwarenstein M. · 2009

Read Making trials matter: pragmatic and explanatory trials and the problem of applicability. by Treweek S, Zwarenstein M. (2009) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2700087.txt.

Introduction

Treweek and Zwarenstein argued in 2009 that medical trials are often built to answer narrow questions cleanly, then fail the real world they were meant to guide. A randomized trial is the best tool for judging whether a treatment works. But randomization alone says nothing about whether results apply to other patients, clinics, or countries.

The authors contrast two attitudes. Explanatory trials test a treatment under ideal, tightly controlled conditions. Pragmatic trials test it under the messy conditions of routine care, with ordinary patients and staff. Most trials lean explanatory, so their results travel poorly, and users of research must guess at applicability. The review sets out how to judge that leap honestly. If evidence is to steer practice and policy, they conclude, more trials should be designed pragmatically from the start.

Catalog record

CREATORTreweek S, Zwarenstein M.
YEAR2009
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC2700087

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Making trials matter: pragmatic and explanatory trials and the problem of applicability By Shaun Treweek, Merrick Zwarenstein Published in: Trials DOI: 10.1186/1745-6215-10-37 ABSTRACT Abstract Randomised controlled trials are the best research design for decisions about the effect of different interventions but randomisation does not, of itself, promote the applicability of a trial's results to situations other than the precise one in which the trial was done. While methodologists and trialists have rightly paid great attention to internal validity, much less has been given to applicability. This narrative review is aimed at those planning to conduct trials, and those aiming to use the information in them. It is intended to help the former group make their trials more widely useful and to help the latter group make more informed decisions about the wider use of existing trials. We review the differences between the design of most randomised trials (which have an explanatory attitude) and the design of trials more able to inform decision making (which have a pragmatic attitude) and discuss approaches used to assert applicability of trial results. If we want evidence from trials to be used in clinical practice and policy, trialists should make every effort to make their trial widely applicable, which means that more trials should be pragmatic in attitude. INTRODUCTION The statistical experiment, or, as we know it in medicine, the randomised controlled trial (RCT) is among the more beautiful intersections between man and mathematics. RCTs minimise the risk of bias (threats to internal validity), particularly selection bias [1,2] and are thus the best research design for decisions about the effect of different interventions, be they treatments, therapies or delivery methods and policies. But, as Cochrane noted, there is a snag: randomisation does not, of itself, promote external validity; that is, the applicability of a trial's results to situations other than the precise one in which the trial was done [3]. It is thus possible for a trial to be free of bias but be of no relevance beyond the immediate setting, patients, and practitioners among whom it was conducted. This question of applicability is central to those who have to choose between therapies for groups of patients (policymakers), for their own patients (clinicians) or for themselves (patients and families). How likely is it, these decision makers may ask, that this treatment (apparently successful in this trial or review) will achieve important benefits in my context, administered to me by my clinicians, by me to my patients, or by clinicians to patients in my organisation? In other words, 'Are these published findings applicable to my decision?'. This narrative review is aimed at those planning to conduct trials, and those aiming to use the information in them. It is intended to help the former group make their trials more widely useful and to help the latter group make more informed decisions about the wider use of existing trials. We review the differences between the design of most RCTs and the design of trials more able to inform decision making, discuss some approaches used to assert applicability and end by proposing: 1. That applicability should be explicitly considered by trialists as they plan their trial, and by decision makers when gathering evidence to guide decisions. 2. That trialists explicitly design their trials to produce results that are more widely applicable than is the case at present. 3. That trialists can and should report their trials in ways that make it easier for others to make judgements about their applicability. 4. That decision makers seek trials with a pragmatic attitude to inform choices that directly affect clinical care, health services delivery and health policies. 5. That researchers conduct, and research funders fund, empirical research to understand the major determinants of applicability. EXPLANATORY AND PRAGMATIC ATTITUDES TO TRIALS Two French statisticians, Schwartz and Lellouch, were acutely aware over 40 years ago of the limited applicability of many trial results beyond the artificial, 'laboratory' environment of the trial [4]. They proposed a distinction between trials aimed at confirming a physiological hypothesis, precisely specified as a causal relationship between administration of an intervention and some physiological outcome (which they called 'explanatory') and the entirely different group of trials aimed at informing a clinical, health service or policy decision, where this decision involves the choice between two or more interventions (called 'pragmatic'). While explanatory trials have an important role in providing knowledge concerning the effects of precisely defined interventions applied to select groups under optimal conditions, healthcare interventions are seldom given under such circumstances [5,6]. Moreover, inadequate consideration of applicability is the most frequent criticism by clinicians of randomised trials, systematic reviews and guidelines [7,8]. For example, a clinician considering a treatment for secondary prevention of stroke might read results from the Heart Outcomes Prevention Evaluation (HOPE) trial and wonder what to make of the long list of exclusion criteria, the exclusion of nearly 1 in 10 of the remaining patients because of non-adherence, side-effects or withdrawal of consent in the pretrial run-in phase, and the use of placebo as comparator rather than aspirin [9]. Calls for more trials with wide applicability have come both from those interested in improved treatment for clinical problems [10-12] and those interested in health policy [13,14]. Schwartz and Lellouch characterised pragmatism as an attitude to trial design rather than a characteristic of the trial itself. Although some authors appear to suggest that a trial is either explanatory or pragmatic [15], there is a continuum rather than a dichotomy between explanatory and pragmatic trials with the pragmatic attitude explicitly favouring design choices that maximise applicability of the trial's results to usual care settings. As Schwartz and Lellouch wrote: ' [m]ost trials done hitherto have adopted the explanatory approach without question; the pragmatic approach would often have been more justifiable'. As summarised in [16-18], we are aware of only a single study that has attempted to identify pragmatic trials (identified using: MeSH term 'clinical trial', keyword 'pragmatic' and authors' judgement that identified studies described clinical trials with a pragmatic attitude) and it found just 95 published between 1976 and 2002 [19]. Since PubMed identifies over 168,000 RCTs for that period, trials with a pragmatic attitude are clearly the exception even if we make allowances for Vallvé et al.'s rather narrow search [19]. This is at least in part due to the requirements of regulatory agencies, especially the US Food and Drug Administration (FDA) [16-18]. Although the FDA offers little guidance on the design of trials, what guidance there is argues against trials with a pragmatic attitude: ' [T]here are numerous ways of conducting a study that can obscure differences between treatments, such as poor diagnostic criteria, poor methods of measurement, poor compliance, medication errors, or poor training of observers. As a general statement, carelessness of all kinds will tend to obscure differences between treatments. Where the objective of a study is to show a difference, investigators have powerful stimuli toward assuring study excellence' [20]. The FDA equates explanatory design choices with study excellence, thereby favouring trials that lack the attributes needed to support decisions about the applicability of a treatment or therapy to usual practice [16-18]. Conversely, the clinical, policy and funding decision makers who are expected to use

Read the complete text