Clarifying differences between review designs and methods.
Gough D, Thomas J, Oliver S. · 2012
Read Clarifying differences between review designs and methods. by Gough D, Thomas J, Oliver S. (2012) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3533815.txt.
Introduction
Systematic reviews now come in many flavors, and David Gough, James Thomas, and Sandy Oliver argue in 2012 that the field lacks the words to tell them apart. Their paper resists building a fixed typology of review types, since reviews overlap along too many dimensions for clean boxes.
Instead they propose naming the dimensions themselves. Reviews vary by aims and approach, including the theory behind them and how they aggregate or configure findings. They vary by structure, how mapping and synthesis pieces fit together. And they vary by breadth, depth, and sheer work done on a question. Shared dimensional language, the authors argue, would serve planning, appraisal, and methods development better than competing brand names.
Catalog record
| CREATOR | Gough D, Thomas J, Oliver S. |
|---|---|
| YEAR | 2012 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Clarifying differences between review designs and methods By David Gough, James Thomas, Sandy Oliver Published in: Systematic Reviews DOI: 10.1186/2046-4053-1-28 ABSTRACT This paper argues that the current proliferation of types of systematic reviews creates challenges for the terminology for describing such reviews. Terminology is necessary for planning, describing, appraising, and using reviews, building infrastructure to enable the conduct and use of reviews, and for further developing review methodology. There is insufficient consensus on terminology for a typology of reviews to be produced and any such attempt is likely to be limited by the overlapping nature of the dimensions along which reviews vary. It is therefore proposed that the most useful strategy for the field is to develop terminology for the main dimensions of variation. Three such main dimensions are proposed: (1) aims and approaches (including what the review is aiming to achieve, the theoretical and ideological assumptions, and the use of theory and logics of aggregation and configuration in synthesis); (2) structure and components (including the number and type of mapping and synthesis components and how they relate); and (3) breadth and depth and the extent of ‘work done’ in addressing a research issue (including the breadth of review questions, the detail with which they are addressed, and the amount the review progresses a research agenda). This then provides an overarching strategy to encompass more detailed descriptions of methodology and may lead in time to a more overarching system of terminology for systematic reviews. BACKGROUND Research studies vary in many ways including the types of research questions they are asking, the reasons these questions are being asked, the theoretical and ideological perspectives underlying these questions, and in the research methods that they employ. Systematic reviews are a form of research; they are (and the theoretical and ideological perspectives underlying these methods) a way of bringing together what is known from the research literature using explicit and accountable methods [1]. Systematic methods of review have been successfully developed particularly for questions concerning the impact of interventions; these synthesize the findings of studies which use experimental controlled designs. Yet the logic of systematic methods for reviewing the literature can be applied to all areas of research; therefore there can be as much variation in systematic reviews as is found in primary research [2,3]. This paper discusses some of the important conceptual and practical differences between different types of systematic review. It does not aim to provide an overall taxonomy of all types of reviews; the rate of development of new approaches to reviewing is too fast and the overlap of approaches too great for that to be helpful. Instead, the paper argues that, for the present at least, it is more useful to identify the key dimensions on which reviews differ and to examine the multitude of different combinations of those dimensions. The paper also does not aim to describe all of the myriad actual and potential differences between reviews; this would be a task too large even for a book let alone a paper. The focus instead is on three major types of dimensions of difference. The first dimension is the aims and approaches of reviews; particularly in terms of their methodologies (their ontological and epistemological foundations and methods of synthesis). The second dimension is the structure and components of reviews. The third dimension is the breadth, depth, and extent of the work done by a review in engaging with a research issue. Once these three aspects of a review are clear, consideration can be given to more specific methodological issues such as methods of searching, identifying, coding, appraising, and synthesizing evidence. The aim of this paper is to clarify some of the major conceptual distinctions between reviews to assist the selection, evaluation, and development of methods for reviewing. Clarifying the nature of variation in reviews As forms of research, systematic reviews are undertaken according to explicit methods. The term ‘systematic’ distinguishes them from reviews undertaken without clear and accountable methods. The history of systematic reviews is relatively recent [4,5] and despite early work on meta-ethnography [6], the field has been dominated by the development and application of statistical meta-analysis of controlled trials to synthesize the evidence on the effectiveness of health and social interventions. Over the past 10 years, other methods for reviewing have been developed. Some of these methods aim to extend effectiveness reviews with information from qualitative studies [7]. The qualitative information may be used to inform decisions made in the statistical synthesis or be part of a mixed methods synthesis (discussed later). Other approaches have been developed from a perspective which, instead of the statistical aggregation of data from controlled trials, emphasize the central role that theory can play in synthesizing existing research [8,9], address the complexity of interventions [10], and the importance of understanding research within its social and paradigmatic context [11]. The growth in methods has not been accompanied by a clear typology of reviews. The result is a complex web of terminology [2,12]. The lack of clarity about the range of methods of review has consequences which can limit their development and subsequent use. Knowledge or consensus about the details of specific methods may be lacking, creating the danger of the over-generalization or inappropriate application of the terminology being used. Also, the branding of different types of review can lead to over-generalizations and simplification with assumptions being made about differences between reviews that only apply to particular stages of a review or that are matters of degree rather than absolute differences. For example, concepts of quality assurance can differ depending upon the nature of the research question being asked. Similarly, infrastructure systems developed to enable the better reporting and critical appraisal of reviews, such as PRISMA [13], and for registration of reviews, such as PROSPERO [14] currently apply predominantly to a subset of reviews, the defining criteria of which may not be fully clear. A further problem is that systematic reviews have attracted criticism on the assumption that systematic reviewing is applicable only to empirical quantitative research [15]. In this way, polarized debates about the utility and relevance of different research paradigms may further complicate terminological issues and conceptual understandings about how reviews actually differ from one another. All of these difficulties are heightened because review methods are undergoing a period of rapid development and so the methods being described are often being updated and refined. Knowledge about the nature and strengths of different forms of review is necessary for: appropriate choice of review methods by those undertaking reviews; consideration of the importance of different issues of quality and relevance for each stage of a review; appropriate and accurate reporting and accountability of such review methods; interpretation of reviews; commissioning of reviews; development of procedures for assessing and undertaking reviews; and development of new methods. Clarifying the nature of the similarities and differences between reviews is a first step to avoiding these potential limitations. A typology of review methods might be a solution. There are many diverse approaches to reviews that can be easily distinguished, such as statistical meta-analysis and meta-ethnography. A more detailed examination, however, reveals that the types of review currently described often have commonalities that vary across types of review and