Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table.
Lewin S, Bohren M, Rashidian A, Munthe-Kaas H, Glenton C, Colvin CJ, Garside R, Noyes J, Booth A, Tunçalp Ö, Wainwright M, Flottorp S, Tucker JD, Carlsen B. · 2018
Read Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. by Lewin S, Bohren M, Rashidian A, Munthe-Kaas H, Glenton C, Colvin CJ, Garside R, Noyes J, Booth A, Tunçalp Ö, Wainwright M, Flottorp S, Tucker JD, Carlsen B. (2018) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5791047.txt.
Introduction
Lewin and co-authors explain how to make an overall CERQual confidence rating for a review finding. This is the second paper in a guidance series. The method judges findings from reviews of qualitative research. Each finding is rated on its own, using four set parts.
The parts cover method limits, coherence, data adequacy, and relevance. Judgments on the parts combine into one of four confidence levels. Two displays carry the result: an evidence profile and a summary table. The guidance was piloted on real reviews before release. Worked examples show each step in practice for readers.
Catalog record
| CREATOR | Lewin S, Bohren M, Rashidian A, Munthe-Kaas H, Glenton C, Colvin CJ, Garside R, Noyes J, Booth A, Tunçalp Ö, Wainwright M, Flottorp S, Tucker JD, Carlsen B. |
|---|---|
| YEAR | 2018 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table By Simon Lewin, Meghan Bohren, Arash Rashidian, Heather Munthe-Kaas, Claire Glenton, Christopher J. Colvin, Ruth Garside, Jane Noyes, Andrew Booth, Özge Tunçalp, Megan Wainwright, Signe Flottorp Published in: Implementation Science : IS DOI: 10.1186/s13012-017-0689-2 ABSTRACT Background The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on making an overall assessment of confidence in a review finding and creating a CERQual Evidence Profile and a CERQual Summary of Qualitative Findings table. Methods We developed this guidance by examining the methods used by other GRADE approaches, gathering feedback from relevant research communities and developing consensus through project group meetings. We then piloted the guidance on several qualitative evidence syntheses before agreeing on the approach. Results Confidence in the evidence is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. Creating a summary of each review finding and deciding whether or not CERQual should be used are important steps prior to assessing confidence. Confidence should be assessed for each review finding individually, based on the judgements made for each of the four CERQual components. Four levels are used to describe the overall assessment of confidence: high, moderate, low or very low. The overall CERQual assessment for each review finding should be explained in a CERQual Evidence Profile and Summary of Qualitative Findings table. Conclusions Structuring and summarising review findings, assessing confidence in those findings using CERQual and creating a CERQual Evidence Profile and Summary of Qualitative Findings table should be essential components of undertaking qualitative evidence syntheses. This paper describes the end point of a CERQual assessment and should be read in conjunction with the other papers in the series that provide information on assessing individual CERQual components. Electronic supplementary material The online version of this article (10.1186/s13012-017-0689-2) contains supplementary material, which is available to authorized users. BACKGROUND GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative Research) is an approach for assessing how much confidence to place in review findings from systematic reviews of qualitative research or qualitative evidence syntheses (QES). The approach is being developed by the GRADE-CERQual Project Group, a subgroup of the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The importance of assessing confidence in qualitative evidence is discussed in the first paper in this series [1]. The GRADE-CERQual approach (hereafter referred to as CERQual) is applied to individual review findings from a qualitative evidence synthesis [1]. We define a review finding as an analytic output (e.g. a theme, category, thematic framework, theory or contribution to theory) from a qualitative evidence synthesis that, based on data from primary studies, describes a phenomenon or an aspect of a phenomenon. By ‘phenomenon’, we mean the issue that is the focus of the qualitative inquiry—that is, ‘what we want our research to understand…[]… explain, or describe’ [2] (p129). How review findings are defined and presented depends on factors such as the review question, the synthesis methods utilised and the intended purpose or audience of the synthesis [3]. For example, the aims of different approaches to QES range from identifying and describing key themes, to seeking more generalizable or interpretive explanations that can be used for building theory [3, 4]. GRADE-CERQual includes four components for assessing how much confidence to place in review findings: (1) the methodological limitations of the individual qualitative studies contributing to a review finding, (2) the coherence of the review finding, (3) the adequacy of data supporting a review finding and (4) the relevance of the data from the primary studies supporting a review finding to the context (perspective or population, phenomenon of interest and/or setting) specified in the review question. Making an overall assessment of confidence in a review finding involves moving from the judgements made for each CERQual component to a final assessment. The overall assessment of confidence in a review finding takes into account the concerns identified in relation to each of the four components and, in our experience, is an iterative process that benefits from discussion among the review team or those making the CERQual assessment (when this is being done for an existing synthesis). AIM The aim of this paper, the second in this series (Fig. 1), is to describe and discuss the process for making an overall assessment of confidence in a review finding and to outline how to create a CERQual Evidence Profile and a Summary of Qualitative Findings table. A key part of communicating an overall CERQual assessment is providing an explanation for the confidence assessment, based on the concerns identified in relation to each component. The review team therefore needs to create a Summary of Qualitative Findings table to display a summary of each review finding, the CERQual assessment of confidence in that finding and the explanation for that assessment.Fig. 1Overview of the GRADE-CERQual series of papers In this paper, we discuss the following processes: firstly, moving from a review finding to a summary of a review finding; secondly, determining the review findings to which CERQual should be applied; thirdly, making an overall CERQual assessment of confidence in each individual review finding; and, finally, creating a CERQual Evidence Profile and a Summary of Qualitative Findings (SoQF) table. Papers 3, 4, 5 and 6 in the series describe how to assess each CERQual component (i.e. methodological limitations [5], coherence [6], adequacy [7], and relevance [8]). These component papers are closely related to this paper on making an overall CERQual assessment of confidence and creating a Summary of Qualitative Findings table. We have placed this paper before the four CERQual component papers as we think that it will be helpful for readers to understand how the component assessments will be used before discussing the details of how to apply each component. Key definitions are provided in Additional file 1. How CERQual was developed The initial stages of the process for developing CERQual, which started in 2010, are outlined elsewhere [3]. Since then, we have used a number of methods to further refine the definitions of each component and the principles for application of the overall approach. We used a pragmatic approach to develop the methods for making an overall CERQual assessment of confidence and creating a Summary of Qualitative Findings table. We examined the methods used by other GRADE approaches, talked to experts in the field of qualitative evidence synthesis, developed consensus through multiple face-to-face CERQual project group meetings and teleconferences, and gained feedback from ongoing engagement with the qualitative evidence synthesis community. We presented an early version of the