Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis.

Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. · 2018

Read Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. by Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. (2018) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc6194404.txt.

Introduction

Taylor and co-authors present a mixed methods study in BMJ Open comparing rapid and traditional qualitative analysis. Clinical leaders often need findings fast. Rapid analysis promises speed, but its validity is questioned. The team tested it against thematic analysis on the same dataset.

The data came from a UK evaluation of a hospital home birth service, gathered in late 2014. Two teams worked independently, one using framework-based thematic analysis, the other a rapid approach. The study measured researcher time and how far the outputs agreed. Many findings matched, but the rapid team missed some points, including midwife training needs. The authors weigh when rapid methods are safe to use for timely decisions.

Catalog record

CREATORTaylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S.
YEAR2018
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC6194404

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis By Beck Taylor, Catherine Henshall, Sara Kenyon, Ian Litchfield, Sheila Greenfield Published in: BMJ Open DOI: 10.1136/bmjopen-2017-019993 ABSTRACT Objectives This study compares rapid and traditional analyses of a UK health service evaluation dataset to explore differences in researcher time and consistency of outputs. Design Mixed methods study, quantitatively and qualitatively comparing qualitative methods. Setting Data from a home birth service evaluation study in a hospital in the English National Health Service, which took place between October and December 2014. Two research teams independently analysed focus group and interview transcript data: one team used a thematic analysis approach using the framework method, and the second used rapid analysis. Participants Home birth midwives (6), midwifery support workers (4), commissioners (4), managers (6), and community midwives (12) and a patient representative (1) participated in the original study. Primary outcome measures Time taken to complete analysis in person hours; analysis findings and recommendations matched, partially matched or not matched across the two teams. Results Rapid analysis data management took less time than thematic analysis (43 hours vs 116.5 hours). Rapid analysis took 100 hours, and thematic analysis took 126.5 hours in total, with interpretation and write up taking much longer in the rapid analysis (52 hours vs 8 hours). Rapid analysis findings overlapped with 79% of thematic analysis findings, and thematic analysis overlapped with 63% of the rapid analysis findings. Rapid analysis recommendations overlapped with 55% of those from the thematic analysis, and thematic analysis overlapped with 59% of the rapid analysis recommendations. Conclusions Rapid analysis delivered a modest time saving. Excessive time to interpret data in rapid analysis in this study may be due to differences between research teams. There was overlap in outputs between approaches, more in findings than recommendations. Rapid analysis may have the potential to deliver valid, timely findings while taking less time. We recommend further comparisons using additional data sets with more similar research teams. Strengths and limitations of this studyOur study explores a strategy to address the time-lag in reporting qualitative findings to clinicians and policymakers, which slows translation of research into practice.This is the first comparison of qualitative analytical methods in applied health research which compares both researcher time and outputs, with a complete study dataset.The work describes the process of comparing time and analytical outputs in detail, to inform others planning further methodological comparisons.Due to the time lag in thematic analysis outputs, our study did not triangulate findings with the original participants.The study uncovered important challenges in comparing analytical approaches between research teams which can inform the design future work in this area. BACKGROUND Applied health research frequently adopts mixed methods, often using qualitative approaches.1 Applications of qualitative methods include: early work to identify areas for focus; throughout a study to explore processes and user experience; and following a trial or intervention implementation to explain outcomes and/or identify stakeholder experiences, to explore in more depth questions or issues identified through quantitative work and to problematise or ‘unpack’ issues or topics taken for granted.2 Increasingly this type of research can include a broader range of contributors, for example, where members of the public, patients, clinicians and researchers are involved in analysing and interpreting data to ensure a multidisciplinary perspective or pragmatically using several researchers to code data in the interests of time.3 4 Typically stakeholders want rapid results,5–7 yet compared with quantitative approaches, traditional qualitative approaches often considerable time is required to manage and interpret data and deliver findings.8 9 In a service context, delays may render the findings out of date, reducing their applicability and relevance. There are examples of apparently more rapid alternatives to traditional qualitative approaches, including specific end-to-end approaches such as Rapid Assessment Process and Rapid Ethnography.6 9–13 There are four broad areas where time can be saved: by reducing data collection time, for example, by allowing less time between data collection episodes6; by reducing data management time, for example, by relying on untranscribed audio recordings, notes, summaries and mind maps10–12; by minimising the time spent analysing data by summarising as opposed to formally coding11 13; and by limiting the time spent on analysis by using a ‘one sheet of paper’ summary to explore a sample of a large precoded dataset.9 Often rapid methods describe a broad approach, including activities from entering the field through to delivery of findings and/or involve mixed methods.6 7 This paper specifically explores whether rapid analysis (RA) of qualitative data (distinct from end-to-end rapid methods) delivers equivalent findings to traditional approaches and how much time may be saved in practice. There are a limited number of studies that have compared different qualitative analytical techniques.11 14–16 In some of the empirical examples identified, methodologists have predominantly compared methods of data collection (eg, interviews vs internet forums14) and focused on the number and content of codes rather than interpretation. Of the three examples identified that compare analytical approaches, one used focus group data to compare thematic analysis (TA) of a partial dataset with mind-mapping of a full dataset.11 While this paper provides minimal detail regarding the method of comparison, it reported differences in time taken to analyse the data and in the number and presentation of codes. The second example compared software-assisted and constant comparative approaches to analysis describing differences in the frequency of codes and coding levels.15 The third example compared analysis of focus group data directly from audio recordings, with TA of transcribed data, and found that themes generated were comparable.16 The work we present here was conducted as part of the Collaboration for Leadership in Applied Health Research and Care (CLAHRC) programme in the West Midlands of England. CLAHRC involves local teams across universities and National Health Service organisations working in partnership to deliver research to improve services for patient benefit.17 As part of a service evaluation study of a new home birth service, we gathered interview, focus group and documentary data. We then compared the speed and outputs of rapid and traditional techniques applied to the same dataset. For the RA, we used the approach developed by Hamilton at UCLA.13 We compared this with TA18 and the framework method, which was selected due to the team’s existing familiarity with this approach and the fact that it is increasingly applied in multidisciplinary health services research.8 19 METHOD This study compares rapid and traditional analyses of a UK health service evaluation dataset to explore differences in researcher time and consistency of outputs. This was a mixed methods study, quantitatively and qualitatively comparing the outputs of qualitative methods. SETTING The data came from a home birth service evaluation study in a hospital in the English National Health Service, which took place between October and December 2014. This was a service innovation put into place by the hospital. A dedicated team of midwives was set up to provide antenatal, birth and postnatal care to women choosing to have a home birth, with the aim of providing a more reliable service and increasing the local home birth rate. Characteristics of participants Home birth midwives (6), midwifery support workers (4), commissioners (4), managers (6) and community midwives (12) and a

Read the complete text