Assessing the quality of decision support technologies using the International Patient Decision Aid Standards instrument (IPDASi).

Elwyn G, O'Connor AM, Bennett C, Newcombe RG, Politi M, Durand MA, Drake E, Joseph-Williams N, Khangura S, Saarimaki A, Sivell S, Stiel M, Bernstein SJ, Col N, Coulter A, Eden K, Härter M, Rovner MH, Moumjid N, Stacey D, Thomson R, Whelan T, van der Weijden T, Edwards A. · 2009

Read Assessing the quality of decision support technologies using the International Patient Decision Aid Standards instrument (IPDASi). by Elwyn G, O'Connor AM, Bennett C, Newcombe RG, Politi M, Durand MA, Drake E, Joseph-Williams N, Khangura S, Saarimaki A, Sivell S, Stiel M, Bernstein SJ, Col N, Coulter A, Eden K, Härter M, Rovner MH, Moumjid N, Stacey D, Thomson R, Whelan T, van der Weijden T, Edwards A. (2009) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2649534.txt.

Introduction

Elwyn and a 25-member international group built in 2009 a scoring tool to judge the quality of patient decision aids. These aids are leaflets, videos, and websites that help patients weigh hard treatment choices alongside their doctors. With groups worldwide making them, shared standards were needed to tell good aids from poor or biased ones.

Their tool, called IPDASi, scores each aid on 47 items grouped into ten areas, then gives a total from 0 to 100. Eight raters tried it on 30 aids chosen at random. Scores spread from 33 to 82, so the tool did separate weak aids from strong ones. Raters also agreed well with each other on the totals. The authors offer the tool as a common yardstick for makers and reviewers of decision aids.

Catalog record

CREATORElwyn G, O'Connor AM, Bennett C, Newcombe RG, Politi M, Durand MA, Drake E, Joseph-Williams N, Khangura S, Saarimaki A, Sivell S, Stiel M, Bernstein SJ, Col N, Coulter A, Eden K, Härter M, Rovner MH, Moumjid N, Stacey D, Thomson R, Whelan T, van der Weijden T, Edwards A.
YEAR2009
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC2649534

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Assessing the Quality of Decision Support Technologies Using the International Patient Decision Aid Standards instrument (IPDASi) By Glyn Elwyn, Annette M. O'Connor, Carol Bennett, Robert G. Newcombe, Mary Politi, Marie-Anne Durand, Elizabeth Drake, Natalie Joseph-Williams, Sara Khangura, Anton Saarimaki, Stephanie Sivell, Mareike Stiel Published in: PLoS ONE DOI: 10.1371/journal.pone.0004705 ABSTRACT Objectives To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids). Design Scale development study, involving construct, item and scale development, validation and reliability testing. Setting There has been increasing use of decision support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation. Methods Scale development study, involving construct, item and scale development, validation and reliability testing. Participants Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). In the fourth Stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30), enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68). Cronbach's alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbach's alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies. The existing IPDASi provides an assessment of the quality of a DST's components and will be used as a tool to provide formative advice to DSTs developers and summative assessments for those who want to compare their tools against an existing benchmark. INTRODUCTION There has been increasing interest in the use of ‘decision aids’ [1], defined as adjuncts to the discussions clinicians have with patients during deliberations about decisions: these aids provide information about options and help clarify personal values [2]. These adjuncts range from leaflets through face to face methods such as coaching or counselling to interactive multimedia websites. To describe this generic family of clinician-patient interventions we will use the term decision support technologies (DSTs) [3], corresponding with the internationally recognised need to assess the impact of ‘health technologies’ [4]. DSTs are complex interventions which require detailed assessment to ensure safe use in healthcare contexts [3] because they help make options explicit, provide information about harms and benefits, clarify patient values' and provide structured means to help people deliberate when making decisions. Although there are published methods to assess the quality of clinical practice guidelines [5], DSTs go further and address issues of equipoise for which patients need to deliberate about difficult choices [6]. However, as yet, there are no reliable methods to assure the quality of DSTs development process, content, potential bias, and method of field testing and evaluation – a gap which we address in this study. We did not intend to develop methods to assess how DSTs are used in practice, in the clinical encounter, although we recognise that this is an important area that requires further work. There are reports that DSTs have achieved a ‘tipping point’ in the US and are widely accessed by increasing numbers of patients [1]. The ability of DSTs to improve the quality of decisions and enable reductions in discretionary surgery and invasive procedures without adverse effects on health outcomes has been demonstrated in clinical trials [2], [7]. The central role that these technologies will play in future healthcare systems is increasingly recognised [1], [8]–[10]. Over the last decade, the interest in developing DSTs has moved beyond research groups and has entered the commercial world. A global interest in developing DSTs has emerged among both for-profit and not-for-profit organisations. It is therefore essential to have a set of internationally accepted standards to assess their quality, to assess whether interests are declared and whether they are unduly biased [8], [9]. The International Patient Decision Aid Standards (IPDAS) Collaboration produced a checklist for the assessment of DSTs [11]. The checklist was rigorously developed in a two stage web-based Delphi process using online rating process to enable international collaboration. A total of 122 individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed background evidence summaries, and rated the importance of 80 criteria in 12 quality dimensions. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones. The IPDAS checklist enabled broad assessments in 12 dimensions: systematic development process, providing information about options; presentation of probabilities; clarification of values; use of patient stories; information about guiding or coaching; disclosure of interests; providing internet access; balanced presentation of options; use of plain language; use of up-to-date evidence; and effectiveness. The IPDAS checklist allows users, developers and others to assess whether these technologies contain the suggested components and judge whether they underwent rigorous development and evaluation. It has been used in updating the Cochrane systematic review of DSTs and to guide the development of DSTs [12], [13]. However, the checklist was not designed to provide precise, quantitative assessments, such that judgements could be made about the quality of DSTs, either at item, dimension or global levels. In addition, because not all checklist items were applicable to every DST, comparability, even at the checklist level, was not possible. Given interest in being able to assess these DSTs at a more precise level of detail — in terms of how they were developed and field tested, whether their content was valid and whether effectiveness had been evaluated with patients facing relevant decisions — the IPDAS Collaboration agreed that achieving this objective would require an instrument capable of quantitatively assessing the quality of DSTs. The aim of this article is to describe the development, validation and inter-rater reliability of an IPDAS instrument (IPDASi), built on the existing framework. METHODS IPDASi was developed in four stages. Stage 1 Refinement and preparation of instrument (IPDASi v1) The published IPDAS checklist required transformation into a quantitative instrument, although we agreed to adopt the dimension-item framework. As part of this preparation, a group of researchers (GE, DS, RT, CB, SB, TW) used the existing checklist and dimension-item framework to score three purposefully selected DSTs, representing different design approaches and where our prior overall assessments indicated variable quality. These were Healthwise's Breast Cancer Surgery (BCS), web-based information, Bastian&McBrides Hormone Replacement Therapy (HRT), an illustrated booklet, and Wolf et al's Prostate Specific Antigen (PSA) screening, a brief text-based script. A binary (yes/no) and ‘not applicable’ scale was

Read the complete text