Meta-DiSc: a software for meta-analysis of test accuracy data.
Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. · 2006
Read Meta-DiSc: a software for meta-analysis of test accuracy data. by Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. (2006) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc1552081.txt.
Introduction
Reviews of test accuracy studies had no software of their own. Zamora and co-authors built one in 2006, called Meta-DiSc. Such reviews pool studies of how well a medical test spots disease. Analysts had to borrow general stats tools.
Meta-DiSc checks why results differ between studies and pools sensitivity and specificity, the two core accuracy measures. It handles both fixed and random effects models and draws forest plots and summary curves ready for papers. Its math was checked against other tools and published reviews. Zamora offers the Windows program free for academic use.
Catalog record
| CREATOR | Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. |
|---|---|
| YEAR | 2006 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Meta-DiSc: a software for meta-analysis of test accuracy data By Javier Zamora, Victor Abraira, Alfonso Muriel, Khalid Khan, Arri Coomarasamy Published in: BMC Medical Research Methodology DOI: 10.1186/1471-2288-6-31 ABSTRACT Abstract Background Systematic reviews and meta-analyses of test accuracy studies are increasingly being recognised as central in guiding clinical practice. However, there is currently no dedicated and comprehensive software for meta-analysis of diagnostic data. In this article, we present Meta-DiSc, a Windows-based, user-friendly, freely available (for academic use) software that we have developed, piloted, and validated to perform diagnostic meta-analysis. Results Meta-DiSc a) allows exploration of heterogeneity, with a variety of statistics including chi-square, I-squared and Spearman correlation tests, b) implements meta-regression techniques to explore the relationships between study characteristics and accuracy estimates, c) performs statistical pooling of sensitivities, specificities, likelihood ratios and diagnostic odds ratios using fixed and random effects models, both overall and in subgroups and d) produces high quality figures, including forest plots and summary receiver operating characteristic curves that can be exported for use in manuscripts for publication. All computational algorithms have been validated through comparison with different statistical tools and published meta-analyses. Meta-DiSc has a Graphical User Interface with roll-down menus, dialog boxes, and online help facilities. Conclusion Meta-DiSc is a comprehensive and dedicated test accuracy meta-analysis software. It has already been used and cited in several meta-analyses published in high-ranking journals. The software is publicly available at http://www.hrc.es/investigacion/metadisc_en.htm. BACKGROUND Accurate diagnosis forms the basis of good clinical care, as without it one can neither prognosticate correctly nor choose the right treatment. Indeed, a wrong diagnosis can harm patients by exposing them to inappropriate or sub-optimal therapy [1]. Thus studies of diagnostic accuracy, and particularly their systematic reviews and meta-analyses, are being recognised as instrumental in underpinning evidence-based clinical practice. Initiatives such as STARD [2] and developments within the Cochrane Collaboration [3] to accept protocols and reviews of test accuracy studies highlight the emphasis being given to evidence-based diagnosis. Currently, there is only one test accuracy meta-analysis package, Meta-Test [4], which addresses some of the unique statistical issues related to test accuracy, such as pooling of sensitivities and specificities and summary receiver operating characteristics (sROC) analysis. However, it is a DOS-based application with an interface that many find difficult to use, and integrate into Windows-based applications. Moreover, it lacks crucial analytical tools such as pooling of likelihood ratios (LRs), tests for heterogeneity and meta-regression facilities. We, therefore, developed, piloted and validated a comprehensive, Windows-based test accuracy meta-analysis software, Meta-DiSc, which is presented in this article, with a worked example. IMPLEMENTATION Meta-DiSc software was created in Microsoft Visual Basic 6, and some mathematical routines have been linked from the NAG C mathematical library [5]. The software is distributed as a single file, downloadable freely from URL: http://www.hrc.es/investigacion/metadisc_en.htm. Its installation is simple, guided by onscreen instructions. The programme has a user-friendly interface with roll-down menus, dialog boxes and online HTML compiled help files. These help files include a user manual and a description of the implemented statistical methods. Meta-DiSc allows data entry into its datasheet in three different ways: a) directly by typing data into the datasheet using the keyboard, b) copying from another spreadsheet (e.g. Microsoft Excel) and pasting into Meta-DiSc datasheet, or c) importing text files from other sources (for example, in the comma delimited format). Several variables can be defined in the datasheet, including study identifiers, accuracy data from each study (true positives, false positives, true negatives and false negatives) and study level co-variates, such as those defining population spectrum or methodological quality of the studies. Once the data have been entered into the datasheet of Meta-DiSc, various statistical analyses can be implemented (Figure 1). The implementation of these statistical procedures needs to be carefully thought through and judicious, as it may be inappropriate (or indeed misleading) to use all the procedures (particularly statistical pooling) in all reviews. Meta-DiSc provides analysts with adequate tools to assess the appropriateness of pooling. Readers interested in details of these methods are referred to statistical methods section of the help files (also available as a PDF standalone document [6] and to existing texts and guidelines on diagnostic meta-analysis [7-10]. [Figure: Available tools in Meta-DiSc. Tools implemented in the software Meta-DiSc to perform different steps of meta-analysis of diagnostic tests accuracy.] Describing the results of individual studies When describing accuracy results from several studies, it is important to get an indication of the magnitude and precision of the accuracy estimates derived from each study, as well as to assess the presence or absence of inconsistencies in accuracy estimates across studies (heterogeneity). As accuracy estimates are paired and often inter-related (sensitivity and specificity, or LR positive and LR negative), it is necessary to report these simultaneously [11]. One accuracy measure that combines these paired measures is diagnostic odd ratio (dOR) [12], which has limited clinical use, although useful in procedures like meta-regression (see below). Meta-DiSc computes accuracy estimates and confidence intervals from individual studies and shows results either as numerical tabulations or graphical plots in two formats: a) forest plots, for sensitivities, specificities, LRs or dOR, with respective confidence intervals; and b) plots of individual study results in ROC space, with or without an sROC curve. Exploring heterogeneity (threshold effect) Exploring heterogeneity is a critical issue to a) understand the possible factors that influence accuracy estimates, and b) to evaluate the appropriateness of statistical pooling of accuracy estimates from various studies. One of the primary causes of heterogeneity in test accuracy studies is threshold effect, which arises when differences in sensitivities and specificities or LRs occur due to different cut-offs or thresholds used in different studies to define a positive (or negative) test result. When threshold effect exists, there is a negative correlation between sensitivities and specificities (or a positive correlation between sensitivities and 1-specificities), which results in a typical pattern of "shoulder arm" plot in a sROC space [8]. It is worth noting that correlation between sensitivity and specificity could arise due to a number of reasons other than threshold (e.g. partial verification bias, different spectrum of patients or different settings). Meta-DiSc allows assessment for threshold effect in three different ways: a) visual inspection of relationship between pairs of accuracy estimates in forest plots. If threshold effect is present, the forest plots will show increasing sensitivities with decreasing specificities, or vice versa. The same inverse relationship will be apparent with LR positive and LR negative; b) representation of accuracy estimates from each study in a sROC space – a typical "shoulder arm" pattern would suggest presence of threshold effect; and c) computation of Spearman correlation coefficient between the logit of sensitivity and logit of 1-specificity. A strong positive correlation would suggest threshold effect. Exploring for heterogeneity (other than threshold effect) Apart from variations due to threshold effect, there are several other factors that can result in variations in accuracy estimates amongst different test accuracy studies in a review. These reasons include chance as well as variations in study population (e.g. severity of disease and co-morbidities), index test (differences in technology, assays, operator etc.), reference standard, and the way a study was designed and conducted [13]. Since such heterogeneity is almost always present in accuracy systematic reviews, testing for the presence