Time-dependent ROC curve analysis in medical research: current methods and applications.

Kamarudin AN, Cox T, Kolamunnage-Dona R. · 2017

Read Time-dependent ROC curve analysis in medical research: current methods and applications. by Kamarudin AN, Cox T, Kolamunnage-Dona R. (2017) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5384160.txt.

Introduction

Adina Najwa Kamarudin, Trevor Cox, and Ruwanthi Kolamunnage-Dona review time-based ROC curve methods in a 2017 paper in BMC Medical Research Methodology. ROC curves judge how well a test or marker sorts people who will get a disease from those who will not. The classic form treats health and marker values as fixed, yet both can change as a study goes on.

The authors sort through the methods built to handle that change over time, for single and repeated marker measures. They note which software can run each method and show the steps on data from a liver disease trial. They find many methods exist but few clinical studies use them, and they urge wider use.

Catalog record

CREATORKamarudin AN, Cox T, Kolamunnage-Dona R.
YEAR2017
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC5384160

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Time-dependent ROC curve analysis in medical research: current methods and applications By Adina Najwa Kamarudin, Trevor Cox, Ruwanthi Kolamunnage-Dona Published in: BMC Medical Research Methodology DOI: 10.1186/s12874-017-0332-6 ABSTRACT Background ROC (receiver operating characteristic) curve analysis is well established for assessing how well a marker is capable of discriminating between individuals who experience disease onset and individuals who do not. The classical (standard) approach of ROC curve analysis considers event (disease) status and marker value for an individual as fixed over time, however in practice, both the disease status and marker value change over time. Individuals who are disease-free earlier may develop the disease later due to longer study follow-up, and also their marker value may change from baseline during follow-up. Thus, an ROC curve as a function of time is more appropriate. However, many researchers still use the standard ROC curve approach to determine the marker capability ignoring the time dependency of the disease status or the marker. Methods We comprehensively review currently proposed methodologies of time-dependent ROC curves which use single or longitudinal marker measurements, aiming to provide clarity in each methodology, identify software tools to carry out such analysis in practice and illustrate several applications of the methodology. We have also extended some methods to incorporate a longitudinal marker and illustrated the methodologies using a sequential dataset from the Mayo Clinic trial in primary biliary cirrhosis (PBC) of the liver. Results From our methodological review, we have identified 18 estimation methods of time-dependent ROC curve analyses for censored event times and three other methods can only deal with non-censored event times. Despite the considerable numbers of estimation methods, applications of the methodology in clinical studies are still lacking. Conclusions The value of time-dependent ROC curve methods has been re-established. We have illustrated the methods in practice using currently available software and made some recommendations for future research. Electronic supplementary material The online version of this article (doi:10.1186/s12874-017-0332-6) contains supplementary material, which is available to authorized users. BACKGROUND In a screening process, an appropriate marker is used to provide information on the individual risk of disease onset. Information and signalling of future disease identification may be given by a single continuous measurement marker or a score. A single measurement could be any clinical measure such as cell percentage in the synthesis phase to detect breast cancer [1], CD4 cell counts to detect AIDS [2] or HIV-1 RNA to detect HIV [3]. A score from a regression of potential factors or some other model to detect disease can also be used as a marker. Chambless and Diao [4] used the score from a logistic regression model, including several traditional and newer risk factors, to detect Coronary Heart Disease (CHD). Lambert and Chevret [5] used the prognostic score of four covariates (age, platelet count, prothrombin time, and serum alpha-fetoprotein level) to predict compensated cirrhosis patients’ survival and also used a score of three baseline characteristics (age, white blood cell and performance status) to predict event-free survival (EFS) in acute leukaemia patients. Moreover, some studies used a published score as a marker in which the score considers the most important mortality predictors of a certain disease. For example, the Framingham risk score is used for cardiovascular patients [6] and the Karnofsky score is used for lung cancer patients [7]. The decision from a diagnostic test is often based on whether the marker value exceeds a threshold value, in which case the diagnosis for the individual is “diseased” and “non-diseased” otherwise. There is a possibility that the diagnostic test gives a positive result for a non-diseased individual or a negative result for a diseased individual. The sensitivity is defined as the probability of a diseased individual being predicted as having the disease (true-positive) and the specificity as the probability of a non-diseased individual being predicted as not having the disease (true-negative). These probabilities change as the threshold value for the marker changes and the value or range of threshold values chosen depends on the trade-off that is acceptable between failing to detect disease and falsely identifying disease with the test [8]. In relation to this, the receiver operating characteristic (ROC) curve is a tool that simply describes the range of trade-offs achieved by a diagnostic test. ROC curve analysis is extensively used in biomedical studies for evaluating the diagnostic accuracy of a continuous marker. It is a graphical display which plots sensitivity estimates (probability of a true positive) against one minus specificity (probability of a false positive) of a marker for all possible threshold values. The performance of a marker is evaluated by the area under the ROC curve (AUC) in which a higher AUC value indicates a better marker performance. The AUC is also equal to the probability of a diseased individual having a higher marker value than a healthy individual [8]. It is usually assumed that a higher marker value is more indicative of disease [8, 9] and we assume this for the rest of this article. Recent research has incorporated time dependency in the sensitivity and specificity in disease (event)-time data for individuals instead of using the standard ROC curve approach. Such methods are proven more effective; however, these methods are still under-used in medical research. Once the time-dependent setting has been applied, the disease status is observed at each time point which yields different values of sensitivity and specificity throughout the study. Let T i denote the time of disease onset and X i is a marker value (usually the value at baseline) for individual i, (i = 1, …, n). Define the observed event time, Z i = min(T i, C i), where C i is a censoring time, and let δ i be the censoring indicator taking value 1 if an event (disease) occurs and 0 otherwise. Let D i(t) be the disease status at time t, taking values 1 or 0. Hereafter, we will refer to X as a “marker”, but X may also denote a risk score computed from a regression or some other model, or a published score. For a given threshold c, the time-dependent sensitivity and specificity can defined respectively by\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \begin{array}{l} Se\left( c, t\right)= P\left({X}_i> c\Big|{D}_i(t)=1\right)\\ {} Sp\left( c, t\right)= P\left({X}_i\le c\Big|{D}_i(t)=0\right).\end{array} $$\end{document}Sect=PXi>c|Dit=1Spct=PXi≤c|Dit=0. Using the above definitions, we can define the corresponding ROC curve for any time t as ROC(t) which plots Se(c,t) against 1-Sp(c,t) for thresholds c and time-dependent AUC is defined by\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ A U C(t)={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}} S e\left( c, t\right) d\left[1- Sp\left( c, t\right)\right] $$\end{document}AUCt=∫−∞∞Sectd1−Spct with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \left[1- Sp\left( c, t\right)\right]=\frac{\partial \left[1- Sp\left( c, t\right)\right]}{\partial c} dc. $$\end{document}1−Spct=∂1−Spct∂cdc. The AUC is equal to the probability that the diagnostic test results from a randomly selected pair of diseased and non-diseased individuals are correctly ordered [10, 11]. Heagerty and Zheng [12] proposed three different definitions for estimating the above time-dependent sensitivity and specificity for censored event-times, namely (1) cumulative/dynamic (C/D), (2) incident/dynamic (I/D) and (3) incident/static (I/S) and these are explained by referring to the illustrations in Fig. 1(a) and (b) below. Figure 1(a) and (b) illustrate the cases and controls that

Read the complete text