Contrasting Diversity Values: Statistical Inferences Based on Overlapping Confidence Intervals Ian MacGregor-Fors1*, Mark E. Payton2 1 Red de Ambiente y Sustentabilidad, Instituto de Ecologı́a, A.C., Xalapa, Veracruz, México, 2 Department of Statistics, Oklahoma State University, Stillwater, Oklahoma, United States of America Abstract Ecologists often contrast diversity (species richness and abundances) using tests for comparing means or indices. However, many popular software applications do not support performing standard inferential statistics for estimates of species richness and/or density. In this study we simulated the behavior of asymmetric log-normal confidence intervals and determined an interval level that mimics statistical tests with P(a) = 0.05 when confidence intervals from two distributions do not overlap. Our results show that 84% confidence intervals robustly mimic 0.05 statistical tests for asymmetric confidence intervals, as has been demonstrated for symmetric ones in the past. Finally, we provide detailed user-guides for calculating 84% confidence intervals in two of the most robust and highly-used freeware related to diversity measurements for wildlife (i.e., EstimateS, Distance). Citation: MacGregor-Fors I, Payton ME (2013) Contrasting Diversity Values: Statistical Inferences Based on Overlapping Confidence Intervals. PLoS ONE 8(2): e56794. doi:10.1371/journal.pone.0056794 Editor: Christopher Quince, University of Glasgow, United Kingdom Received September 7, 2012; Accepted January 14, 2013; Published February 20, 2013 Copyright: ß 2013 MacGregor-Fors, Payton. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: The authors have no support or funding to report. Competing Interests: The authors have declared that no competing interests exist. * E-mail: ian.macgregor@inecol.edu.mx calculated using the software Distance [13], for which asymmetric 95% confidence intervals, based on assuming the distributions of the density estimate is log-normal, are output as a default by the program. As software programs such as EstimateS and Distance output results that cannot be contrasted directly though inferential statistics, degree of overlap between confidence intervals has been proposed to assess statistical differences [14]. Such comparisons allow testing null hypotheses regarding different environmental conditions (e.g., habitats, treatments). Although other approaches to hypothesis testing for Distance have been shown to contrast density values effectively (e.g., ANOVA, t-tests), they often require experience using sophisticated processes in statistical packages. As demonstrated by Payton et al. [14], when comparing overlapping 95% confidence intervals of independent treatments with similar standard errors, non-overlapping confidence intervals represent significant differences in expectations with extremely low probabilities of Type I error (a ,0.01), while no statistical inferences can be drawn with certainty if confidence intervals overlap but are not coincident. However, Payton et al. [14] showed that comparing 83–84% confidence intervals, instead of 95%, represents statistical tests with an a level of 0.05 (Fig. 1), the conventional criterion of significance for biological and ecological analyses [15]. As the 83–84% rule has previously been demonstrated only for normally distributed confidence intervals, in this study we simulated how asymmetric log-normal confidence intervals behave and determined a confidence interval level for mimicking twosample statistical tests with a = 0.05. As the log-normal distribution is a normal distribution on the log-scale, we predicted that the 83– 84% rule should also apply to asymmetric log-normal confidence intervals. We also describe how to calculate different percentage Introduction Measuring biodiversity is one of the major goals of ecologists around the world [1]. As suggested by Hubbell [2], biodiversity can be summarized by the species richness and relative abundances of a community in a given space and time. For decades, ecologists have used many different methods to calculate and contrast species richness, relative abundances, and/or diversity values. Most simply, ecologists often contrast species richness and abundances relative to sampling effort among different conditions using tests for comparing means (e.g., ANOVA, Kruskal-Wallis; [3–5]). Also, many indices have been developed to measure species richness and diversity (see Moreno [1,6–8] for further details). However, many popular software applications do not support performing standard inferential statistics for estimates of diversity (e.g., species richness, density). Recently, the use of two methods for quantifying species richness and individual densities have became very popular due to their robustness: (1) rarefaction curves produced by randomly resampling the pool of total individuals or sampling units, plotting the estimated number of species in relation to a given number of individuals or sampling units [9–11], and (2) distance-sampling calculation of densities (number of individuals per area unit - e.g., hectares, square kilometers), calculated based on the probability of detection of individuals at increasing distances from the observer and the size of the successfully surveyed area [8]. Both methods can be calculated using freeware. Rarefaction curves can be generated using the output from the software EstimateS [12], which computes the expected number of species as a function number of accumulated samples (sample-based rarefaction, denoted Sobs [Mao Tao] in EstimateS) with symmetric 95% confidence intervals (Sobs 95% CI Upper and Lower Bounds). Densities can be PLOS ONE | www.plosone.org 1 February 2013 | Volume 8 | Issue 2 | e56794 Confidence Intervals in Diversity Statistics Results For almost all of the scenarios contrasting samples with different means, the 84% confidence intervals provided overlap probability that best mimicked a two-tailed two population test with a 0.05 error rate. To mimic a 0.01 test, 94% confidence intervals would appear to be the proper choice. Confidence intervals at the 76% level best mimic a test with a 0.10 error rate (Tables 1, 2). Discussion As predicted, our results show that comparing the overlap, or lack of it, between 84% asymmetric confidence intervals pertaining to different means mimics 0.05 tests surprisingly well (Fig. 1, 2). Thus, this study provides empirical evidence that the 84% rule is suitable for mimicking 0.05 statistical tests for both symmetric and asymmetric confidence intervals. However, we did not explore the statistical power of the method (regarding Type II errors), since the primary concern of this paper was to create a process that best mimicked an alpha-level test, and the use of overlapping 84% confidence intervals for this method would be more powerful, by definition, than using 95% intervals. Assessing power for this situation would involve constructing distributions with different means (and, by virtue of the nature of the log-normal distribution, different variances) and assessing the ability of the method to detect differences in overlapping confidence intervals with different means. Though our results have been demonstrated effective only for normal symmetric intervals and for log-normal asymmetric intervals, we believe that the 84% rule for mimicking 0.05 tests with overlapping confidence intervals might work effectively for other distributions. For example, comparing 84% confidence intervals for species estimation comparisons using widely used non-parametric estimators (e.g., Chao1, Chao2, ICE, ACE, Jackknife, Bootstrap), could mimic 0.05 tests. However, it remains to be tested. Figure 1. Hypothetical scenario comparing diversity values with 95% and 84% confidence intervals. In the example on the left (A vs. B), with 95% confidence intervals, no conclusion can be drawn regarding statistical difference in diversity values at P = 0.05. In the example on the right (A9 vs B9), with 84% confidence intervals but the same means as on the left, we can confidently infer that diversity values differ at P,0.05. doi:10.1371/journal.pone.0056794.g001 confidence intervals for rarefaction curves and distance-sampling based densities and indicate how to contrast them, representing a novel way to statistically compare species richness and density values robustly. Materials and Methods Simulations for Mimicking Pairwise Tests Based on Asymmetric Confidence Intervals Table 1. Simulation results of 10,000 iterations calculating the overlap of confidence intervals of various sizes generated from log-normal populations with mean of 12.2 and variance of 0.08 (log-normal parameter values of m = 2.5 and s2 = 0.0005). We performed simulations to establish the confidence intervals at which P,0.01, 0.05, and 0.10 Type I error was achieved, mimicking pairwise tests with PC SAS [16]. In order to explore how the proposed method behaves for various types of log-normal distributions, we created several combinations of the two parameters of the log-normal distribution (m and s). Specifically, we created 48 different log-normal distributions by utilizing 6 different levels for m and 8 different levels for s in an effort to cover a variety of different distributions. For the purposes of these simulations, we generated samples from parent populations which were generated by assuming different means and corresponding standard errors, which are functions of the parameters utilized to create these parent populations. Thus, as we assessed the behavior of asymmetric confidence intervals, we calculated a confidence interval for each of two samples drawn from the same population, each with alpha values varying from 0.05 to 0.25, at 0.01 increments. We calculated 10,000 iterations of each simulation scenario, including populations with different means extracted from the same parent populations. For each iteration, we calculated 0.75% to 0.95% confidence intervals in 1% increments, and we used this series of confidence intervals to determine the proportion of times the simulated confidence intervals overlap for each nominal level of confidence. Note that the log-normal distribution’s coefficient of variation is a function of s only [17], so changing the mean of the distribution changes, by definition, the variance also. PLOS ONE | www.plosone.org Size of CIs (%) Average overlap Size of CIs (%) Average overlap 95 0.9946 84* 0.9543* 94 0.9898 83 0.9494 93 0.9899 82 0.9448 92 0.9855 81 0.9312 91 0.9838 80 0.9318 90 0.9802 79 0.9276 89 0.9768 78 0.9149 88 0.9687 77 0.909 87 0.9671 76* 0.9026* 86 0.9624 75 0.8966 85 0.9608 Values that represent the preferred choice of confidence interval to mimic tests with alpha of 0.05 and 0.10 are marked with an asterisk (*). This table represents only one set of parameter values considered (among 48 sets considered) and is meant to represent typical results associated with the other population values considered in our simulations. doi:10.1371/journal.pone.0056794.t001 2 February 2013 | Volume 8 | Issue 2 | e56794 Confidence Intervals in Diversity Statistics Table 2. Appropriate sizes of confidence intervals to simulate P = 0.05 and P = 0.10 size tests for various combinations of lognormal parameter values and associated means and variances. m s2 Mean 4.5 0.0005 90.04 0.001 Variance Size of CIs (%) Average overlap 4.05 84 90.06 8.12 0.0015 90.08 12.18 0.002 90.11 16.25 0.0025 90.13 20.33 0.003 90.15 24.41 0.0035 90.17 28.51 0.004 32.61 90.2 5.5 0.0005 244.75 0.001 244.81 0.0015 244.88 0.002 244.94 0.0025 245 0.003 245.06 0.0035 245.12 0.004 245.18 6.5 0.0005 665.31 0.001 665.47 0.0015 665.64 0.002 665.81 0.0025 665.97 0.003 29.96 59.96 90.01 120.11 150.25 180.43 210.66 240.94 221.37 443.08 665.12 887.49 1110.19 666.14 1333.23 0.0035 66.31 1556.6 0.004 666.47 1780.3 0.9541 76 0.9032 84 0.9546 76 0.903 84 0.9507 76 0.9049 84 0.953 76 0.9036 83 0.9506 76 0.9065 84 0.9541 75 0.9001 84 0.9566 77 0.9114 83 0.9505 76 0.9042 84 0.952 77 0.9071 83 0.9517 76 0.9043 84 0.9554 75 0.904 84 0.9509 76 0.9049 85 0.9582 76 0.901 84 0.9534 76 0.9022 84 0.9525 76 0.9057 83 0.9514 76 0.9041 84 0.9533 76 0.9039 84 0.9535 76 0.9011 84 0.9557 75 0.9017 84 0.9524 76 0.9012 84 0.9517 76 0.901 83 0.9508 77 0.9091 83 0.9504 76 0.9042 84 0.9521 76 0.9028 Figure 2. Comparison of the use of 95% and 84% confidence intervals in three replicates of our simulations. For this representative example, the data were created from a log-normal population with a mean of 90.2 and variance of 32.6. In case 1, the both sets of intervals overlap, both suggesting that no significant (NS) differences exist. Note, however, that the 95% confidence intervals will yield an error rate of less than 1%, while the 84% confidence intervals better mimic a 0.05 level test. In case 2, 95% confidence intervals slightly overlap, while 84% ones do not. For this situation, these two approaches would lead to different conclusions: (a) significant differences (*) when considering 84% confidence intervals, and (b) no statistical differences can be inferred using 95% confidence intervals (?). In case 3, none of the sets of intervals overlap, both suggesting that significant differences exist. Note, however, that statistical differences using 95% confidence intervals are assumed with an error rate of less than 1%, while that of 84% confidence intervals better mimic a 0.05 level test. doi:10.1371/journal.pone.0056794.g002 In order to generate 84% confidence intervals for rarefaction analyses, the standard deviation of the observed species (Mao Tao SD) from the output file from EstimateS is needed. As standard deviations equal standard errors in EstimateS because infinite degrees of freedom are assumed in the calculation of Mao Tao SD, the latter must be multiplied by 1.372, the quantile (normal curve z-score) corresponding to two-sided intervals of 84% probabilities, with alpha = 0.16, and cumulative probabilities of 0.08 and 0.92. For example, if Mao Tao SD = 5.55, for example, 84% confidence intervals for that specific value of Mao Tao SD, which can vary in relation to the number of accumulated computed individuals in a rarefaction plot, are equal to the average value 67.61. As the Distance program can calculate user-selected levels for confidence intervals (default = 0.95) for distance-sampling density calculations, setting the confidence interval limits solves the issue. To accomplish this, go the ‘‘Analyses’’ button on the toolbar, select ‘‘Analysis details’’ and a new window will appear. Finally, select the ‘‘Misc’’ tab and modify the default value for confidence intervals (i.e., 95) to 84. Results output from the Distance program will now include 84% confidence intervals. Wildlife species richness and density measurements of ecosystems are imperative in order to concentrate conservation actions in highly biodiverse areas [1]. In this paper, we demonstrated that the 84% rule mimics 0.05 pairwise statistical tests for both symmetric and asymmetric confidence intervals, with detailed users’ guides for calculating 84% confidence intervals in two of the most robust and highly-used freeware applications related to biodiversity (i.e., EstimateS, Distance). Thus, we encourage ecologists to use these programs to calculate species richness and individual Results from simulations including 10,000 iterations. doi:10.1371/journal.pone.0056794.t002 PLOS ONE | www.plosone.org 3 February 2013 | Volume 8 | Issue 2 | e56794 Confidence Intervals in Diversity Statistics density statistical expectations, applying this easy-to-use overlapping confidence interval method when making statistical inferences, which represents an alternative to the use of diversity indices. Author Contributions Conceived and designed the experiments: IMF MEP. Performed the experiments: MEP. Analyzed the data: MEP IMF. Contributed reagents/ materials/analysis tools: MEP IMF. Wrote the paper: IMF MEP.