Estimation of significance thresholds for genomewide association scans.

Dudbridge F, Gusnanto A. · 2008

Read Estimation of significance thresholds for genomewide association scans. by Dudbridge F, Gusnanto A. (2008) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2573032.txt.

Introduction

Dudbridge and Gusnanto tackled a practical puzzle in 2008. It concerns genome-wide association studies. How strong must a signal be to count as real, when a million markers are tested at once? Theory had suggested thresholds, but nobody had checked them against real data. The team used genotypes collected by the Wellcome Trust Case-Control Consortium, a landmark British study collection.

They thinned the markers to increasing densities. Using permutation, shuffling the data to build a null picture, they found the cut-off giving a 5 percent family-wise error rate. Extrapolating to infinite density gave a genome-wide threshold near 7.2 times ten to the minus eight. A shortcut based on eigenvalues underestimated the true number of tests tenfold. The authors conclude permutation is still needed, but thresholds can be standardised per population.

Catalog record

CREATORDudbridge F, Gusnanto A.
YEAR2008
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC2573032

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Estimation of significance thresholds for genomewide association scans By Frank Dudbridge, Arief Gusnanto Published in: Genetic Epidemiology DOI: 10.1002/gepi.20297 ABSTRACT Abstract The question of what significance threshold is appropriate for genomewide association studies is somewhat unresolved. Previous theoretical suggestions have yet to be validated in practice, whereas permutation testing does not resolve a discrepancy between the genomewide multiplicity of the experiment and the subset of markers actually tested. We used genotypes from the Wellcome Trust Case-Control Consortium to estimate a genomewide significance threshold for the UK Caucasian population. We subsampled the genotypes at increasing densities, using permutation to estimate the nominal P-value for 5% family-wise error. By extrapolating to infinite density, we estimated the genomewide significance threshold to be about 7.2 × 10−8. To reduce the computation time, we considered Patterson's eigenvalue estimator of the effective number of tests, but found it to be an order of magnitude too low for multiplicity correction. However, by fitting a Beta distribution to the minimum P-value from permutation replicates, we showed that the effective number is a useful heuristic and suggest that its estimation in this context is an open problem. We conclude that permutation is still needed to obtain genomewide significance thresholds, but with subsampling, extrapolation and estimation of an effective number of tests, the threshold can be standardized for all studies of the same population. Keywords: multiple testing, permutation test, Bonferroni, eigenvalue INTRODUCTION The question of what strength of evidence should be considered significant has yet to be fully resolved in genetic association analysis. On the one hand, multiple testing issues arise in most studies, whether based on candidate genes or genomewide scans, with attendant issues of how to quantify the multiplicity, what error rate to control and which method to use [Manly et al., 2004]. On the other hand, even under strong control of the type–1 error, many associations have not been replicated and are thought to have been false positives. This disappointing aspect can be attributed to the use of traditional thresholds of significance, even after adjustment for multiple testing, which reflect over–optimistic prior belief in the tested hypotheses [Ioannidis, 2005]. In the current period of genomewide association scans, using a dense but incomplete panel of single nucleotide polymorphism (SNP) markers [Barrett and Cardon, 2006], such considerations take on greater importance owing to the high profile and cost of these studies. Here we discuss some aspects of assessing significance in genomewide scans, and estimate a genomewide significance threshold for the UK Caucasian population using data from the recently completed Wellcome Trust Case-Control Consortium [2007] (WTCCC) study. We allow for a potentially saturated dense marker panel to distinguish the genomewide multiplicity of the experiment from the set of markers actually tested. This approach has a long history in linkage analysis [Morton, 1955; Lander and Kruglyak, 1995] but has generally not been taken up in association mapping. Our approach brings together previous ideas based on the hypothesized extent of multiplicity in the genome, with emerging marker data that allow this multiplicity to be estimated. The multiple testing problem arises because, if many hypotheses are tested simultaneously, some test statistics will be surprisingly extreme, even if no associations exist. Multiple test procedures are designed to exercise control over the entire set of hypotheses, to prevent study–wide conclusions being drawn that could be attributed to chance alone. The family–wise error rate (FWER) is the probability of committing at least one type–1 error, and may be controlled in the weak sense, when all null hypotheses are true, or in the strong sense, when any subset of hypotheses is true [Hochberg and Tamhane, 1987]. More recently, the false discovery rate (FDR) [Benjamini and Hochberg, 1995] and variations [Efron and Tibshirani, 2002] have gained support, as we may tolerate some type–1 errors so long as they are a small proportion of the rejected hypotheses. Bayes factors have also been advocated to quantify the strength of evidence in each test [WTCCC, 2007]. Here we are not concerned with discriminating between different error measures, but note that when the number of false hypotheses is small, control of the standard FDR is close to weak control of the FWER [Benjamini and Hochberg, 1995]. Moreover, in this situation strong control of FWER is close to the best methods for weak control [Dudbridge and Koeleman, 2003]. Also when the evidence is strong and the power is high, there is a strong correlation between the Bayes factor and the P–value [Thomas and Clayton, 2004]. In what follows we will therefore consider strong control of FWER by the Bonferroni method, or its permutational equivalent [Westfall and Young, 1993], as a conservative baseline to which other methods can be calibrated. The first well–known suggestion for a genomewide significance threshold for association was by Risch and Merikangas [1996]. They assumed a gene-centric study of 100,000 genes, with an average of five SNPs tested in each gene, leading to a Bonferroni significance threshold of 5 × 10−8 for one–sided tests of each allele. Although these assumptions are now obsolete, this threshold has proved remarkably durable, and valid in the sense that, to our knowledge, no association reported as exceeding it has proved a false positive. Another estimate for a gene-centric study assumed 30,000 genes, with 10 genes having normally distributed effects, leading to a threshold of 1 × 10−6 for a genewide test [Clayton, 2003; Thomas and Clayton, 2004]. More recently, attention has shifted from gene-centric studies to genomewide scans of anonymous SNPs. In this context the International HapMap Consortium [2005] used permutation tests of high density genotypes in 10 genomic regions to estimate an “effective number of independent tests” of 150 per 500 kbp. Scaling up to a 3 Gb genome suggests a significance threshold of 5.5 × 10−8 for two–sided tests of SNPs. The only threshold that has to date been rigorously applied to more than one scan is that of the WTCCC [2007]. Assuming 106 independent regions of the genome, 10 disease–causing genes and average power 50% to detect an associated gene, posterior odds of 10:1 in favour of association would be achieved by a nominal P–value of 5 × 10−7. With genomewide SNP data now available, permutation tests have been suggested to obtain appropriately adjusted P–values [Churchill and Doerge, 1994]. The difficulty is that standard permutation tests only account for the markers actually tested, whereas the multiplicity spans the whole genome. This is important because although P–values are the usual output of classical tests, the real quantities of interest are the posterior odds for association or the closely related false–positive reporting probability [Wacholder et al., 2004] and positive predictive value [Ioannidis, 2005]. For a class of tests significant when a statistic T>t, the posterior odds may be expressed in terms of the prior odds as [formula: (1)] where α, Beta are the type–1 and type–2 error rates of the test, respectively. Consideration of the prior odds is intrinsic to the Bayesian perspective, but it is also implicit in frequentist testing, if it is to be used responsibly. Suppose one chose a single SNP at random, tested for association to a trait and obtained a P–value of exactly 0.05. As one test was performed, no correction for multiplicity is required, and from the

Read the complete text