Assessing Genome-Wide Statistical Significance for Large p Small n Problems

Abstract

Assessing genome-wide statistical significance is an important issue in genetic studies. We describe a new resampling approach for determining the appropriate thresholds for statistical significance. Our simulation results demonstrate that the proposed approach accurately controls the genome-wide type I error rate even under the large p small n situations.

QUANTITATIVE trait loci (QTL) mapping plays an important role in understanding the genetic variations in experimental crosses. A critical issue concerns assessing the genome-wide significance (GWS), since statistical tests are performed at many putative loci. Analytic methods to determine GWS have been investigated by several authors including Lander and Kruglyak (1995) and Zou et al. (2004) and these involve specific assumptions on the experimental design and genetic map density. Churchill and Doerge (1994) proposed a permutation test to address these issues. However, their method is computationally intensive due to repeated analyses of the permuted data sets, and its validity relies on the assumption of complete exchangeability under the null hypothesis, which can frequently be violated (see, for instance, Manichaikul et al. 2007).

To overcome the limitations of the permutation methods, Zou et al. (2004) proposed a resampling procedure requiring one analysis of the data set only, thereby reducing the computational complexity. In this note, we propose a further modification of the resampling approach of Zou et al. (2004) to improve the power of the tests while retaining the same computational complexity.

We begin by considering n independent subjects from an experimental cross and statistical testing at p putative loci. Let β_j be a vector of the genetic effects at the jth location and H₀: β₁ = … = β_p = 0 denote the null hypothesis of no genetic effects at all loci. It is well known that given a statistical model, the likelihood-ratio test for testing the hypothesis H_0j: β_j = 0 can be approximated by the score statistic

$$W_j=U_j^{\mathrm{T}}{V}_j^{-1}{U}_j,$$

where $U_j={\displaystyle {\sum }_{i=1}^n{U}_{ij}}$, $V_j={\displaystyle {\sum }_{i=1}^n{U}_{ij}{U}_{ij}^{\text{T}}}$, and U_ij is the efficient score from the ith subject, defined to be the projection of the score function for β_j on the orthocomplement space of the score functions for nuisance parameters (Bickel et al. 1993, p. 30). The test statistic for testing H₀ is max_1≤j≤p W_j, whose null distribution can be approximated by the resampling algorithm of Zou et al. (2004) given below. Theoretical justification for this approximation can be provided along the line of Kuelbs and Vidyashankar (2010). The resampling algorithm for determining the threshold at GWS level α (Zou et al. 2004) follows:

k = 0

repeat

k ← k + 1

$G_i\left(k\right)\stackrel{\text{i.i.d.}}{\sim }N\left(0,1\right),\hspace{0.5em}i=1,\cdots ,n$

$U_j^{\mathrm{\ast }}\left(k\right)={\displaystyle {\sum }_{i=1}^n{U}_{ij}{G}_i\left(k\right)},\hspace{0.17em}{W}_j^{\mathrm{\ast }}\left(k\right)=U_j^{\mathrm{\ast }\text{T}}\left(k\right)V_j^{-1}{U}_j^{\mathrm{\ast }}\left(k\right)$

$W^{\mathrm{\star }}\left(k\right)={\text{max}}_{1\le j\le p}{W}_j^{\mathrm{\star }}\left(k\right)$

untilk ≥ B

Calculate the 100(1 − α)th percentile of {W⋆(1), …, W⋆(B)}

We modify the above algorithm by generating $G_i\left(k\right)\stackrel{\text{i.i.d.}}{\sim }2\times \text{Bernoulli}\left(0.5\right)-1$; i.e., G_i(k)’s are i.i.d. from the Rademacher distribution, since the error in approximating the distribution of the score statistic is of the order n^−3/2 when using Rademacher weights (RW) while it is 3n^−3/2 for N(0, 1) weights. This distribution is commonly used in the multiplier bootstrap literature (Praestgaard 1990), econometrics, and learning theory (Bartlett and Mendelson 2003; Koltchinskii and Panchenko 2000) and measures how well correlated the most-correlated hypothesis is to a random labeling of the efficient scores.

We conducted simulation studies to study the effect of using RW with sample of sizes 50 and 100 and genetic maps of 1, 10, and 20 chromosomes. Each chromosome has a length of 100 cM and 100 equally spaced markers. We use the function sim.cross in R/qtl (Broman et al. 2003) to generate the genotype data. Under H₀, we generate the quantitative traits from N(0, 1) while under H₁, we generate from N(μ, 1), μ ∈ [0.2, 1.0], representing an additive effect at 35 cM on chromosome one.

Figure 1 presents the thresholds for the single-marker analysis at the GWS level of 0.05 and 0.01 and compares them to both the empirical thresholds and that of Zou et al. (2004) based on 10,000 replicates and B = 10, 000 in the algorithm. When n = p, the thresholds based on both methods match the empirical thresholds, under both H₀ and H₁ (data not presented under H₁). When n is small and <p, the thresholds using RW still match the empirical thresholds, whereas the thresholds from Zou et al. (2004) are overestimated. The standard errors of the thresholds were also calculated using the function quantileSE in the R package broman (detailed results are presented in supporting information, File S1). Figure 2 presents the sizes and powers of the two resampling approaches. The proposed approach has type I error rates close to the nominal level under all situations and is substantially more powerful than Zou et al. (2004) under the large p small n scenarios.

Figure 1.

Thresholds at the targeted GWS levels of α. The solid, dashed, and dotted curves correspond to the average thresholds based on the proposed method and the method of Zou et al. (2004) from 10,000 simulated data sets and the empirical thresholds based on 10,000 simulated data sets under the null hypothesis, respectively.

Figure 2.

Sizes/powers(%) at nominal GWS level of α. The black and blue curves correspond to sizes/powers from the proposed method and the method of Zou et al. (2004), respectively. The solid, dashed, and dotted curves correspond to the sizes/powers under the scenarios when p = 100, 1000, and 2000, respectively.

In summary, we proposed a new resampling approach for assessing GWS in QTL mapping. This new approach retains all the attractive features of the resampling approach of Zou et al. (2004) and outperforms it under the large p small n situation. Additional simulation studies with n = 500 and P = 2000 showed that the two methods yielded similar results (detailed results are presented in File S1).