Sample size and power analysis for sparse signal recovery in genome-wide association studies

Abstract

Genome-wide association studies have successfully identified hundreds of novel genetic variants associated with many complex human diseases. However, there is a lack of rigorous work on evaluating the statistical power for identifying these variants. In this paper, we consider sparse signal identification in genome-wide association studies and present two analytical frameworks for detailed analysis of the statistical power for detecting and identifying the disease-associated variants. We present an explicit sample size formula for achieving a given false non-discovery rate while controlling the false discovery rate based on an optimal procedure. Sparse genetic variant recovery is also considered and a boundary condition is established in terms of sparsity and signal strength for almost exact recovery of both disease-associated variants and nondisease-associated variants. A data-adaptive procedure is proposed to achieve this bound. The analytical results are illustrated with a genome-wide association study of neuroblastoma.

1. Introduction

Genome-wide association studies have emerged as an important tool for discovering regions of the genome that harbour genetic variants conferring risk for complex diseases. Such studies often involve scanning hundreds of thousands of single-nucleotide polymorphism markers in the genome. Many novel genetic variants have been identified from recent genome-wide association studies, including variants for age-related macular degenerative diseases (Klein et al., 2005), breast cancer (Hunter et al., 2007) and neuroblastoma (Maris et al., 2008). The Welcome Trust Case-Control Consortium (2007) published a study of seven diseases using 14 000 cases and 3000 shared controls and has identified several markers associated with each of these complex diseases. The success of these studies has provided solid evidence that the genome-wide association studies represent a powerful approach to the identification of genes involved in common human diseases.

Due to the genetic complexity of many common diseases, it is generally assumed that multiple genetic variants are associated with disease risk. The key question is to identify true disease-associated markers from the multitude of nondisease-associated markers. The most common approach to this is to perform a single marker score test derived from the logistic regression model and to control for multiplicity of testing using stringent criteria such as the Bonferroni correction (McCarthy et al., 2008). Similarly, the most commonly used approach for sample size analysis or power analysis for such large-scale association studies is based on detecting markers of given odds ratios using a conservative Bonferroni correction. However, this correction is often too conservative for large-scale multiple testing problems, leading to reduced power in detecting disease-associated markers. Analytical and simulation studies by Sabatti et al. (2003) have shown that the false discovery rate procedure of Benjamini & Hochberg (1995) can effectively control the false discovery rate for the dependent tests encountered in case–control association studies and increase power.

Despite the success of genome-wide association studies, questions remain as to whether current sample sizes are large enough to detect most or all of the disease markers. This is related to power and sample size analysis and is closely related to sparse signal detection and discovery. However, power analysis has not been addressed fully for simultaneous testing of hundreds of thousands of null hypotheses. Efron (2007) presented an important alternative for power analysis for large-scale multiple testing problems in the framework of local false discovery rate. Gail et al. (2008) investigated the probability of detecting the disease-associated markers in case–control genome-wide association studies in the top T largest chi-square values from the trend tests of association. The detection probability is related to false non-discovery and T is related to false discovery, although these terms are not explicitly used in that paper.

The goal of the present paper is to provide an analytical study of the power and sample size issues in genome-wide association studies. We treat the vector of the score statistics across all the markers as a sequence of Gaussian random variables. Since we expect only a small number of markers to be associated with disease, the true mean of the vector of score statistics should be very sparse, although the degree of sparsity is unknown. Our goal is to recover those sparse and relevant markers. This is deeply connected with hypothesis testing in the context of multiple comparisons and false discovery rate control. Both false discovery rate control and sparse signal recovery have been areas of intensive research in recent years. Sun & Cai (2007) showed that the large-scale multiple testing problem has an equivalent weighted classification formulation, in the sense that the optimal solution to the multiple testing problem is also the optimal decision rule for the weighted classification problem. They further proposed an optimal false discovery rate controlling procedure that minimizes the false non-discovery rate. Donoho & Jin (2004) studied the detection of sparse heterogeneous mixtures using higher criticism, focusing on testing the global null hypothesis against the alternative where only a fraction of the data comes from a normal distribution with a common nonnull mean. It is particularly important that the detectable region is identified on the amplitude or sparsity plane so that higher criticism can completely separate the two hypotheses asymptotically.

In this paper, we present analytical results on sparse signal recovery in case–control genome-wide association studies. We investigate two frameworks for detailed analysis of the statistical power for detecting and identifying the disease-associated markers. In a similar setting as in Gail et al. (2008), we first present an explicit sample-size formula for achieving a given false non-discovery rate while controlling the false discovery rate based on the optimal false discovery rate controlling procedure of Sun & Cai (2007). This provides results on how odds ratios and marker allele frequencies affect the power of identifying the disease-associated markers. We also consider sparse marker recovery, establishing the theoretical boundary for almost exact recovery of both the disease-associated markers and nondisease-associated markers. Our results further extend the amplitude or sparsity boundary of Donoho & Jin (2004) for almost exact recovery of the signals. Finally, we construct a data adaptive procedure to achieve this bound.

2. Problem setup and score statistics

We consider a case-control genome-wide association study with m markers genotyped, where the minor allele frequency for marker i is p_i. Let G_i = 0, 1, 2 be the number of minor alleles at marker i (i = 1, …, m), where G_i ∼ Bin(2, p_i). Assume that the total sample size is n and n₁ = rn and n₂ = (1 − r)n are the sample sizes for cases and controls. Let Y = 1 for diseased and Y = 0 for nondiseased individuals. Suppose that in the source population, the probability of disease is given by

$$\text{logit}\{\text{pr}(Y|G_i,i=1,\dots ,m)\}=a+\sum _{i=1}^m{G}_i\hspace{0.17em}{b}_i,$$

where only a very small fraction of the b_is are nonzero. Gail et al. (2008) showed that for rare diseases or for more common diseases over a confined age range such as 10 years, for case-control population, if the markers are independent of each other, it follows that

$$\text{logit}\{\text{pr}(Y|G_i)\}=a_i+G_i\hspace{0.17em}{b}_i\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}(i=1,\dots ,m),$$ (1)

approximately. This implies that a logistic regression model can be fitted for each single marker i separately.

Based on the results of Prentice & Pyke (1979), the maximum likelihood estimation for a cohort study applied to case-control data with model (1) yields a fully efficient estimate of b_i and a consistent variance estimate. For a given case-control study, let j be the index for individuals in the samples, Y_j be the disease status of the j th individual and G_ij be the genotype score for the jth individual at the ith marker. The profile score statistic to test ${\text{H}}_i^0:b_i=0$ can be written as

$$U(b_i)=\sum _j{G}_{ij}(Y_j-\overline{Y}),$$

where $\overline{Y}={\sum }_{j=1}^n{Y}_j$. In the following, we let pr_i0, E_i0, var_i0 denote the probability, expectation and variance calculated under the null hypothesis ${\text{H}}_i^0:b_i=0$, and pr_i1, E_i1, var_i1 those calculated under the alternative hypothesis ${\text{H}}_i^1:b_i=1$. Under ${\text{H}}_i^0:b_i=0$, pr_i0(Y_j = y | G_ij = g) = K I (y = 1) + (1 − K) I (y = 0), where I (·) is the indicator function. We have E_i0{U(b_i) | Y} = 0, var_i0{U(b_i) | Y} = 2p_i (1 − p_i)r(1 − r)n.

Under the alternative, if b_i is known,

$$\begin{array}{l}{u}_{1gi}={\text{pr}}_{i1}(Y_j=1|G_{ij}=g)=\frac{\text{exp}(a_i+b_i\hspace{0.17em}g)}{1+\text{exp}(a_i+b_i\hspace{0.17em}g)},\\ u_{0gi}={\text{pr}}_{i1}(Y_j=0|G_{ij}=g)=\frac{1}{1+\text{exp}(a_i+b_i\hspace{0.17em}g)},\end{array}$$

where a_i can be determined by ${\sum }_{g=0}^2{u}_{1gi}\hspace{0.17em}\text{pr}(G_{ij}=g)=D$, with D being the disease prevalence and $\text{pr}(G_{ij}=g)=p_i^2\hspace{0.17em}\text{I}(g=2)+2\hspace{0.17em}{p}_i(1-p_i)\hspace{0.17em}\text{I}\hspace{0.17em}(g=1)+{(1-p_i)}^2\hspace{0.17em}\text{I}\hspace{0.17em}(g=0)$. Then

$$w_{ygi}={\text{pr}}_{i1}\hspace{0.17em}(G_{ij}=g|Y_j=y)=\frac{u_{y,g,i}\hspace{0.17em}\text{pr}(G_{ij}=g)}{{\sum }_{g^{\prime }=0}^2\hspace{0.17em}{u}_{y{g}^{\prime }i}\hspace{0.17em}\text{pr}(G_{ij}=g^{\prime })}$$

and

$$\begin{array}{l}\hspace{0.17em}{E}_{i1}\{U(b_i)|Y\}=r(1-r)n\{E_{i1}(G_{ij}|Y_j=1)-E_{i1}(G_{ij}|Y_j=0)\},\\ {\text{var}}_{i1}\{U(b_i)|Y\}=r(1-r)n\{(1-r)\hspace{0.17em}{\text{var}}_{i1}(G_{ij}|Y_j=1)+r{\text{var}}_{i1}(G_{ij}|Y_j=0)\},\end{array}$$

where $E_{i1}(G_{ij}|Y_j=y)={\sum }_{g=0}^2\hspace{0.17em}g{w}_{ygi}$, $E_{i1}(G_{ij}^2|Y_j=y)={\sum }_{g=0}^2\hspace{0.17em}{g}^2\hspace{0.17em}{w}_{ygi}$, and

$${\text{var}}_{i1}(G_{ij}|Y_j=y)=E_{i1}(G_{ij}^2|Y_j=y)-{\{E_{i1}(G_{ij}|Y_j=y)\}}^2.$$

Let X_i = U(b_i)/var_i0{U(b_i)|Y}^1/2 be the score statistic for testing the association between the marker i and the disease. If b_i is known, then under ${\text{H}}_i^0$, X_i ∼ N(0, 1) and under ${\text{H}}_i^1$, X_i ∼ N(μ_n,i, σ_i) asymptotically, where

$${\mu }_{n,i}=n^{1/2}\hspace{0.17em}{\mu }_i\equiv n^{1/2}\hspace{0.17em}{\left\{\frac{r(1-r)}{2\hspace{0.17em}{p}_i(1-p_i)}\right\}}^{1/2}\hspace{0.17em}\{E_{i1}(G_{ij}|Y_j=1)-E_{i1}(G_{ij}|Y_j=0)\},$$ (2)

$${\sigma }_i={\left\{\frac{(1-r)\hspace{0.17em}{\text{var}}_{i1}(G_{ij}|Y_j=1)+r\hspace{0.17em}{\text{var}}_{i1}(G_{ij}|Y_j=0)}{2\hspace{0.17em}{p}_i(1-p_i)}\right\}}^{1/2}.$$ (3)

Given b_i, p_i, r and D, μ_n,i and σ_i can be determined. The alternative mean μ_n,i increases with the sample size n at the order √n. Intuitively, as the sample size increases, it is easier to distinguish the signals and zeros. So, the problem of power analysis and sample size analysis can be formulated as the sparse normal mean problem: we have score statistics X_i, (i = 1, …, m) and only a very small proportion ∊_m of them have nonzero means, and the goal is to identify those markers whose score statistics have nonzero means.

3. Marker recovery

3.1. Effect size and false discovery rate control

As in Genovese & Wasserman (2002) and Sun & Cai (2007), we use the marginal false discovery rate, defined as mfdr = E(N₀₁)/E(R), and marginal false non-discovery rate, defined as mfnr = E(N₁₀)/E(S), as our criteria for multiple testing, where R is the number of rejections, N₀₁ is the number of nulls among these rejections, S is the number of non-rejections and N₁₀ is the number of nonnulls among these nonrejections. Genovese & Wasserman (2002) and Sun & Cai (2007) showed that under weak conditions, the marginal false discovery rate (mfdr) and the false discovery rate (fdr), the marginal false non-discovery rate (mfnr) and the false non-discovery rate (fnr) are asymptotically the same in the sense that mfdr = fdr + O(m^−1/2) and mfnr = fnr + O(m^−1/2). In the following, we use mfdr and mfnr for our analytical analysis, but we use the notation, fdr and fnr.

Consider the sequence of score statistics X_i (i = 1, …, m), as defined in the previous section, where under ${\text{H}}_i^0$, X_i ∼ N(0, 1) and under ${\text{H}}_i^1$, X_i ∼ N(μ_n,i, ${\sigma }_i^2$), μ_n,i ≠ 0, σ_i ⩾ 1. Usually, μ_n,i and σ_i are different across different disease-associated markers due to different minor allele frequencies and different effect sizes as measured by the odds ratios; see (2) and (3). However, since most of the markers in genome-wide association studies are not rare and the observed odds ratios range from 1.2 to 1.5, we can reasonably assume that they are on a comparable scale. In addition, for the purpose of sample size calculation, one should always consider the worst case scenarios for all the markers, which leads to a conservative estimate of the required sample sizes. To simplify the notation and analysis and to obtain closed-form analytical results, we thus assume that all the markers considered have the same minor allele frequency and all the relevant markers have the same effect size. Specifically, we assume that under ${\text{H}}_i^1$, X_i ∼ N(μ, σ²), σ ⩾ 1. Suppose that the proportion of nonnull effects is ∊_m. Defining a sequence of binary latent variables, θ = (θ₁, … ,θ_m), the model under consideration can be stated as follows:

$$\begin{array}{l}{\theta }_1,\dots ,{\theta }_m\text{ are independently and identically distributed as Bernoulli}({∊}_m),\\ \text{if }{\theta }_i=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{X}_i\sim \text{N}(0,1);\hspace{0.17em}{\theta }_i=1,\hspace{0.17em}\hspace{0.17em}{X}_i\sim \text{N}(\mu ,\hspace{0.17em}\hspace{0.17em}{\sigma }^2).\end{array}$$ (4)

Assume that σ is known. Our goal is to find the minimum μ that allows us to identify θ with the false discovery rate asymptotically controlled at α₁ and the false non-discovery rate asymptotically controlled at α₂. We particularly consider the optimal false discovery rate controlling procedure of Sun & Cai (2007), which simultaneously controls the false discovery rate at α₁ and minimizes the false non-discovery rate asymptotically. Different from the p-value-based false discovery rate procedures, this approach considers the distribution of the test statistics and involves the following.

Algorithm 1. Calculating the optimal false discovery rate.

• Step 1. Given the observation X = (X₁, …, X_m), estimate the nonnull proportion ∊̂_m using the method in Cai & Jin (2010). This estimator is based on Fourier transformation and the empirical characteristic function, which can be written as ${\widehat{∊}}_m=1-m^{-(1-\eta )}{{\sum }_{i=1}^m\text{cos}\{(2\eta \hspace{0.17em}\text{log}\hspace{0.17em}m)}^{1/2}\hspace{0.17em}{X}_i\}$, where η ∈ (0, 1/2) is a tuning parameter, which needs to be small for the sparse case considered in this paper. Based on our simulations, η = 10⁻⁴ works well.

• Step 2. Use a kernel estimate f̂ to estimate the mixture density f of the X_i,

  $$\widehat{f}(x)=m^{-1+\rho }\sum _{j=1}^{m}K\{m^{\rho }(x-X_j)\},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\rho \in (0,1/2),$$ (5)

  where K is a kernel function and ρ determines the bandwidth. We use ρ = 1.34 m^−1/5 as recommended by Silverman (1986).

• Step 3. Compute T̂_i = (1 − ∊̂_m) φ (X_i)/f̂(X_i), where φ(.) is the standard normal density function.

• Step 4. Let

  $$k=\text{max}\hspace{0.17em}\left\{i:i^{-1}\sum _{j=1}^i{\widehat{T}}_{(j)}⩽{\alpha }_1\right\},$$

  then reject all H_(i) (i = 1, …, k). This adaptive step-up procedure considers the average posterior probability of being null and is adaptive to both the global feature ∊_m and local feature of φ(X_i)/f(X_i).

The procedure has a close connection with a weighted classification problem. Consider the loss function

$$L_{\lambda }\hspace{0.17em}(\theta ,\hspace{0.17em}\delta )=\sum _{i=1}^m\{\lambda \hspace{0.17em}\text{I}\hspace{0.17em}({\theta }_i=0){\delta }_i+\text{I}\hspace{0.17em}({\theta }_i=1)\hspace{0.17em}(1-{\delta }_i)\}.$$ (6)

The Bayes rule under the loss function (6) is

$${\theta }_i\hspace{0.17em}(\lambda )=\text{I}\hspace{0.17em}\left\{\mathrm{\Lambda }(X_i)=\frac{(1-{∊}_m)\hspace{0.17em}{f}^0\hspace{0.17em}(X_i)}{{∊}_m\hspace{0.17em}{f}^1\hspace{0.17em}(X_i)}<\frac{1}{\lambda }\right\},$$ (7)

where f⁰(x) = φ (x; 0, 1) and f¹(x) = φ (x; μ, σ²). Sun & Cai (2007) show that the threshold λ is a decreasing function of false discovery rate, and for a given level α₁, one can determine a unique thresholding λ to ensure the false discovery rate level to be controlled under α₁. The following theorem gives the minimal signal μ required in order to obtain the pre-specified false discovery rate and false non-discovery rate using this optimal procedure.

Theorem 1. Consider the model in (4) and the optimal oracle procedure of Sun & Cai (2007). In order to control the false discovery rate under α₁ and the false non-discovery rate under α₂, the minimum μ has to be no less than (σ² − 1)d̂, where (ĉ, d̂) are the roots of the equations

$$\{\begin{array}{l}\mathrm{\Phi }(c-d\sigma )-\mathrm{\Phi }(-c-d\sigma )=(1-c_1)/(c_2-c_1),\hfill \\ \mathrm{\Phi }(-c\sigma -d)+\mathrm{\Phi }(-c\sigma +d)=c_1(c_2-1)/(c_2-c_1),\hfill \end{array}$$ (8)

where Φ is the cumulative density function of the standard normal distribution, and

$$c_1=\frac{{\alpha }_1{∊}_m}{(1-{\alpha }_1)\hspace{0.17em}(1-{∊}_m)},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{c}_2=\frac{(1-{\alpha }_2){∊}_m}{{\alpha }_2(1-{∊}_m)}.$$

As discussed in § 2, in genome-wide association studies, μ = √nμ₁, where μ₁ as defined in (2) can be determined by the effect of the marker, log odds ratio b, the minor allele frequency p and the case ratio or control ratio r. The following corollary provides the minimum sample size needed in case-control genome-wide association studies in order to control false discovery rate and false non-discovery rate.

Corollary 1. Consider a case-control genome-wide association study, with m markers, a total sample size of n, with n₁ = nr cases and n₂ = (1 − r)n controls. Assume that all the markers have the same minor allele frequency of p_i = p and all the relevant markers have the same log odds ratio b_i = b. In order to control false discovery rate and false non-discovery rate under α₁ and α₂ asymptotically, the minimum total sample size must be at least n = {d̂(σ² − 1)/μ₁}², where d̂ is the root of (8), and μ₁ and σ₁ are defined in (2) and (3).

Corollary 1 gives the minimum sample size required in order to identify the markers with minor allele frequency p and effective odds ratio of b with false discovery rate less than α₁ and false non-discovery rate less than α₂ using the optimal false discovery rate controlling procedure of Sun & Cai (2007). Since this procedure asymptotically controls false discovery rate and minimizes false non-discovery rate, the sample size obtained here is a lower bound. This implies that asymptotically, one cannot control both false discovery rate and false non-discovery rate using a smaller sample size with any other procedure. In practice, α₂ should be set to less than ∊_m, which can be achieved by not rejecting any nulls.

3.2. Illustration

We present the minimum sample sizes needed for different levels of false discovery rate and false non-discovery rate in Fig. 1 for ∊_m = 2 × 10⁻⁴, which corresponds to assuming 100 disease-associated markers. We assume a disease prevalence of D = 0.10. As expected, the sample size needed increases as the false discovery rate level or the target false non-discovery rate level decreases. It also decreases with higher signal strength. The false non-discovery rate levels presented in the plot are very small since it cannot exceed the proportion of the relevant markers ∊_m. When m is extremely large, ∊_m is usually very small. In Fig. 1, ∊_m = 2 × 10⁻⁴ and false non-discovery rate we choose should be less than ∊_m, since fnr = ∊_m can be achieved by not rejecting any of the markers. Among the 100 relevant markers, fnr = 10⁻⁴, 2 × 10⁻⁵ and 10⁻⁶ corresponds to an expected number of nondiscovered relevant markers of fewer than 50, 10 and 0.5, respectively. As a comparison, with the same sample size for fdr = 0.05 and fnr = 10⁻⁴, 2 × 10⁻⁵ and 10⁻⁶, the power of the one-sided score test using the Bonferonni correction to control the genome-wide error rate at 0.05 is 0.22, 0.64 and 0.95, respectively. We observe that the sample sizes needed strongly depend on the false non-discovery rate and the effect size of the markers, but are less dependent on false discovery rate. Similar trends were observed for ∊_m = 2 × 10⁻⁵, which corresponds to assuming 10 disease-associated markers.

Fig. 1.

Minimum sample size needed for a given false discovery rate and false non-discovery rate for a genome-wide association study with m = 500 000 markers and the relevant markers proportion ∊_m = 2 × 10⁻⁴. Assume that the case proportion r = 0.40 and the minor allele frequency p = 0.40. (a) fdr = 0.05 and different values of odds ratio; (b) fdr = 0.20 and different values of odds ratio; (c) odds ratio of 1.20 and different values of false discovery rate; (d) odds ratio of 1.50 and different values of false discovery rate. For each plot, the solid, dash and dot lines correspond to fnr = 10⁻⁴, 2 × 10⁻⁵ and 10⁻⁶, respectively.

We performed simulation studies to evaluate the sample size formula presented in Corollary 1. We assume that m = 500 000 markers are tested with 100 being associated with disease, which corresponds to ∊_m = 2 × 10⁻⁴. We first considered the setting that all the 100 relevant markers have the same odds ratio on disease risk with a minor allele frequency of 0.40. For a pre-specified fdr = 0.05 and fnr = 10⁻⁴, we determined the sample size based on Corollary 1. We then applied the false discovery rate controlling procedure of Sun & Cai (2007) to the simulated data and examined the values of the observed false discovery rates and false non-discovery rates. Table 1 shows the empirical false discovery rates and false non-discovery rates over 100 simulations for different values of the odds ratios. The empirical false discovery rates and false non-discovery rates are very close to the pre-specified values, indicating that the sample size formula can indeed result in the specified false discovery rate and false non-discovery rate.

Table 1.

Empirical false discovery rate and false non-discovery rate (10⁻⁴ unit) and their standard errors for the optimal false discovery rate procedure based on 100 simulations where the sample sizes are determined by Corollary 1 for the independent sequences with the same alternative odds ratio for fdr = 0.05 and fnr = 10⁻⁴.

Independent sequence with the same alternative odds ratios			Short-ranged dependent sequence with different alternative odds ratios
Odds ratio	Empirical fdr	Empirical fnr	Empirical fdr	Empirical fnr
1.20	0.050 (0.028)	1.04 (0.10)	0.051 (0.030)	1.04 (0.10)
1.23	0.048 (0.028)	1.03 (0.12)	0.051 (0.032)	1.04 (0.11)
1.26	0.045 (0.030)	1.04 (0.11)	0.055 (0.033)	1.04 (0.11)
1.29	0.046 (0.030)	1.15 (0.13)	0.053 (0.031)	1.03 (0.11)
1.33	0.053 (0.033)	1.04 (0.11)	0.054 (0.032)	1.05 (0.10)
1.36	0.047 (0.029)	1.01 (0.13)	0.049 (0.029)	1.00 (0.12)
1.39	0.051 (0.030)	1.02 (0.11)	0.051 (0.033)	1.01 (0.11)
1.43	0.046 (0.033)	1.05 (0.12)	0.044 (0.029)	1.04 (0.11)
1.46	0.046 (0.027)	1.04 (0.10)	0.046 (0.025)	1.02 (0.11)
1.50	0.051 (0.027)	1.03 (0.13)	0.053 (0.033)	1.03 (0.12)

fdr, false discovery rate; fnr, false non-discovery rate.

Since our power and sample size calculation is derived under the assumption of common effect sizes for all associated markers and independent test statistics, we also evaluated our results under the assumption of short-range dependency and unequal effect sizes. To approximately mimic the dependency structure among markers due to linkage disequilibrium, we assumed a block-diagonal covariance structure for the score statistics with exchangeable correlation of 0.20 for block sizes of 50. We simulated marker effects with a mean odds ratio ranging from 1.20 to 1.50. For a given mean odds ratio, we obtained the sample size and simulated the corresponding marker effects by generating μ_n,i from N (4.40, 1.00). We then applied the false discovery rate controlling procedure and calculated the empirical false discovery rates and false non-discovery rates for different mean odds ratios. The results are presented in Table 1, indicating that the false discovery rate controlling procedure can indeed control false discovery rates and obtain the pre-specified false non-discovery rates.

4. A procedure for sparse marker recovery

4.1. Oracle sparse marker discovery

Besides controlling false discovery rate and false non-discovery rate, an important alternative in practice is to control the numbers of false discoveries and false non-discoveries. One limitation with the use of false discovery rate and false non-discovery rate in power calculations is that they do not provide a very clear idea about roughly how many markers are falsely identified and how many are missed until the analysis is conducted. In genome-wide association studies, the goal is to make the number of misidentified markers, including both the false discoveries and false non-discoveries, converge to zero when the number of the markers m and the sample sizes are sufficiently large. In this section, we investigate the conditions on the parameters μ = μ_m and ∊_m in (4) that can lead to almost exact recovery of the null and nonnull markers. This in turn provides a useful formula for the minimum sample size required for an almost exact recovery of disease-associated markers.

Consider (4) and a decision rule δ_i (i = 1, …, m). We define false discoveries as the expected number of null scores that are misclassified to the alternative, and false non-discoveries as the expected number of nonnull scores that are misclassified to null,

$$\text{FD}=E\hspace{0.17em}\left(\sum _{i=1}^m\text{I}\hspace{0.17em}({\theta }_i=0){\delta }_i\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{FN}=E\hspace{0.17em}\left(\sum _{i=1}^m\text{I}\hspace{0.17em}({\theta }_i=1)\hspace{0.17em}(1-{\delta }_i)\right).$$

Define the loss function as

$$L(\theta ,\hspace{0.17em}\delta )=\sum _i\text{I}\hspace{0.17em}({\theta }_i=0){\delta }_i+\text{I}\hspace{0.17em}({\theta }_i=1)\hspace{0.17em}(1-{\delta }_i)=\sum _i\text{I}\hspace{0.17em}({\theta }_i\ne {\delta }_i),$$

which is simply the Hamming distance between the true θ and its estimate δ. Note that E{L(θ, δ)} = fd + fn.

Under (4), the Bayes oracle rule (7) with λ = 1, defined as

$${\delta }_{\text{OR},i}=\text{I}\hspace{0.17em}\left\{{\mathrm{\Lambda }}_{\text{OR}}\hspace{0.17em}(X_i)=\frac{(1-{∊}_m)\hspace{0.17em}{f}^0\hspace{0.17em}(X_i)}{{∊}_m\hspace{0.17em}{f}^1\hspace{0.17em}(X_i)}<1\right\},$$ (9)

minimizes the risk E{L(θ, δ)}. We consider the condition on μ_m and ∊_m such that this risk converges to zero as m goes to infinity, where the nonnull markers are assumed to be sparse. We calibrate ∊_m and μ_m as follows. Define

$${∊}_m=m^{-\beta },\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\beta \in (1/2,\hspace{0.17em}1),$$ (10)

which measures the sparsity of the disease-associated markers, and

$${\mu }_m={(2\tau \hspace{0.17em}\text{log}\hspace{0.17em}m)}^{1/2},$$ (11)

which measures the strength of the disease-associated markers. Here we assume that the disease markers are sparse. We first study the relationship between τ and β so that both false discoveries and false non-discoveries converge to zero as m goes to infinity.

Theorem 2. Consider (4) and reparameterize ∊_m and μ = μ_m in terms of β and τ as in (10) and (11). For any γ ⩾ 0, under the condition

$$\tau ⩾{\{{(1+\gamma )}^{1/2}+\sigma {(1+\gamma -\beta )}^{1/2}\}}^2,$$ (12)

the Bayes rule (9) guarantees that both false discoveries and false non-discoveries converge to zero with convergence rate 1/{m^γ (log m)^1/2}.

Theorem 2 shows the relationship between the strength of the signal and the convergence rate of the expected false discoveries and false non-discoveries. As expected, a stronger nonzero signal leads to faster convergence of the expected number of false identifications to zero and easier separation of the alternative from the null.

In (12), when γ = 0, the convergence rate of false discoveries and false non-discoveries is the slowest, 1/(log m)^1/2. Letting σ → 1 yields the phase diagram in Fig. 2, which is an extension to the phase diagram in Donoho & Jin (2004). Three lines separate the τ − β plane into four regions. The detection boundary

$$\tau =\{\begin{array}{ll}\beta -1/2,\hfill & (1/2<\beta ⩽3/4),\hfill \\ {\{1-{(1-\beta )}^{1/2}\}}^2,\hfill & (3/4<\beta <1),\hfill \end{array}$$

separates the detectable region and the undetectable region, when μ exceeds the detection boundary, the null hypothesis H₀ : X_i ∼ N(0, 1), (i = 1, …, m) and the alternative ${\text{H}}_1^{(m)}$: X_i ∼ (1 − ∊_m)N(0, 1) + ∊_m N (μ, 1), (i = 1, …, m) separate asymptotically. Above the estimation boundary τ = β, it is possible not only to detect the presence of nonzero means, but also to estimate these means. These two regions were identified by Donoho & Jin (2004). Based on Theorem 2, we can obtain the almost exact recovery boundary

$$\tau ={\{1+{(1-\beta )}^{1/2}\}}^2,$$

which provides the region that we can recover the whole sequence of θ with a very high probability converging to one when m → ∞, since

$$\text{pr}\hspace{0.17em}\left\{\sum _i\text{I}\hspace{0.17em}({\theta }_i\ne {\delta }_i)\right\}⩽E\left({\Vert \theta -\delta \Vert }_2^2\right)\asymp 1/{(\text{log}\hspace{0.17em}m)}^{1/2},$$

where ≍ represents asymptotic equivalence. In other words, in this region, we can almost fully classify the θ_i into the nulls and the non-nulls.

Fig. 2.

Four regions of the τ − β plane, where ∊_m = m^−β represents the percentage of the true signals and μ_m = (2τ log m)^1/2 measures the strength of the signals: almost exact recovery (dots); estimable (dashes); detectable (circles); undetectable (triangles).

In large-scale genetic association studies, μ = √nμ₁. Based on Theorem 2, we have the following corollary.

Corollary 2. Consider the same model as in Corollary 1. If the sample size

$$n⩾{\left[\frac{{\{2(1+\gamma \hspace{0.17em})\hspace{0.17em}\text{log}\hspace{0.17em}m\}}^{1/2}+\sigma {\{2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m+2\hspace{0.17em}\text{log}\hspace{0.17em}{∊}_m\}}^{1/2}}{{\mu }_1}\right]}^2,$$

then both false discoveries and false non-discoveries of the oracle rule converge to zero at rate 1/{m^γ (log m)^1/2}.

4.2. An adaptive sparse recovery procedure

Theorem 2 shows that the convergence rate 1/{m^γ (log m)^1/2} can be achieved using the oracle rule (9). However, this rule involves unknown parameters, ∊_m, f¹(x) and f⁰(x), which need to be estimated. Estimation errors can therefore affect the convergence rate. We instead propose an adaptive estimation procedure for model (4) that can achieve the same convergence rate for false discoveries and false non-discoveries under a slightly stronger condition than that in Theorem 2.

The Bayes oracle rule given in (9) can be rewritten as

$${\delta }_{\text{OR},i}=\text{I}\{(1-{∊}_m)\hspace{0.17em}{f}^0\hspace{0.17em}(X_i)<{∊}_m\hspace{0.17em}{f}^1\hspace{0.17em}(X_i)\}=\text{I}\hspace{0.17em}\left\{\frac{f(X_i)}{f^0(X_i)}>2(1-{∊}_m)\right\},$$ (13)

where f (x) = (1 − ∊_m) f⁰(x) + ∊_m f¹(x) is the mixture density of the X_i. The density f is typically unknown, but is easily estimable. Let f̂(x) be a kernel density estimate based on the observations X₁, X₂, …, X_m, defined by (5). For i = 1, …, m, if |X_i| > {2(1 + γ) log m}^1/2, then reject the null ${\text{H}}_i^0:{\theta }_i=0$; otherwise, calculate Ŝ(X_i) = f̂(X_i)/φ(X_i) and reject the null if and only if Ŝ(X_i) > 2. The threshold value 2 is based on (13) since the proportion ∊_m is vanishingly small in our setting.

The next theorem shows that this adaptive procedure achieves the same convergence rate as the optimal Bayesian rule under a slightly stronger condition.

Theorem 3. Consider (4) and reparameterize ∊_m = n^−β and μ = μ_m = (2τ log m)^1/2 as in (10) and (11). Suppose that the Gaussian kernel density estimate f̂, defined in (5), with ρ = 1.34 × m^−1/5 is used in the adaptive procedure. For any γ ⩾ 0, if

$$\tau >{\{{(1+\gamma )}^{1/2}+\sigma {(1+\gamma -\beta )}^{1/2}\}}^2,$$

then the adaptive procedure guarantees that both false discoveries and false non-discoveries converge to zero with convergence rate 1/{m^γ (log m)^1/2}.

4.3. Illustration of marker recovery

The sample sizes needed to obtain an almost exact recovery of the true disease-associated markers are presented in Fig. 3 for ∊_m = 2 × 10⁻⁴ and ∊_m = 2 × 10⁻⁵, which correspond to 100 and 10 disease-associated markers. We assume a disease prevalence of D = 0.10. We observe that a larger γ implies a faster rate of convergence, and therefore also larger sample size required to the achieve the rate. Generally speaking, the sample sizes in Fig. 3 are much larger than those in Fig. 1, simply because almost exact recovery of the disease-associated markers is a much stronger goal than controlling the false discovery rate and false non-discovery rate at given levels. In addition, for markers with the same effect size, it is easier to achieve full recovery of the relevant markers for the sparser cases, see of Fig. 3(b).

Fig. 3.

Minimum sample size required for almost exact recovery of the disease-associated markers for a case-control genome-wide association study with the number of markers m = 500 000, case proportion r = 0.40, minor allele frequency p = 0.40 for all the markers. (a) The relevant markers proportion is ∊_m = 2 × 10⁻⁴; (b) The relevant markers proportion is ∊_m = 2 × 10⁻⁵. For each plot, the solid, dash and dot lines correspond to γ values of 0.00, 0.20 and 0.40.

We performed simulation studies to investigate how sample size affects the convergence rate of false discoveries and false non-discoveries. We considered the same setting as above, where the number of markers m = 500 000, case proportion r = 0.40, minor allele frequency p = 0.40 for all the markers and the relevant markers proportion is ∊_m = 2 × 10⁻⁴. For a given odds ratio and a given γ parameter, we obtained the minimum sample size n for almost exact recovery and evaluated the convergence rate of the oracle and the adaptive procedures under different sample sizes. Table 2 shows the average number of fd+fn over 100 simulations. We observe a clear decrease of fd+fn as we increase the sample sizes for both the oracle and the adaptive procedures. For the adaptive procedure, when the sample size is larger than the minimum sample size, the decrease in fd+fn is small since the minimum sample size has already achieved almost exact recovery.

Table 2.

The performances of the oracle and adaptive false discovery and false non-discovery controlling procedures for varying sample sizes, where n is the theoretical sample size. For a given γ, the average number of fd+fn and its standard error over 100 simulations are shown for each sample size and odds ratio combination.

		γ = 0			γ = 0.2
or		0.8n	n	1.2n	0.8n	n	1.2n
	o	0.77 (0.89)	0.14 (0.35)	0.02 (0.14)	0.08 (0.27)	0.00 (0.00)	0.00 (0.00)
1.20	a	3.08 (3.11)	3.10 (3.23)	2.51 (3.04)	2.57 (3.02)	3.12 (3.24)	2.59 (3.12)
	o	0.57 (0.79)	0.13 (0.37)	0.01 (0.10)	0.03 (0.17)	0.00 (0.00)	0.00 (0.00)
1.30	a	3.93 (4.22)	2.61 (2.97)	2.92 (3.36)	3.41 (3.99)	2.66 (3.00)	2.92 (3.34)
	o	0.50 (0.72)	0.1 (0.30)	0.04 (0.20)	0.02 (0.14)	0.00 (0.00)	0.01 (0.10)
1.40	a	3.31 (3.30)	2.57 (3.11)	2.89 (3.45)	2.97 (3.39)	2.55 (3.11)	2.84 (3.42)

or, odds ratio; o, oracle; a, adaptive.

5. Genome-wide association study of neuroblastoma

Neuroblastoma is a paediatric cancer of the developing sympathetic nervous system and is the most common form of solid tumour outside the central nervous system. It is a complex disease, with rare familial forms occurring due to mutations in paired-like homeobox 2b or anaplastic lymphoma kinase genes (Mosse et al., 2008), and several common variations being enriched in sporadic neuroblastoma cases (Maris et al., 2008). The latter genetic associations were discovered in a genome-wide association study of sporadic cases, compared with children without cancer, conducted at The Children’s Hospital of Philadelphia. After initial quality controls on samples and marker genotypes, our discoveries dataset contained 1627 neuroblastoma case subjects of European ancestry, each of which contained 479 804 markers. To correct the potential effects of population structure, 2575 matching control subjects of European ancestry were selected based on their low identity-by-state estimates with case subjects. Analysis of this dataset has led to identification of several markers associated with neuroblastoma, including three markers on 6p22 containing the predicted genes FLJ22536 and FLJ44180 with allelic odds ratio of about 1.4 (Maris et al., 2008), and markers in BARD1 genes on chromosome 2 (Capasso et al., 2009) with odds ratio of about 1.68 in high-risk cases. The question that remains to be answered is whether the current sample size is large enough to identify all the neuroblastoma-associated markers.

We estimated ∊_m = 2 × 10⁻⁴ using the procedure of Cai and Jin (2010) with tuning parameter η = 10⁻⁴. A small value of η was chosen to reflect that we expect a very small number of disease-associated markers. With this estimate, we assume that there are 96 markers associated with neuroblastoma. For the given sample size and number of markers to be tested, Fig. 4(a) shows the minimum detectable odds ratios for markers with different minor allele frequencies for various false discovery rate and false non-discovery rate levels. The false non-discovery rates of 10⁻⁴, 5 × 10⁻⁵ and 10⁻⁵ correspond to an expected number of nondiscovered markers of fewer than 48, 24 and 4.8, respectively. This plot indicates that with the current sample size, it is possible to recover about half of the disease-associated markers with an odds ratio around 1.4 and a minor allele frequency of around 20%.

Fig. 4.

Power analysis for neuroblastoma genome-wide association study. Plots (a) and (b) assume that ∊ = 2 × 10⁻⁴, which corresponds to 98 relevant markers and plots (c) and (d) assume that ∊ = 10⁻⁴, which corresponds to 49 relevant markers. (a) and (c): detectable odds ratios for markers with different minor allele frequencies. For plot (a), the solid, dot, dash and dot-dash lines correspond to (fdr = 5%, fnr = 10⁻⁴), (fdr = 1%, fnr = 10⁻⁴), (fdr = 5%, fnr = 10⁻⁵), and (fdr = 1% and fnr = 10⁻⁵), respectively. For plot (c), the solid, dot, dash and dot-dash lines correspond to (fdr = 5%, fnr = 5 × 10⁻⁵), (fdr = 1%, fnr = 5 × 10⁻⁵), (fdr = 5%, fnr = 5 × 10⁻⁶), and (fdr = 1% and fnr = 5 × 10⁻⁶), respectively. (b) and (d): detectable odds ratios for markers with different minor allele frequencies for almost exact recovery with different convergence rates determined by γ = 0.10 (solid) and 0.40 (dash).

Figure 4(b) shows the minimum detectable odds ratios for markers with different minor allele frequencies for almost exact recovery at different rates of convergence as specified by the parameter γ, indicating that the current sample size is not large enough to obtain almost exact recovery of all the disease-associated markers with odds ratio around 1.4.

Figure 4(c) and (d) shows similar plots if we assume that ∊_m = 10⁻⁴, which implies that there are 48 associated markers. This plot indicates that with the current sample size, it is possible to recover most of the disease-associated markers with odds ratio around 1.4 and minor allele frequency around 20% for false discovery rates of 1 or 5%. However, the sample size is still not large enough to obtain almost exact recovery of all disease-associated markers with odds ratio around 1.4.

6. Discussion

Our work complements and differs from that of Zaykin & Zhivotovsky (2005) and Gail et al. (2008), who based their analyses on the individual marker P-values, aiming to evaluate how likely the markers selected based on the P-value ranking are disease-associated. Although both papers considered the issues of false positives and false negatives, neither links these to false discovery and false non-discovery rates in a formal way. Our analysis is mainly based on the distribution of the score statistics derived from the logistic regressions for case-control data and is set in the framework of multiple testing and sparse signal recovery. In the setting of thousands of hypotheses, it is natural to consider the power and sample size in terms of false discovery and false non-discovery rates, which correspond to Type I and Type II errors in single hypothesis testing. In addition, we have derived an explicit sample size formula under the assumption of fixed allele frequency and fixed common marker effect.

In presenting the results for genome-wide association studies, we made several simplifications. First, we assumed that the score statistics were independent across all the m markers. This obviously does not hold due to linkage disequilibrium among the markers. However, we expect such dependency to be short-range and our recent unpublished results show that the optimal false discovery rate controlling procedure of Sun & Cai (2007) is still valid and remains optimal. The results and the sample size formulae remain valid for such short-range dependent score statistics. Zaykin & Zhivotovsky (2005) and Gail et al. (2008) showed that correlations of P-values within linkage disequilibrium blocks of markers or among such blocks have little effect on selection or detection probabilities because such correlations do not extend beyond a small portion of the genome. It is likely, therefore, that our results were also little affected by such correlations. This was further verified in our simulation studies shown in Table 1. Second, to simplify the presentation and the derivation and to obtain a clean sample size formula as we presented in Corollaries 1 and 2, we assumed that all the markers had the same minor allele frequency and the effect sizes of the relevant markers were the same. One can only employ simulation for power analysis if the minor allele frequencies and the marker effects are all different. In fact, the analytical results that were used to validate their simulations in Gail et al. (2008) were also derived under the assumption of fixed allele frequency and fixed common effect. As in most of the power calculations, in practice, one should consider different scenarios in terms of minor allele frequencies and odds ratios.

Appendix

We present the proofs of Theorem 1, Theorem 2 and Theorem 3. We use the symbol ≍ to represent asymptotic equivalence. If a ≍ b, then a = O(b) and b = O(a).

Proof of Theorem 1. When δ_i = 1, we have

$$\frac{(1-{∊}_m)\phi (X_i;\hspace{0.17em}0,\hspace{0.17em}1)}{{∊}_m\phi (X_i;\hspace{0.17em}\mu ,\hspace{0.17em}{\sigma }^2)}<\frac{1}{\lambda },$$

which is equivalent to (X_i − μ)/σ ∈ (−∞, −c − dσ) ∪ (c − dσ, +∞), where

$$c={\left[\frac{2}{{\sigma }^2-1}\hspace{0.17em}\text{log}\hspace{0.17em}\left\{\frac{\lambda \sigma (1-{∊}_m)}{{∊}_m}\right\}+\frac{{\mu }^2}{{({\sigma }^2-1)}^2}\right]}^{1/2},d=\frac{\mu }{{\sigma }^2-1}.$$ (A1)

Since

$$m\text{FDR}=\frac{{\sum }_{i=1}^m\text{pr}({\delta }_i=1|{\theta }_i=0)\hspace{0.17em}(1-{∊}_m)}{{\sum }_{i=1}^m\{\text{pr}({\delta }_i=1|{\theta }_i=0)\hspace{0.17em}(1-{∊}_m)+\text{pr}({\delta }_i=1|{\theta }_i=1){∊}_m\}}⩽{\alpha }_1,$$

we have

$$\mathrm{\Phi }(-c\sigma -d)+\mathrm{\Phi }(-c\sigma +d)⩽c_1\{\mathrm{\Phi }(-c-d\sigma )+\mathrm{\Phi }(-c+d\sigma )\},$$ (A2)

where c₁ = α₁∊_m /{(1 − α₁)(1 − ∊_m)}. Similarly, since

$$\text{mFNR}=\frac{{\sum }_{i=1}^m\text{pr}({\delta }_i=0|{\theta }_i=1){∊}_m}{{\sum }_{i=1}^m\{\text{pr}({\delta }_i=0|{\theta }_i=0)\hspace{0.17em}(1-{∊}_m)+\text{pr}({\delta }_i=0|{\theta }_i=1){∊}_m\}}⩽{\alpha }_2,$$

we have

$$\mathrm{\Phi }(c\sigma -d)-\mathrm{\Phi }(-c\sigma -d)⩾c_2\{\mathrm{\Phi }(c-d\sigma )-\mathrm{\Phi }(-c-d\sigma )\},$$ (A3)

where c₂ = (1 − α₂)∊_m /{α₂(1 − ∊_m)}. Setting (A2) and (A3) as equalities and simplifying, we obtain (8). Let (ĉ, d̂) be the solution of this equation, then (A1) leads to μ̂ = d̂(σ² − 1).

Proof of Theorem 2. Define c and d as in (A1). Since

$$\text{FD}=m\text{pr}({\delta }_i=1|{\theta }_i=0)\hspace{0.17em}(1-{∊}_m)=m(1-{∊}_m)\hspace{0.17em}\{\mathrm{\Phi }(-c\sigma -d)+\mathrm{\Phi }(-c\sigma +d)\},$$

in order for false discoveries to converge to zero, −cσ + d should be negative and small enough. In addition, since

$$\begin{array}{c}-c\sigma +d=-\sigma \hspace{0.17em}{\left[\frac{2}{{\sigma }^2-1}\text{log}\hspace{0.17em}\{\sigma (1-m^{-\beta })\}+\frac{2\{\beta ({\sigma }^2-1)+\tau \}}{{({\sigma }^2-1)}^2}\hspace{0.17em}\text{log}\hspace{0.17em}m\right]}^{1/2}+\frac{{(2\tau \hspace{0.17em}\text{log})}^{1/2}}{{\sigma }^2-1}\\ \asymp \frac{\surd 2}{{\sigma }^2-1}\hspace{0.17em}\left[-\sigma \hspace{0.17em}{\{\beta ({\sigma }^2-1)+\tau \}}^{1/2}+{\tau }^{1/2}\right]\hspace{0.17em}{(\text{log}\hspace{0.17em}m)}^{1/2},\end{array}$$

as m → ∞, −cσ + d < 0. Because Φ(−cσ − d) ⩽ Φ(−cσ + d), we have

$$\text{FD}\asymp \frac{m}{c\sigma -d}\phi (c\sigma -d)\asymp \frac{1}{{(\text{log}\hspace{0.17em}m)}^{1/2}}{m}^{1-C_1},$$

where

$$C_1=\frac{{[\sigma {\{\beta ({\sigma }^2-1)+\tau \}}^{1/2}-{\tau }^{1/2}]}^2}{{({\sigma }^2-1)}^2}.$$

In order for false discoveries to converge to zero with the rate 1/{m^γ (log m)^1/2}, we require 1 − C₁ ⩽ − γ, which leads to τ ⩾ {(1 + γ)^1/2 + σ (1 + γ − β)^1/2}². Similarly,

$$\text{FN}=m\text{pr}({\delta }_i=0|{\theta }_i=1){∊}_m=m{∊}_m\{\mathrm{\Phi }(c-d\sigma )+\mathrm{\Phi }(-c-d\sigma )\},$$

and

$$\begin{array}{c}c-d\sigma ={\left[\frac{2}{{\sigma }^2-1}\text{log}\hspace{0.17em}\{\sigma (1-m^{-\beta })\}+\frac{2\{\beta ({\sigma }^2-1)+\tau \}}{{({\sigma }^2-1)}^2}\hspace{0.17em}\text{log}\hspace{0.17em}m\right]}^{1/2}-\frac{\sigma {(2\tau \hspace{0.17em}\text{log}\hspace{0.17em}m)}^{1/2}}{{\sigma }^2-1}\\ \asymp \frac{\surd 2}{{\sigma }^2-1}\left[{\{\beta ({\sigma }^2-1)+\tau \}}^{1/2}-\sigma {\tau }^{1/2}\right]\hspace{0.17em}{(\text{log}\hspace{0.17em}m)}^{1/2}.\end{array}$$

As m → ∞, in order for c − dσ < 0, we require {β(σ² − 1) + τ}^1/2 < σ/τ^1/2, which implies τ > β. Then,

$$\text{FN}\asymp \frac{m}{d\sigma -c}\phi (d\sigma -c)\asymp \frac{1}{{(\text{log}\hspace{0.17em}m)}^{1/2}}\hspace{0.17em}{m}^{1-\beta -C_2},$$

where

$$C_2=\frac{{[\sigma {(\tau )}^{1/2}-{\{\beta ({\sigma }^2-1)+\tau \}}^{1/2}]}^2}{{({\sigma }^2-1)}^2}.$$

It is easy to show that

$$1-\beta -C_2⩽-\gamma ,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\tau >\beta ,$$

is equivalent to τ ⩾ {(1 + γ)^1/2 + σ (1 + γ − β)^1/2}².

Proof of Theorem 3. Define S(x) = f (x)/φ(X_i), S̃(x) = S(x)/(1 − ∊_m). We have

$$\begin{array}{ll}\text{pr}\{\widehat{S}(X_i)>2|{\theta }_i=0\}\hfill & ⩽\text{pr}[|X_i|>{\{2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m\}}^{1/2}|{\theta }_i=0]\hfill \\ \hfill & \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\text{pr}[|X_i|⩽{\{2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m\}}^{1/2},\hspace{0.17em}\tilde{S}(x)>3/2|{\theta }_i=0]\hfill \\ \hfill & \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\text{pr}[|X_i|⩽{\{2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m\}}^{1/2},\hspace{0.17em}|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|{\theta }_i=0]\hfill \\ \hfill & =(A)+(B)+(C),\hfill \end{array}$$

say. We control (A), (B) and (C), respectively. First, we have

$$(A)=2\mathrm{\Phi }[-{\{2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m\}}^{1/2}]⩽C{m}^{-1-\gamma }/{(\text{log}\hspace{0.17em}m)}^{1/2}.$$

Using the same technique as the proof of Theorem 2, if τ ⩾ {(1 + γ)^1/2 + σ (1 + γ − β)^1/2}², then

$$(B)⩽\text{pr}\{\tilde{S}(X_i)>3/2|{\theta }_i=0\}=\text{pr}\{{\mathrm{\Lambda }}_{OR}(X_i)<2|{\theta }_i=0\}⩽C{m}^{-1-\gamma }/{(\text{log}\hspace{0.17em}m)}^{1/2}.$$

Consider the set 𝒳_i = {X_i : |X_i| ⩽ {2(1 + γ) log m}^1/2}, for all X_i ∈ 𝒳_i,

$$\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|X_i\}⩽\text{pr}\{|m\widehat{S}(X_i)-mS(X_i)|+|mS(X_i)-m\tilde{S}(X_i)|>m/2|X_i\},$$

and

$$|mS(X_i)-m\tilde{S}(X_i)|=\frac{m^{1-\beta }\hspace{0.17em}f(X_i)}{(1-{∊}_m)\phi (X_i)}.$$

Since the alternative density can be modelled as a Gaussian location-scale mixture, we have

$$f(X_i)=(1-{∊}_m)\phi (X_i)+\sum _{l=1}^L\frac{1}{{\sigma }_l}\phi \hspace{0.17em}\left(\frac{X_i-{\mu }_l}{{\sigma }_l}\right){∊}_{m,l},$$

where ∊_m,l is the mixture proportion, representing the probability that X_j is from N (μ_l, σ_l)). Under the condition

$${\mu }_l⩾\tau {\{{(1+\gamma )}^{1/2}+{\sigma }_l{(1+\gamma -{\beta }^{\prime })}^{1/2}\}}^2$$

and σ_l > 1 for all l = 1, …, L, we have f (X_i)/φ(X_i) < m^β′, for all X_i. Then,

$$|mS(X_i)-m\tilde{S}(X_i)|⩽2m^{1-(\beta -{\beta }^{\prime })},\hspace{0.17em}\beta >{\beta }^{\prime }.$$

Therefore, for m sufficiently large,

$$\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|X_i\in {𝒳}_i\}⩽\text{pr}\{|m\widehat{S}(X_i)-mS(X_i)|>m/3|X_i\}.$$

Let h = m^−ρ, we have

$$E\{m\widehat{S}(X_i)|X_i\}-mS(X_i)=m\int \frac{K(y)}{\phi (X_i)}\hspace{0.17em}f(X_i-yh)\hspace{0.17em}\text{d}y-m\frac{f(X_i)}{\phi (X_i)}.$$

Further, we take K (y) = φ(y) or any other symmetric kernel with a smaller tail than φ when |y| > {2(1 + γ) log m}^1/2. Using the fact that ∫ K (y) dy = 1 and ∫ yK (y) dy = 0,

$$E\{m\widehat{S}(X_i)|X_i\}-mS(X_i)=m\sum _{l=2}{(-1)}^l\hspace{0.17em}{h}^l\hspace{0.17em}\left\{\int y^l\hspace{0.17em}K(y)\text{d}y\right\}\frac{f^{(l)}\hspace{0.17em}(X_i)}{\phi (X_i)}.$$

Let H^l(x) be the lth Hermite polynomial. For X_i ∈ 𝒳_i,

$$|{(-1)}^l\hspace{0.17em}{f}^{(l)}\hspace{0.17em}(X_i)/\phi (X_i)|⩽(1-{∊}_m)|H^l\hspace{0.17em}(X_i)|+{∊}_m\sum _{k=1}^L\frac{1}{{\sigma }_k^{l+1}}\left|H^l\hspace{0.17em}\left(\frac{X_i-{\mu }_k}{{\sigma }_k}\right)\right|⩽C{(\text{log}\hspace{0.17em}m)}^{l/2}.$$

Therefore, for m sufficiently large,

$$\begin{array}{ll}\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|X_i\}\hfill & ⩽\text{pr}[|m\widehat{S}(X_i)-mE\{S(X_i)|X_i\}|\hfill \\ \hfill & \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+|E\{m\widehat{S}(X_i)|X_i\}-mS(X_i)|>m/3|X_i]\hfill \\ \hfill & ⩽\text{pr}[|m\widehat{S}(X_i)-mE\{S(X_i)|X_i\}|+C{m}^{1-2\rho }\hspace{0.17em}\text{log}\hspace{0.17em}m>m/3|X_i]\hfill \\ \hfill & ⩽\text{pr}[|m\widehat{S}(X_i)-mE\{S(X_i)|X_i\}|>m/4|X_i].\hfill \end{array}$$

Let $X_k^l=(X_k,\dots ,X_l)$, l ⩾ k. For example, $X_1^m=(X_1,\dots ,X_m)=X$. Given X_i ∈ 𝒳_i, define

$$g(X)=\frac{1}{h}\sum _{j=1}^m\frac{K\left(\frac{X_i-X_j}{h}\right)}{\phi (X_i)}=\widehat{S}(X_i),$$

then

$$\left|\frac{K\left(\frac{X_i-X_k}{h}\right)}{h\phi (X_i)}\right|⩽\frac{1}{h}\hspace{0.17em}\text{exp}\hspace{0.17em}\left[-\frac{1-h^2}{2h^2}{\left\{X_i-\frac{h}{1-h^2}\hspace{0.17em}{X}_k\right\}}^2-\frac{1-2h^2}{2h^2(1-h^2)}{X}_k^2\right]⩽\frac{1}{h}.$$

For k ≠ i,

$$|g(X_1^k,\hspace{0.17em}{X}_k,\hspace{0.17em}{X}_{k+1}^m)-g(X_1^k,\hspace{0.17em}{\tilde{x}}_k,\hspace{0.17em}{X}_{k+1}^m)|=|\frac{K\left(\frac{X_i-X_k}{h}\right)-K\left(\frac{X_i-{\tilde{x}}_k}{h}\right)}{h\phi (X_i)}|⩽\frac{2}{h}.$$

By McDiarmid’s inequality (McDiarmid, 1989),

$$\begin{array}{ll}\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|X_i\hspace{0.17em}\text{with }{X}_i\in {𝒳}_i\}\hfill & ⩽\text{pr}[|m\widehat{S}(X_i)-mE\{S(X_i)|X_i\}|>m/4|X_i\hspace{0.17em}\text{with }{X}_i\in {𝒳}_i].\hfill \\ \hfill & ⩽2\hspace{0.17em}\text{exp}(-m^{1-2\rho }/8).\hfill \end{array}$$

Therefore,

$$(C)⩽{\int }_{X_i\in {𝒳}_i}2\hspace{0.17em}\text{exp}(-m^{1-2\rho }/8)\phi (X_i)\text{d}{X}_i⩽2\hspace{0.17em}\text{exp}(-m^{1-2\rho }/8).$$

Thus,

$$\text{FD}=m(1-{∊}_m)\text{pr}\{\widehat{S}(X_i)>2|{\theta }_i=0\}⩽C{m}^{-\gamma }/{(\text{log}\hspace{0.17em}m)}^{1/2}.$$

Similarly,

$$\begin{array}{ll}\text{FN}\hfill & =m{∊}_m\hspace{0.17em}\text{pr}\{\widehat{S}(X_i)⩽2|{\theta }_i=1\}\hfill \\ \hfill & =m^{1-\beta }\hspace{0.17em}\text{pr}\{\widehat{S}(X_i)⩽2,|X_i|⩽{\{2(1+\gamma )\text{log}\hspace{0.17em}m\}}^{1/2}|{\theta }_i=1\}\hfill \\ \hfill & ⩽m^{1-\beta }[\text{pr}\{S(X_i)⩽5/2|{\theta }_i=1\}\hfill \\ \hfill & \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2,|X_i|⩽{[2(1+\gamma )\hspace{0.17em}\text{log}\hspace{0.17em}m]}^{1/2}|{\theta }_i=1\}]\hfill \\ \hfill & =m^{1-\beta }\{(D)+(E)\}.\hfill \end{array}$$

Using the same technique as the proof of Theorem 2, if

$$\tau ⩾{\{{(1+\gamma )}^{1/2}+\sigma {(1+\gamma -\beta )}^{1/2}\}}^2,$$

then (D) ⩾ Cm^−1+β−γ /(log m)^1/2. Finally,

$$(E)⩽{\int }_{X_i\in {𝒳}_i}\text{pr}\{|\widehat{S}(X_i)-\tilde{S}(X_i)|>1/2|X_i\}{f}^1\hspace{0.17em}(X_i)\hspace{0.17em}\text{d}{X}_i⩽2\hspace{0.17em}\text{exp}(-m^{1-2\rho }/8).$$

Thus,

$$\text{FD}=m(1-{∊}_m)=m\{(D)+(E)\}⩽C{m}^{-\gamma }/{(\text{log}\hspace{0.17em}m)}^{1/2}.$$