A Weighted-Holm Procedure Accounting for Allele Frequencies in Genomewide Association Studies

Abstract

In the context of genomewide association studies where hundreds of thousand of polymorphisms are tested, stringent thresholds on the raw association test P-values are generally used to limit false-positive results. Instead of using thresholds based on raw P-values as in Bonferroni and sequential Sidak (SidakSD) corrections, we propose here to use a weighted-Holm procedure with weights depending on allele frequency of the polymorphisms. This method is shown to substantially improve the power to detect associations, in particular by favoring the detection of rare variants with high genetic effects over more frequent ones with lower effects.

THE development of high-throughput genotyping technologies, allied to a deeper understanding of the pattern of human sequence variation, has recently offered the opportunity to perform association studies with hundreds of thousands of single-nucleotide polymorphisms (SNPs) expected to cover the whole genome. Recent successes in human complex diseases such as type II diabetes mellitus (Scott et al. 2007; Sladek et al. 2007; Zeggini et al. 2007), Crohn's disease (Duerr et al. 2006; Rioux et al. 2007), prostate cancer (Yeager et al. 2007), and coronary artery disease (Helgadottir et al. 2007; Samani et al. 2007) provided compelling proof-of-principle. Most recent genomewide association studies (GWAS) identified “at-risk” SNPs with minor allele frequency (MAF) ∼0.30 and associated with allelic odds ratio ∼1.30. This could be explained by the fact that the SNP arrays used to perform GWAS were designed under the hypothesis that genetic susceptibility to complex diseases involves common variants that confer moderate to low relative risk (i.e., the common disease–common variants hypothesis) and are thus enriched in common variants. However, examples from the literature (see Cambien and Tiret 2007) show that rare variants with stronger effects may also contribute to the genetic architecture of complex diseases and it might then be of interest to develop statistical methods that will favor their detection over that of most frequent variants.

The statistical analysis of such a large number of SNPs requires imposing stringent thresholds on the association tests' P-values to control for the risk of false positives. In the context of multiple-testing procedures (MTP) controlling the familywise error rate (FWER), several methods based on raw P-values including Bonferonni, sequential Sidak (SidakSD), and Holm (Holm 1979) are available. These standard MTPs can be improved by weighting P-values according to some specific criteria to increase power (Benjamini and Hochberg 1997; Kropf and Lauter 2002; Kropf et al. 2004; Westfall et al. 2004). In the field of GWAS, Roeder et al. (2007) have recently shown that grouping SNPs within a priori sets subsequently used for defining weights could considerably improve the power to detect association. Instead of grouping SNPs, we here proposed to directly weight association tests' P-values according to the SNP's MAF estimated in the control sample and to apply the weighted-Holm (WH) procedure (Benjamini and Hochberg 1997). This strategy is expected to favor the detection of rare SNPs with a high odds ratio (OR) over that of frequent SNPs with a moderate OR. A simulation study was carried out to study the statistical properties of the proposed WH procedure and demonstrated that this very simple WH procedure can greatly improve the power of GWAS over standard MTPs controlling the FWER.

Following the general notations of Benjamini and Hochberg (1997), we define, for each SNP i (i = 1–N) with MAF f_i in the control sample, a standardized weight w_i as w_i = .

Let p₁, p₂, … , p_N be the P-values of the association test for each SNP i (i = 1, 2, … , N) and be their weighted counterpart. Rank these in increasing order ( and finally note as the standardized weight corresponding to the SNP associated with . To control the FWER at level α, our proposed sequential WH procedure declares significant at level α any SNP whose satisfies for j = 1–i. The smaller the MAF, the larger the weight w_i and thus the greater the power for rejecting the null hypothesis of no association with the disease. When all weights are fixed to 1, this procedure is equivalent to the standard Holm procedure (Holm 1979).

Simulated data:

Data were simulated under a balanced case–control design of 1000 individuals genotyped for N independent SNPs. A 10-locus disease model was considered with independent genetic effects. For the N − 10 nondisease SNPs, MAFs were randomly drawn from a uniform distribution U[0.05–0.50]. Concerning the disease SNPs, their MAFs in controls were randomly drawn from U[0.05–0.15] for 3 of them, from U[0.15 – 0.25] for another 3, and from U[0.30–0.40] for the remaining 4. The corresponding frequencies in cases were set to be compatible with predefined allelic effects as described below. For each particular setting, MAFs were fixed across the 2000 simulated replicates. Genotypes were then generated under the assumption of Hardy–Weinberg equilibrium in cases and in controls. Association was tested at the 0.05 significance level, using a simple allelic χ²-test statistic with 1 d.f. Different patterns of association and different values for the number N of SNPs (1000, 2000, 5000, 10,000, 20,000, 50,000) were investigated. The performances of our proposed WH procedure were compared in terms of type I error and power to those of several standard unweighted MTPs, including Bonferroni, Holm, and SidakSD (Holm 1979). The allele frequencies used for computing the weights w_i were estimated from the control sample of each replicate.

Results:

For each studied MTP, we first checked that, under the complete null hypothesis (no SNP associated with the disease), the proportion of replicates in which at least one nondisease SNP was found significantly associated with the disease was always in agreement with the 0.05 threshold for the FWER (data not shown). False-positive SNPs tended, however, to be slightly less frequent with the WH procedure than with the unweighted methods (data not shown). These observations also hold under the different patterns of association considered (see below), confirming that all the studied MTPs correctly controlled the FWER at the desired level of 0.05.

Three different patterns of associations were considered when investigating the MTPs' power. For ease of presentation, results obtained with the Bonferroni procedure were not shown since they were very similar to those obtained with the Holm method. The first situation considered was a situation where disease SNPs with MAF <0.15 were associated with an allelic OR of 1.8, those with MAF between 0.15 and 0.25 were associated with an allelic OR of 1.5, while the last 4 SNPs with MAF between 0.30 and 0.40 were associated with an OR of 1.3. As shown in Figure 1, the WH procedure was more powerful than the Holm and SidakSD procedures. Interestingly, the power improvement increased with the number of tested SNPs, ranging from an ∼1.3% increase (e.g., 0.961 vs. 0.947) when N = 5000 to a 3.0% increase (e.g., 0.817 vs. 0.790) when N = 50,000. The good performance of the WH procedure over other methods was also illustrated by an improvement in the average number of identified disease SNPs per replicate. When N = 5000, the WH procedure provided a 3.0% increase in the number of true disease SNPs detected by comparison to SidakSD and a 7.5% increase when N = 50,000. When conditioning on there being at least one identified disease SNP, the number of detected SNPs is still higher with the WH method even though the gain of efficiency is less important. To illustrate how weighting influenced these results, Table 1 shows the power to detect the effect of each individual SNP when the total number of simulated SNPs was N = 20,000. As expected, the WH procedure favored the detection of SNPs with low MAF and high ORs over SNPs with higher MAF but lower ORs, while controlling for the expected FWER. The same observations hold for other values of N (data not shown).

Figure 1.—

Overall power of the different multiple-testing procedures when allelic odds ratios were, respectively, 1.80, 1.50, and 1.30 for the disease SNPs with low, moderate, and frequent MAF. Numbers above each bar correspond to the expected number of disease SNPs identified per replicate while numbers given below correspond to the expected number of disease SNPs identified conditional on there being at least one. SidakSD, sequential Sidak; WH_low, proposed weighted-Holm procedure.

TABLE 1.

Power (SE) for detecting the effect of each SNP under the first pattern of association considered (N = 20,000)

	Allele frequency	Allelic OR	Holm	SidakSD	WH
SNP1	0.055	1.80	0.0875	0.0890	0.1490
			(0.0063)	(0.0064)	(0.0079)
SNP2	0.102	1.80	0.3590	0.3610	0.4160
			(0.0107)	(0.0107)	(0.0110)
SNP3	0.104	1.80	0.3880	0.3900	0.4445
			(0.0109)	(0.0109)	(0.0111)
SNP4	0.209	1.50	0.1990	0.2005	0.1930
			(0.0089)	(0.0089)	(0.0088)
SNP5	0.232	1.50	0.2405	0.2425	0.2295
			(0.0096)	(0.0096)	(0.0094)
SNP6	0.238	1.50	0.2420	0.2435	0.2305
			(0.0096)	(0.0096)	(0.0094)
SNP7	0.360	1.30	0.0300	0.0300	0.0250
			(0.0038)	(0.0038)	(0.0035)
SNP8	0.361	1.30	0.0325	0.0335	0.0260
			(0.0040)	(0.0040)	(0.0036)
SNP9	0.384	1.30	0.0355	0.0355	0.0245
			(0.0041)	(0.0041)	(0.0035)
SNP10	0.389	1.30	0.0335	0.0340	0.0235
			(0.0040)	(0.0040)	(0.0034)
FWER			0.055	0.0560	0.0540

SidakSD, sequential Sidak; WH, proposed weighted-Holm procedure; FWER, familywise error rate (proportion of replicates in which at least one nondisease SNP was found significantly associated with the disease).

In the second pattern of association considered, the allelic effects of the 10 disease SNPs were defined such that they each correspond to an attributable fraction of 0.05. Results are summarized in Figure 2. Again, the WH procedure outperformed the unweighted MTPs in terms of power and of average number of detected disease SNPs. When N = 1000, the power improvement over the SidakSD was ∼16% (0.684 vs. 0.589) and increased up to 41% (0.244 vs. 0.172) when N = 50,000. In terms of average number of identified disease SNPs, the WH efficiency over the SidakSD increased from 20% when N = 1000 to 41% when N = 50,000. Again, the power of detecting SNPs with low MAF but higher OR was increased using the WH procedure (Table 2).

Figure 2.—

Overall power of the different multiple-testing procedures when the 10 disease SNPs were associated with an attributable fraction of 0.05. Numbers above each bar correspond to the expected number of disease SNPs identified per replicate while numbers given below correspond to the expected number of disease SNPs identified conditional on there being at least one. SidakSD, sequential Sidak; WH_low, proposed weighted-Holm procedure.

TABLE 2.

Power (SE) for detecting the effect of each SNP associated with an attributable fraction of 0.05 (N = 20,000)

	Allele frequency	Allelic OR	Holm	SidakSD	WH
SNP1	0.056	1.93	0.2075	0.2085	0.3000
			(0.0091)	(0.0091)	(0.0102)
SNP2	0.125	1.42	0.0250	0.0250	0.0375
			(0.0035)	(0.0035)	(0.0042)
SNP3	0.131	1.40	0.0285	0.0285	0.0335
			(0.0037)	(0.0037)	(0.0040)
SNP4	0.176	1.30	0.0100	0.0115	0.0135
			(0.0022)	(0.0024)	(0.0026)
SNP5	0.192	1.27	0.0075	0.0075	0.0075
			(0.0019)	(0.0019)	(0.0019)
SNP6	0.243	1.22	0.0015	0.0015	0.0015
			(0.0009)	(0.0009)	(0.0009)
SNP7	0.338	1.16	0.0005	0.0005	0.0005
			(0.0005)	(0.0005)	(0.0005)
SNP8	0.356	1.15	0.0010	0.0010	0.0010
			(0.0007)	(0.0007)	(0.0007)
SNP9	0.371	1.14	0.0015	0.0015	0.0005
			(0.0009)	(0.0009)	(0.0005)
SNP10	0.372	1.14	0.0000	0.0000	0.0000
			(-)	(-)	(-)
FWER			0.0570	0.0580	0.0490

SidakSD, sequential Sidak; WH, proposed weighted-Holm procedure; FWER, familywise error rate (proportion of replicates in which at least one nondisease SNP was found significantly associated with the disease).

However, looking carefully at Tables 1 and 2 revealed that the two patterns of association we have considered up to now corresponded to situations where SNPs with low MAF had much more power to be detected than common SNPs, whatever the MTP used. Therefore, a third situation was considered, where genetic effects were defined such that all allelic effects have approximately the same chance to be detected. For a given MAF in controls, the MAF in cases was computed using the arcsin transformation such that the power (uncorrected for multiple testing) for detecting an allelic case–control difference was 50% at a 0.05 significance level. Results are reported in Figure 3. Again, the WH procedure is more powerful than the unweighted methods but the gain in power is quite moderate. It achieved only ∼1.8% (0.566 vs. 0.556) when N reached 50,000, and the average number of disease sites detected is only 3.9% larger than with the unweighted procedures. Under this scenario, we found that both Holm and SidakSD corrections tend to disadvantage rare disease susceptibility alleles in favor of more frequent ones as illustrated in Table 3. One explanation for this result could be linked to the fact that the variance of the conditional distribution of the test statistic is lower for frequent alleles. As expected, the WH procedure does the opposite with a 6.7% power increase for SNP1 with the lowest MAF and a 2% decrease for SNP10 with the highest MAF.

Figure 3.—

Overall power of the different multiple-testing procedures when each disease SNP had the same power to be detected without correcting for multiple testing. Numbers above each bar correspond to the expected number of disease SNPs identified per replicate while numbers given below correspond to the expected number of disease SNPs identified conditional on there being at least one. SidakSD, sequential Sidak; WH_low, proposed weighted-Holm procedure.

TABLE 3.

Power (SE) for detecting the effect of each SNP when all SNPs have approximately the same power to be detected using an allelic association test not corrected for multiple testing (N = 20,000)

	Allele frequency	Allelic OR	Holm	SidakSD	WH
SNP1	0.055	1.82	0.0905	0.0920	0.1590
			(0.0064)	(0.0065)	(0.0082)
SNP2	0.074	1.71	0.1070	0.1090	0.1525
			(0.0069)	(0.0070)	(0.0080)
SNP3	0.129	1.54	0.1025	0.1055	0.1300
			(0.0068)	(0.0069)	(0.0075)
SNP4	0.184	1.46	0.1115	0.1130	0.1155
			(0.0070)	(0.0071)	(0.0071)
SNP4	0.230	1.43	0.1055	0.1065	0.1025
			(0.0069)	(0.0069)	(0.0068)
SNP6	0.236	1.42	0.1095	0.1110	0.1025
			(0.0070)	(0.0070)	(0.0068)
SNP7	0.312	1.39	0.1115	0.1135	0.0945
			(0.0070)	(0.0071)	(0.0065)
SNP8	0.346	1.38	0.1120	0.1130	0.0890
			(0.0071)	(0.0071)	(0.0064)
SNP9	0.361	1.38	0.1270	0.1285	0.1050
			(0.0074)	(0.0075)	(0.0069)
SNP10	0.390	1.37	0.1125	0.1135	0.0925
			(0.0071)	(0.0071)	(0.0065)
FWER			0.0380	0.0395	0.0410

SidakSD, sequential Sidak; WH, proposed weighted-Holm procedure; FWER, familywise error rate (proportion of replicates in which at least one nondisease SNP was found significantly associated with the disease).

To illustrate how the proposed weighting could affect the results of an association study, we compare the results of the weighted-Holm and Holm procedures applied to the analysis of 397,857 SNPs in a sample of 1926 coronary artery disease patients and 2938 controls participating in the Wellcome Trust Case–Control Consortium (WTCCC) initiative (Wellcome Trust Case–Control Consortium 2007). Even if, after controlling for multiple testing using either the Holm or the weighted-Holm procedure, no SNP reached statistical significance, Table 4 clearly indicates that the weighted-Holm procedure alters the order of “significance” in favor of less frequent SNPs by comparison to the Holm procedure. The same phenomenon holds for the SidakSD method (data not shown).

TABLE 4.

Application of the weighted-Holm procedure to the analysis of 397,857 SNPs in a case–control study for coronary artery disease including 1926 cases and 2938 controls

Minor-allele frequency		P-value		Thresholda		Rank
Controls (N = 2938)	Cases (N = 1926)	Standard P	Weighted P	Holm	WH	Holm	WH
0.2520	0.2946	2.8493E-06	6.2791E-06	1.9986E-06	1.9986E-06	1	1
0.0076	0.0159	0.00010776	7.1922E-06	1.9987E-06	1.9986E-06	3	2
0.0140	0.0247	9.6101E-05	1.1773E-05	1.9986E-06	1.9998E-06	2	3
0.0310	0.0449	0.00030311	8.2198E-05	1.9997E-06	2.0004E-06	15	4
0.0083	0.0150	0.00178229	0.00012953	2.0044E-06	2.0007E-06	74	5
0.0706	0.0912	0.00021625	0.00013362	1.9991E-06	2.0018E-06	8	6
0.1396	0.1131	0.00011519	0.00014064	1.9988E-06	2.0020E-06	4	7
0.0139	0.0071	0.00155897	0.00018916	2.0034E-06	2.0020E-06	62	8
0.1396	0.1137	0.00015854	0.00019353	1.9989E-06	2.0027E-06	5	9
0.1366	0.1112	0.00019061	0.00022766	1.9990E-06	2.0028E-06	6	10
0.0250	0.0363	0.00107441	0.00023489	2.0023E-06	2.0028E-06	48	11
0.0739	0.0934	0.00052638	0.00034032	2.0004E-06	2.0032E-06	24	12
0.0732	0.0924	0.00058348	0.00037358	2.0006E-06	2.0033E-06	27	13
0.2136	0.2456	0.00020242	0.00037817	1.9990E-06	2.0034E-06	7	14
0.0182	0.0106	0.00240808	0.00038244	2.0076E-06	2.0035E-06	113	15
0.0191	0.0113	0.00231635	0.00038787	2.0074E-06	2.0040E-06	111	16
0.0195	0.0116	0.00228809	0.00038980	2.0070E-06	2.0045E-06	106	17
0.0465	0.0615	0.00099541	0.00040437	2.0019E-06	2.0050E-06	43	18
0.0576	0.0427	0.00099790	0.00050262	2.0020E-06	2.0052E-06	44	19
0.0929	0.0733	0.00062577	0.00050818	2.0009E-06	2.0053E-06	30	20
0.2338	0.2028	0.00025398	0.00051937	1.9994E-06	2.0054E-06	11	21
0.2687	0.3025	0.00023629	0.00055516	1.9992E-06	2.0054E-06	9	22
0.0735	0.0565	0.00089512	0.00057518	2.0017E-06	2.0055E-06	40	23
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0745	0.0574	0.00089725	0.00058456	2.0018E-06	2.0056E-06	41	24
0.0492	0.0358	0.00142020	0.00061103	2.0030E-06	2.0057E-06	57	25

Only SNPs significantly associated at 0.05 FWER are shown. WH, weighted Holm.

^aStatistical threshold for declaring significance at 0.05 familywise error rate.

In this brief report, we showed how the weighted-Holm procedure could be used for slightly improving the power of association analyses when the number of tested SNPs is large and when MTPs controlling FWER are considered. This simple procedure just consists of weighting association-test P-values by the inverse allele frequencies. It is more appropriate when polymorphisms underlying the disease of interest are assumed to have low allele frequency but larger genetic effects rather than moderate/high frequency with smaller effects. Such SNPs with low MAF but larger genetic effects are expected to be of more clinical importance since they could “constitute potential diagnostic and prognostic tools” (Cambien and Tiret 2007) even though they may be less easily tagged by current DNA chips (Evans et al. 2008). In view of the recent finding by Evans et al., our method may then be more suitable for genomewide scans of nonsynonymous variants (Evans et al. 2008) that are particularly useful for the detection of “rare variants with intermediate penetrances” rather than common SNPs with moderate effects. It must be emphasized that all simulations were performed assuming uniform distribution of allele frequencies, a situation that is not realistic since SNPs with low MAF are expected to be more frequent over the genome than SNPs with higher MAF. However, it might represent the content of currently available DNA chips as the number of SNPs is now increasing up to 1 million and as reflected by the frequency spectrum in HapMap (see Figure 5 in the International HapMap Consortium 2005). Another assumption underlying these simulations is the independence, i.e., absence of linkage disequilibrium (LD), between SNPs. While one cannot rule out the possibility that LD may affect the behavior of our WH procedure, it must be stressed that LD is not taken into account in most published GWAS to select SNPs that would deserve replication in independent studies. This issue is becoming of less importance as current DNA chips tend to rely on tagging SNPs to eliminate part of the dependency between SNPs. All this work was based on the MAF inverse functions for defining weighted P-values. We have also looked at other functions (identity, log, square root, entropy) but the inverse function was the one that improved best the power for detecting a low-MAF allele. Weights giving more chance to frequent SNPs did not substantially outperform the SidakSD or the Holm procedure (data not shown). Further work would, however, be required for deeply investigating that point.

Moreover, since in the simulations the weights used with the WH procedure were derived from allele-frequency estimates obtained on each control sample replicate, they were not completely independent of the P-value. We do not think it invalidates the approach since all simulations showed that the WH procedure correctly controls for the desired FWER. However, it could have an impact on the power since weights of rare variants are then estimated with less precision than those of common variants. Some additional work will be needed to investigate this issue with, for example, an evaluation of the relevance of accounting for the variance of the allele-frequency estimates in the weights.

Compared to the approach proposed by the WTCCC (Wellcome Trust Case–Control Consortium 2007) that consists of using Bayes factors, our proposed WH procedure does not make any assumption on the effect sizes of the different SNPs. However, as shown by our simulations, the performances of the WH procedure strongly depend on the effect of the different SNPs. Should the SNPs with low MAF have effects (measured here by the allelic odds ratio) similar to the ones of SNPs with high MAF, then the procedure will not be of any interest. Indeed, it will reduce the power to detect the high-MAF SNPs that were the ones that should be the most easily detectable. In this respect, it will do just the opposite of the Bayes factor approach. However, in the situation where the expected power to detect SNPs with low and high MAFs is the same by unweighted approaches (see Table 3), then the proposed approach will boost the SNPs with low MAF that are also most likely to be nonsynonymous SNPs themselves or proxies for nonsynonymous SNPs. It could be of interest to compare the WH procedure to a Bayes factor approach where priors will be higher from SNPs with low MAFs than from SNPs with high MAFs.

As noted by Dudoit et al. (2003, p. 74), “within the class of multiple testing procedures that control a given type I error rate at an acceptable level α, one seeks procedures that maximize power, that is, minimize a suitably defined type II error rate.” In our work, we have chosen to control the FWER as this is a more conservative error criterion than the false discovery rate (FDR) one and also because most published GWAS rely on FWER rather than FDR. Our simulations showed that the WH procedure controls the FWER at the correct level. However, application of the weighted FDR proposed by Benjamini and Hochberg (1997) in the context of GWAS also deserves attention.