Improving power of association tests using multiple sets of imputed genotypes from distributed reference panels

Abstract

The accuracy of genotype imputation depends upon two factors: the sample size of the reference panel and the genetic similarity between the reference panel and the target samples. When multiple reference panels are not consented to combine together, it is unclear how to combine the imputation results to optimize the power of genetic association studies. We compared the accuracy of 9,265 Norwegian genomes imputed from three reference panels – 1000 Genomes Phase 3 (1000G), Haplotype Reference Consortium (HRC), and a reference panel containing 2,201 Norwegian participants from the population-based Nord Trøndelag Health Study (HUNT) from low-pass genome sequencing. We observed that the population-matched reference panel allowed for imputation of more population-specific variants with lower frequency (minor allele frequency (MAF) between 0.05% and 0.5%). The overall imputation accuracy from the population-specific panel was substantially higher than 1000G and was comparable with HRC, despite HRC being 15-fold larger. These results recapitulate the value of the population-specific reference panels for genotype imputation. We also evaluated different strategies to utilize multiple sets of imputed genotypes to increase the power of association studies. We observed that testing association for all variants imputed from any panel results in higher power to detect association than the alternative strategy of including only one version of each genetic variant, selected for having the highest imputation quality metric. This was particularly true for lower-frequency variants (MAF < 1%), even after adjusting for the additional multiple testing burden.

Introduction

Many novel disease-associated signals for a wide variety of diseases and traits have been successfully identified using imputation-based meta-analyses(Cheng & Thompson, 2016; Cooper et al., 2008; De Jager et al., 2009; Ge et al., 2016; Horikoshi et al., 2015; Houlston et al., 2008; Jin et al., 2016; Loos et al., 2008; Ruth et al., 2015; Zeggini et al., 2008; Zeggini et al., 2007). Genotype imputation is the process of inferring missing genotypes in study samples using a reference panel of high-density haplotypes(Li, Willer, Sanna, & Abecasis, 2009). Imputation allows variants that are not directly genotyped to be studied without other costs than computation. Previous simulations showed that imputation substantially increases the power of association studies to detect causal loci(Marchini & Howie, 2010; Spencer, Su, Donnelly, & Marchini, 2009). Imputation-based genome-wide association studies (GWAS) have successfully identified novel signals that were undetected in chip-based studies. For example, two disease-associated signals were detected in the 1000G-based imputation(Auton et al., 2015) for the Wellcome Trust Case Control Consortium phase 1 Data (WTCCC), which were missed in the original WTCCC GWAS study that was performed four years before(Burton et al., 2007; J. Huang, Ellinghaus, Franke, Howie, & Li, 2012). Imputation also facilitates fine-mapping studies by allowing most polymorphic variants, including causative ones, to be tested in known disease associated loci. For example, the strongest association signal, observed at the imputed variant rs7903146 of the TCF7L2 locus in the WTCCC type 2 diabetes scan, is suggested to be causal association in the locus(Mahajan et al., 2014; Marchini, Howie, Myers, McVean, & Donnelly, 2007). Furthermore, imputation allows for meta-analysis between samples that have been genotyped using different arrays, increasing power.

However, for studies that have access to population-matched genome sequenced individuals, there is uncertainty in deciding between a smaller, ancestry-matched reference panel and a larger publicly-available cosmopolitan reference panel. An ideal reference panel is expected to have closely matched ancestry to study samples because the genetic similarity increases the accuracy of imputation(Deelen et al., 2014; G. H. Huang & Tseng, 2014; J. Huang et al., 2015; Low-Kam et al., 2016; Mitt et al., 2017; Okada, Momozawa, Ashikawa, Kanai, & Matsuda, 2015; Pistis et al., 2015; Roshyara & Scholz, 2015; Walter et al., 2015). On the other hand, the imputation accuracy increases when larger reference panels are used, especially for lower-frequency variants(Browning & Browning, 2009; B. N. Howie, Donnelly, & Marchini, 2009; L. Huang et al., 2009; Y. Li et al., 2009; Roshyara & Scholz, 2015). Furthermore, different whole-genome reference panels may generate discordant imputed genotypes for the same variants in the same study samples. This brings in challenges for the follow-up association tests. The optimal strategy to perform association tests using genotypes imputed by different reference panels remains unclear. IMPUTE2 provides one possible approach to merge all reference panels to a single larger panel for genotype imputation when multiple reference panels are available (B. N. Howie et al., 2009), which may avoid the problem that different versions of genotypes are imputed for the same variants. The Genome of the Netherlands Consortium and the UK10K study has further shown that the combined reference panel of 1000G and the population-specific reference resulted in better imputation results compared to the two individual panels for rare variants(Deelen et al., 2014; J. Huang et al., 2015). However, this approach is not feasible when individual-level haplotypes within the reference panel are not accessible, as is the case with the Haplotype Reference Consortium (HRC)(McCarthy et al., 2016), primarily due to ethical issues surrounding sharing of individual-level genetic data(McCarthy et al., 2016). Here we genotyped 9,265 Norwegian participants from the HUNT study(Krokstad et al., 2013) for 350,270 polymorphic autosomal variants using the Illumina Human CoreExome array with approximately 240,000 GWAS tagging markers. We created a population-matched reference panel by whole-genome sequencing (WGS) 2,021 individuals from the HUNT study to a mean depth of 5×. We imputed variants from the HUNT WGS reference panel as our ethnically matched panel. We also performed imputation with two additional imputation reference panels: the HRC(McCarthy et al., 2016) and 1000G Phase 3(Auton et al., 2015). First, we systematically evaluated and compared the imputation results from the three reference panels, including the number of successfully imputed variants as well as the imputation accuracy. Next, we evaluated and compared the power of association tests between two approaches to incorporate multiple versions of imputed genotypes. First is the “best Rsq” approach, which retains imputed genotypes only from the panel with highest imputation quality metrics for each variant. Second is the “best p-value” approach that tests association with all imputed genotypes and uses the most significant association p value, adjusting for the additional variants tested.

Materials and Methods

Array-based genotyping

9,265 samples from the HUNT Biobank in Norway were genotyped at 350,270 polymorphism autosomal variants using an Exome + GWAS chip array (HumanCoreExome-12 v1.0, Illumina). Genotype calling was performed using GenTrain version 2.0 in GenomeStudio V2011.1 (Illumina). Samples with <98% genotype calls (N = 37), evidence of gender discrepancy (N = 21), duplicates (N = 66) as well as individuals with non-Norwegian ancestry identified by plotting the first 10 genotype-driven principal components(Springer-Verlag, 1986) (N = 7) were excluded from further analysis (N = 131, 1.19%). As Figure S1 shows, the HUNT GWAS samples have similar ancestry to the samples in the HUNT WGS reference panel. All HUNT research subjects provided informed written consent and IRB approval was obtained for genetic studies.

Relatedness was evaluated based on the estimation of the proportion of identity by descent (IBD) by PLINK(Purcell et al., 2007). We excluded 1,644 samples from the HUNT GWAS sample due to 1^st or 2^nd degree relatedness to samples in HUNT WGS, defined as IBD ≥ 0.25. We excluded samples that were related to samples within the reference panel to avoid inflating imputation statistics for regions inherited IBD. We performed variant-level quality control by excluding 19,872 variants that met any of the following criteria; variants with a cluster separation score < 0.3 reported by GenomeStudio V2011.1 (Illumina), < 95% genotype call rate, or deviation from Hardy–Weinberg equilibrium (P < 1 × 10⁻⁵).

Genotype imputation

Genotype imputation with the 1000G Phase 3(Auton et al., 2015) and the HRC(McCarthy et al., 2016) reference panels was conducted using the Michigan Imputation Server(Das, Forer, Schonherr, Sidore, & Locke, 2016) and imputation with the HUNT WGS reference panel was conducted using a local server. The study samples were phased using SHAPEIT2(v2.r790)(Delaneau, Zagury, & Marchini, 2013) followed by imputation using minimac3(v2.0.1)(Fuchsberger, Abecasis, & Hinds, 2015; B. Howie, Fuchsberger, Stephens, Marchini, & Abecasis, 2012). Two imputation metrics output by minimac3 were used for evaluating the imputation quality: ImpRsq and EmpRsq. ImpRsq is previously known as r̂² in different versions of the MaCH/minimac(Fuchsberger et al., 2015; B. Howie et al., 2012; Li, Willer, Ding, Scheet, & Abecasis, 2010). ImpRsq is defined for both genotyped and ungenotyped variants in the chip array as an estimate of the squared correlation between imputed dosages and true, unobserved genotypes, calculated as the observed variance over the expected variance. EmpRsq is defined only for genotyped variants in the chip array as the squared correlation between leave-one-out imputed dosages and the true, observed genotypes (See “Estimated Imputation Accuracy” section at http://genome.sph.umich.edu/wiki/Minimac_Diagnostics for details).

Reference panels

The HUNT WGS reference panel contains 1,101 earliest onset cases with myocardial infarction and 1,100 age and sex matched controls that were selected from the HUNT study(Krokstad et al., 2013). Whole genome sequencing to ~5× depth was performed on either Illumina HiSeq 2000 or 2500. We followed the GotCloud SNP calling pipeline to process the whole genome sequencing data(Jun, Wing, Abecasis, & Kang, 2015). The variant sites and genotype likelihood were called using SAMtools(H. Li et al., 2009) and the genotypes for SNPs were refined and phased using Beagle v4(Browning & Browning, 2013). After quality control, 20.2 million single nucleotide variants were retained in 2,201 samples, of which 4 million were unique to our study; not observed in dbSNP 144(Sherry et al., 2001), 1000 Genomes Phase 3(Auton et al., 2015), UK10K(Walter et al., 2015), ESP6500(W. NHLBI GO Exome Sequencing Project (ESP) Seattle, 2013), or ExAC.r0.3(Lek et al., 2016) (Table 1). The individuals in the HUNT WGS panel have similar ancestry to the HUNT study samples (Figure S1) and are from the same geographic region, although we excluded in the genotyped samples any 1^st or 2^nd degree relatives of the sequenced samples to avoid biased estimates of the accuracy of imputation. Additionally, there were no close relatives within the sequenced samples. The other two reference panels that we used for genotype imputation are the 1000 Genomes Phase 3 (1000G)(Auton et al., 2015) and the HRC release 1(McCarthy et al., 2016) containing 32,488 individuals, both of which are pre-stored in the Michigan Imputation Server(Das et al., 2016) (Table 2). The HUNT cohort contributed an early freeze of whole genome sequencing data consisting of 1,023 samples to the HRC consortium. Thus, the HUNT WGS and the HRC reference panels have 1,023 samples in common. Variants with minor allele counts (MAC) less than or equal to 5 were excluded from HRC(McCarthy et al., 2016).

Table 1.

Summary of the variants in the HUNT whole-genome sequencing reference panel containing 2,201 individuals with average sequencing depth 5×.

Variant Type	Total number of variants	Mean number of variants per individual (SD)	Mean number of unique variants per individual (SD)	% in 1000 Genomes	Number of novel variants*
Splice	1,265	71.5(4.6)	0.2(0.47)	36.6	355
Nonsense	2,432	71.5(6)	0.43(0.74)	36.6	585
Missense	113,576	9,480(113)	13.8(13.6)	56.3	13,927
Synonymous	77,699	10,707(100)	7.1(7.5)	68.5	5,935
Noncoding	20,050,237	3,342,839(15,415)	1531(906)	68.7	4,030,199
Total	20,245,209	3,363,168(15,522)	1,552(919)	68.6	4,051,001

^*Novel: not reported in dbSNP 144(Sherry et al., 2001), 1000 Genomes Phase 3(Auton et al., 2015), UK10K(Walter et al., 2015), ESP6500(W. NHLBI GO Exome Sequencing Project (ESP) Seattle, 2013), or ExAC.r0.3(Lek et al., 2016)

Table 2.

Reference panels used for genotype imputation

Reference Panels	Variants	Sample Size	Population
Haplotype Reference Consortium(McCarthy et al., 2016) (HRC)	39 million SNPs (MAC ≥ 5)	32,488a	Cosmopolitan (mostly European)
1000 Genomes Phase 3 Version 5(Auton et al., 2015) (mean depth < 8×)	81 million Biallelic SNPs, indels, deletions, complex short substitutions and other structural variant classes (MAC ≥ 2)	2,504	Cosmopolitan
HUNT Whole Genome Sequencing (HUNT WGS) (mean depth ~ 5×)	20 million SNPs	2,201a	Norwegian

^aHRC and HUNT whole-genome sequencing data set have 1,023 samples in overlap.

MAC: minor allele count

Permutation test

To determine the genome-wide significance thresholds for association tests using the two approaches to incorporate imputed genotypes, we performed permutation tests. The measurements of the high-density lipoprotein (HDL) cholesterol for the study samples were permuted 1,000 times. Each permutation was followed by a genome-wide association test (GWAS) using the permuted phenotypes. The most significant p-values from each of the 1,000 GWAS were ranked. And the significant threshold with family-wise error rate (FWER) n/1000 equals to the nth smallest p-value. Because the “best p-8 value” approach tests more variants, it will be a more stringent significant threshold than the “best Rsq” approach.

Power estimation

In order to estimate the power to detect association under the two approaches to incorporate imputed genotypes from multiple reference panels, we considered directly genotyped variants as causal variants, and used multiple sets of imputed genotypes to evaluate the power. First, we obtained the leave-one-variant-out imputed dosages for those directly genotyped variants. The official release of minimac3 performs leave-one-out hidden Markov model (HMM) calculation internally to calculate leave-one-out Rsq summary statistics, but does not output individual dosages (Fuchsberger et al., 2015; B. Howie et al., 2012). We modified minimac3 to include the individual leave-out-out dosages in the output VCF for the genotyped variants. Second, we simulated phenotypes based on the genotypes obtained by the chip array. Finally, we evaluated power of the two approaches by performing association tests between the simulated phenotypes and the imputed dosages based on either “best Rsq” or “best p-value” approaches.

The details of simulation follow the steps described below:

1. Select the non-centrality parameter corresponding to the association test p-value p_t. We calculate the non-centrality parameter Nr² as a chi-square statistics corresponding to the upper-tail probability p_t, where N is the total number of study subjects. This ensures that the median p-value is p_t when the true phenotypic variance explained by the genotype is r².

2. For each variant, we randomly draw ε from the normal distribution with mean 0 and standard deviation $\sqrt{1-r^2}$. We calculate the effect size β as $\sqrt{r^2/2f(1-f)}$, where f is the minor allele frequency (MAF) estimated using the chip genotypes of the variant. The phenotype value y is then calculated as Gβ + ε, where the chip genotypes G is 0, 1, or 2. The phenotypic variance explained by G and ε will be r² and 1−r², respectively.

3. We perform the linear regression using the leave-one-variant-out dosages for this variant, which were imputed using the three different reference panels respectively, and the phenotype y.

4. For the “best p-value” approach, the final association p value equals to most significant one among the three p values associated with the three different versions of imputed dosages. With the “best Rsq” approach, the final p value equals to the one corresponding to the reference panel with the highest imputation quality (ImpRsq), an estimated value for the correlation between imputed genotypes and true, unobserved genotypes.

5. The power to detect association signals equals to the percentage of final p values exceeding the genome-wide significance threshold determined for each approach by the permutation tests described above.

We performed linkage disequilibrium(LD) based variant pruning for the 289,376 directly genotyped variants that were found by all three reference panels using PLINK(Purcell et al., 2007) and obtained 132,183 variants with LD r²< 0.2 among each other. Then we randomly selected 3,000 variants for each of the MAF categories: MAF ≤ 0.001, MAF > 0.001 and ≤ 0.01, MAF > 0.1 and ≤ 0.05, and MAF > 0.05. We applied ImpRsq > 0.3, 0.5 and 0.8 to remove poorly imputed genotypes and variants that were successfully imputed from at least two references were used for this simulation study. All 5 steps above were repeated given different pt’s ranging from 5×10⁻⁸ to 1×10⁻¹³. Additionally, the entire process was repeated 5 times across the selected variants to average power.

Partial correlation estimation

To quantify the net gain of imputation accuracy obtained by including another reference panel on top of an existing panel, we estimated the partial correlation between the leave-one-out imputed dosages from the additional panel and the chip genotypes, conditioned on the leave-one-out imputed dosages from the existing panel. The correlation has been estimated for every pair of reference panels among the three on each of the 289,376 genotyped variants that were found in all three panels. For example, to estimate the net gain of including 1000G panel on top of HUNT panel (PartialRsq [1000G,Chip | HUNT]), we first obtained the leave-one-out dosages based on 1000G and HUNT WGS (details described in the Power estimation subsection). Secondly, for each variant, we performed three linear regressions on the chip genotypes: the first one has the imputed dosages from 1000G and HUNT WGS as covariates (model 1), the second one has the imputed dosages from HUNT WGS only as a covariate (model 2), and the third one does not have any other covariate except for the intercept (model 3). Lastly, we obtained sum of squared residuals (SSR) for the three linear regressions and calculated the partial correlation (partial Rsq) as $\frac{{\mathit{\text{SSR}}}_{\mathit{\text{model}}2}-{\mathit{\text{SSR}}}_{\mathit{\text{model}}1}}{{\mathit{\text{SSR}}}_{\mathit{\text{model}}3}}$. In a similar notation, the EmpRsq is equivalent to $\frac{{\mathit{\text{SSR}}}_{\mathit{\text{model}}3}-{\mathit{\text{SSR}}}_{\mathit{\text{model}}2}}{{\mathit{\text{SSR}}}_{\mathit{\text{model}}3}}$, and their sum should be equivalent to the proportion of explained variance by both sets of imputed dosages. Our intuition is that the more extra information the additional reference panel provides, the higher the partial correlation will be.

Results

Evaluating successfully imputed variants using different reference panels

In total, ~23.8 million variants were successfully imputed using minimac 3(Fuchsberger et al., 2015; B. Howie et al., 2012) from at least one of the three reference panels and exceeded the threshold of estimated imputation quality (ImpRsq) ≥ 0.3 (Figure 1). The three reference panels yielded roughly equal number of SNPs with MAF more than 1%, but the 1000G uncovered more unique variants; approximately 75.3% (1,068,228 out of 1,418,417) that were uniquely imputed from 1000G are indels or structural variants, a category of variation that is not available in the other two reference panels. We observed that imputation from the HRC panel resulted in more extremely rare variants (MAF less than 0.05%) than from HUNT WGS and 1000G. Imputation from the HUNT WGS panel uncovered more variants with MAF between 0.05% and 1% than the other two reference panels (Table 3). Approximately 3.6 million variants were uniquely imputed by the HUNT WGS panel (Figure 1) and the majority of them have MAF less than or equal to 0.05%(Figure 2). A threshold ≥ 0.3 for ImpRsq was applied as recommended to remove most of poorly imputed variants while retaining the vast majority of well imputed SNPs(Y. Li et al., 2009). We observed that the average EmpRsq remained above 0.6 for all MAF categories from all three reference panels when the ImpRsq ≥ 0.3 threshold was applied (Figure S2).

Figure 1. Number of variants that are imputed by different reference panels.

The corresponding percentage is the variants number out of all 23.8 million variants that are successfully imputed by any of the three reference panels.

Table 3.

Numbers of imputed variances contributed by each reference panel categorized by MAF. The threshold ImpRsq > 0.3 was applied. Each reference panel contributed uniquely imputed variants.

	HRC Release 1 (39.2M SNPs, 32,488 samples including 1,203 HUNT samples)			1000G Phase3 v5 (81.2M markers, 2,504 samples)			HUNT 5× WGS (20.2M SNPs, 2,201 samples)
MAF	Number of Passed Variants	Percent of Passed Variants	Number of Uniquely Imputed Variants	Number of Passed Variants	Percent of Passed Variants	Number of Uniquely Imputed Variants	Number of Passed Variants	Percent of Passed Variants	Number of Uniquely imputed Variants
(0, 0.0005)	4,337,138	23.9%	3,009,729	567,481	2.4%	230,186	2,291,216	50.6%	1,570,259
(0.0005, 0.001)	1,339,096	91.1%	373,964	501,248	11.4%	176,252	1,668,837	94.4%	901,106
(0.001, 0.005)	2,964,988	97.5%	140,318	2,119,956	33.6%	475,376	3,917,801	98.0%	982,320
(0.005, 0.01)	1,125,181	99.2%	7,426	1,074,885	68.9%	126,616	1,279,200	98.6%	47,426
(0.01, 0.05)	2,314,490	99.6%	10,525	2,554,206	89.2%	295,991	2,538,140	99.1%	55,490
> 0.05	5,158,670	99.8%	10,692	6,547,887	98.1%	1,122,426	5,507,946	99.6%	44,866
Total	17,239,563	55.1%	3,552,654	13,365,663	29.5%	2,426,847	17,203,140	87.4%	3,601,467

The greatest number of the uniquely imputed variants among the three reference panels for variants in each MAF category is highlighted in red.

MAF: minor allele frequency; ImpRsq, imputation quality metric R²

Figure 2. Distribution of numbers of variants that are imputed from only one reference panel or from multiple reference panels in different MAF categories.

Variants that are imputed by 1000G only are categorized as SNPs and non-SNP variants, including indels, deletions, complex short substitutions and other structural variant classes. 1000G, 1000 Genomes Phase 3; WGS, whole-genome sequencing; HRC, Haplotype Reference Consortium; MAF, minor allele frequency

Comparing imputation accuracy from different reference panels

To compare the imputation accuracy across the three reference panels, we examined all 289,376 variants that were directly genotyped by the chip array and available in all three reference panels. “Leave-one-variant-out” imputation results were used for these directly genotyped variants, meaning that one-by-one, each genotyped variant was masked, imputed, and then compared to the directly genotyped calls. The EmpRsq was estimated for each genotyped variant from each panel, which is the squared Pearson correlation between the imputed allele dosages and the genotypes called by direct genotyping. Figure 3a compares the average EmpRsq for all genotyped variants categorized by MAF among different reference panels. The MAF is estimated using the genotypes called by the chip array. Imputation from HRC has higher imputation accuracy for rare variants with MAF < 0.5% than the other two reference panels, which is expected because the number of samples available in HRC is much larger than the other two panels and the imputation accuracy for extremely rare variants depends on the number of copies of alternate alleles(Roshyara & Scholz, 2015). What is unexpected is that for variants with MAF ≥ 0.5%, HRC and HUNT WGS panels show comparable imputation accuracy, even though the size of the HUNT WGS panel is 15 times smaller than HRC. Consistent to previous studies, this result demonstrated the value of whole-genome sequencing for ancestry matched samples as a reference panel for genotype imputation(Deelen et al., 2014; G. H. Huang & Tseng, 2014; J. Huang et al., 2015; Low-Kam et al., 2016; Okada et al., 2015; Pistis et al., 2015; Roshyara & Scholz, 2015; Walter et al., 2015). It is also noticed that imputation from 1000G has lower average ImpRsq than the other two reference panels (Figure 3b–d), which is consistent to the lower proportion of variants passing the various ImpRsq thresholds in 1000G observed in Figure S2.

Figure 3. HRC and HUNT WGS panels show comparable imputation quality.

a. comparing the mean empirical R² (y axis) reported by different reference panels for variants that are directly genotyped categorized by the MAF (x axis) without any ImpRsq threshold applied.

b. comparing the mean Imputation R² (y axis) reported by different reference panels for variants that are directly genotyped categorized by the MAF (x axis) without any ImpRsq threshold applied.

c. comparing the mean Imputation R² (y axis) reported by different reference panels for all imputed variants (ImpRsq > 0.3) by the MAF (x axis).

d. comparing the mean Imputation R² (y axis) reported by different reference panels for all imputed variants by the MAF (x axis) without any ImpRsq threshold applied.

1000G, 1000 Genomes Phase 3; WGS, whole-genome sequencing; HRC, Haplotype Reference Consortium; MAF, minor allele frequency; ImpRsq, imputation quality metric R²

To further evaluate the impact of the sample size of the HUNT WGS panel on the imputation accuracy, we have randomly drawn 500, 1000, and 1500 samples from the original HUNT reference panel for imputation. Figure S3 shows the comparison of the average EmpRsq for all genotyped variants categorized by MAF among the target samples, across all reference panels. As expected, increases in the sample size of the HUNT WGS reference panels resulted in higher imputation accuracy, particularly for less frequent variants with MAF < 0.5%. Interestingly, we observed that the HUNT WGS with 500 samples outperforms 1000G(Auton et al., 2015) for variants with MAF > 0.5%. These results are consistent with other studies with population specific reference panels(Mitt et al., 2017; Pistis et al., 2015). The subset of 1000 samples provides better imputation accuracy than 1000G(Auton et al., 2015) even for variants with MAF as low as 0.1% and comparable imputation accuracy to HRC(McCarthy et al., 2016) for variants with MAF > 0.5%.

We examined whether our evaluation of imputation accuracy is biased in favor of HUNT WGS due to relatedness. Previous studies have shown that the relatedness between study samples and reference samples increases genotype imputation efficiency since related individuals tends to share longer haplotype stretches than unrelated ones(G. H. Huang & Tseng, 2014). To avoid the bias of imputation accuracy due to the relatedness between our study samples and the samples in the HUNT WGS reference panel, we excluded 1,644 study samples who are up to 2nd degree relatives of HUNT WGS samples. Relatedness was based on the estimation of the proportion of IBD by PLINK(Purcell et al., 2007). We observed that excluding these study samples did not affect the imputation accuracy except causing a slight decrease of the imputation accuracy for those very rare variants with MAF < 0.05% (Figure S4).

Evaluating two possible association test strategies to use multiple sets of imputed genotypes

As Figure 1 shows, approximately 60% of all successfully imputed variants were imputed from more than one reference panel, which makes it unclear how to perform downstream association tests. We compared two possible strategies: the “best p-value” and the “best Rsq” approaches. The “best p-value” approach uses each version of imputed genotypes to choose the lowest association p-value, thereby increasing the burden of adjusting for multiple hypothesis testing. The “best Rsq” approach selects the imputed variant with the highest estimated imputation quality ImpRsq, which is expected to be a reasonable approximation of the association between imputed and true genotypes, especially for common variants (Figure S5). We have compared the power of the two approaches to detect association signals accounting for the fact that the “best p-value” approach needs adjusting for the additional variants tested. To determine the significant thresholds for association tests with a family-wise error rate (FWER) 0.05, we estimated the number of independent tests using 1,000 permutations. For the “best Rsq” approach, where fewer ‘variants’ are analyzed, the significance threshold is 4.69×10⁻⁹ (2.10×10⁻⁹ with a Bonferroni correction) and for the best p-value approach, it is 2.53×10⁻⁹ (1.05×10⁻⁹ with a Bonferroni correction).

Using the permutation-derived significance thresholds above, we evaluated the power of the two approaches for association tests with quantitative traits through a simulation study (details described in methods). Our results indicated that the “best p-value” approach has more power to detect association signals than the “best Rsq” approach, particularly for rare variants with MAF < 1%, no matter how stringent the ImpRsq threshold was used for filtering out the poorly imputed genotypes (Figure 4, Figure S6 and Table S1). This is probably because the estimated imputation quality ImpRsq does not always agree with empirical imputation quality EmpRsq especially for rare variants (Figure S5), resulting in loss of variants with highest empirical imputation quality when selecting the “best Rsq” strategy. In addition, the distributions of the ImpRsq are quite different from different panels. Notably, from 1000G(Auton et al., 2015), the ImpRsq and EmpRsq were substantially lower for low-frequency variants (0.5% < MAF < 5%), and ImpRsq tends to underestimate EmpRsq(Figure S5). The two approaches have comparable association power for variants with MAF ≥ 1%, where estimated and empirical imputation qualities highly agree with each other (Figure S5). Our observation suggests that the inaccurate prediction of imputation quality have a higher impact than increased burden of multiple testing in association test with rare variants.

Figure 4. Comparison of power to detect true associations between best p-value and best Rsq approaches via simulation studies.

For each MAF category, 3,000 directly genotyped variants were randomly selected based on their MAF estimated with genotypes obtained from the chip array to estimate the power. The power is calculated as the proportion of significantly associated variants across three imputed panels based on each strategy given the corresponding significance threshold. ImpRsq > 0.3 was applied to remove poorly imputed genotypes. The numbers of variants that were successfully imputed from at least two reference panels and used in the simulation studies are: 2,513 with MAF > 0 and ≤ 0.001; 2,989 with MAF > 0.001 and ≤ 0.01; 3,000 with MAF > 0.01 and ≤ 0.05; and 3,000 with MAF > 0.05. MAF, minor allele frequency; ImpRsq, imputation quality metric R²

Evaluating net gain of imputation accuracy by including an additional reference panel

Finally, we quantified the net gain of imputation accuracy by including an additional reference panel as a “partial Rsq” conditioned on the imputed genotypes from an existing reference panel (See Materials and Methods for details). Intuitively, this represents the difference between the “optimal EmpRsq” linearly combined between two sets of imputed genotypes and the EmpRsq from the original imputed genotypes. 289,376 genotyped variants that were found in all three panels were used to evaluate the additional information that were gained from one reference panel given imputed dosages based on another panel. As Figure S7 presents, each reference panel is able to provide additional information to improve imputation accuracy. However, relatively less information could be be gained by including 1000G(Auton et al., 2015) panel on top of HRC across all MAF categories. This is expected since 1000G samples are included in the HRC panel, with the caveat that only single nucleotide variants with minor allele count ≥ 5 were retained. Note that evaluation of indels and structural variants absent in HRC were not included in this experiment. In contrast, given the imputed dosages from 1000G, both HUNT WGS and HRC provide substantial net gain of imputation accuracy, which is consistent to our observations. Furthermore, HUNT WGS and HRC provide additional information conditional on each other. More specifically, more extra information was obtained from HRC given HUNT WGS than those were obtained from HUNT WGS given HRC for these genotyped variants, which is also consistent to our observations in Figure 3.

Discussion

Many studies have performed whole genome sequencing of a subset of samples followed by imputation into samples with GWAS data(Holm et al.; Lane, Vlasac, & Anderson, 2016; Nalls et al., 2014; van Leeuwen et al., 2016). However, the trade-offs between the panel size, imputable variant types, and population specificity across different reference panels make it challenging to decide on the optimal strategy for imputation and downstream association analysis. We evaluated methods for genotype imputation when different reference panels are available. Our findings have demonstrated the benefits of uncovering novel variants with low frequency by using population-specific reference panels as has been reported by previous studies(J. Huang et al., 2015). Since the population-specific HUNT panel shared 1,023 samples with HRC(McCarthy et al., 2016), we expect to see an even bigger advantage in the number of novel low frequency variants imputed by the population-specific panel if there were no overlap between the two reference panels.

We have also observed that large-scale publicly available reference panels, as exemplified by HRC (McCarthy et al., 2016) and 1000G(Auton et al., 2015), contribute a large number of variants that are not captured by population-specific reference panels. More specifically, HRC(McCarthy et al., 2016), which has much larger sample size and contains more general European populations, contributes 3.5 million variants that could not be imputed by the other two panels. Since 1000G(Auton et al., 2015) has additional advantages that indels and structural variants are comprehensively detected and genotyped, 1.3 million non-SNP variants have only been imputed by 1000G(Auton et al., 2015). Furthermore, each reference panel may provide additional information to improve imputation accuracy. Therefore, to increase the variant coverage and imputation accuracy as much as possible, we recommend using all three reference panels for imputation if available. If a single panel has to be chosen, each option will have different advantages and disadvantages. We have shown that imputation from population-specific reference panels provides comparable imputation accuracy for variants with MAF > 0.1%. as using reference panels with 15 times larger sample size with only broad ancestry-matching (i.e. European). Although panel sizes are similar, the population-specific reference panel results in higher imputation accuracy than the mixed-ancestry 1000G panel (Auton et al., 2015) for variants with MAF ≥ 0.05%. This has also been observed by a recently published study on Estonians(Mitt et al., 2017).

To address the issue of imputing different versions of the same variant from different reference panels, we propose the “best p-value” approach, which analyzes all versions of each imputed variant and accounts for the multiple testing. Our simulation study demonstrated that this approach has higher power for detecting association signals than selecting the imputed variant with highest imputation quality given the distributions of the imputation quality metrics from different reference panels may be quite different, even adjusting for additional variants tested.

The UK10K study and the Genome of the Netherlands (GoNL) Consortium suggested that merging multiple reference panels to a larger reference panel would improve imputation performance, especially for less frequent variants(Deelen et al., 2014; J. Huang et al., 2015). Compared to this approach, our “best p-value” approach does not require access to all reference panes and is feasible even if not all reference panel haplotypes are directly accessible. If large imputation reference panels, such as the HRC(McCarthy et al., 2016), are not directly accessible, conducting association tests for all imputed versions of genotype with slightly higher computational cost will be an effective strategy.

In summary, we recommend creating a small size ancestry-matched reference panel using whole genome sequencing to allow for improved imputation of low frequency variants that may be enriched in that ancestral group, performing genotype imputation using the ancestry-matched reference panel and other large publicly available databases, and analyzing all versions of imputed variants in downstream association testing.