Leveraging functional annotations in genetic risk prediction for human complex diseases

Abstract

Genetic risk prediction is an important goal in human genetics research and precision medicine. Accurate prediction models will have great impacts on both disease prevention and early treatment strategies. Despite the identification of thousands of disease-associated genetic variants through genome wide association studies (GWAS), genetic risk prediction accuracy remains moderate for most diseases, which is largely due to the challenges in both identifying all the functionally relevant variants and accurately estimating their effect sizes in the presence of linkage disequilibrium. In this paper, we introduce AnnoPred, a principled framework that leverages diverse types of genomic and epigenomic functional annotations in genetic risk prediction for complex diseases. AnnoPred is trained using GWAS summary statistics in a Bayesian framework in which we explicitly model various functional annotations and allow for linkage disequilibrium estimated from reference genotype data. Compared with state-of-the-art risk prediction methods, AnnoPred achieves consistently improved prediction accuracy in both extensive simulations and real data.

Author summary

Genetic risk prediction plays a significant role in precision medicine. Accurate prediction models could have great impact on disease prevention and early treatment strategies. For example, mutations in BRCA1 and BRCA2 have been used to evaluate women’s breast cancer risk and as a guideline for early screening. However, genetic risk prediction models also present important challenges, including extreme high-dimensionality, limited access to and efficient computational methods for individual-level genotype data. To make use of rich GWAS summary statistics, we propose a novel method to address these challenges by integrating genomic functional annotations, which have been successfully applied in GWAS to generate biological insights. We demonstrate the improvement in accuracy in both extensive simulation studies and real data analysis of breast cancer, Crohn’s disease, celiac disease, rheumatoid arthritis and type-II diabetes.

This is a PLOS Computational Biology Methods paper.

Introduction

Achieving accurate disease risk prediction using genetic information is a major goal in human genetics research and precision medicine. Accurate prediction models will have great impacts on disease prevention and early treatment strategies [1]. Advancements in high-throughput genotyping technologies and imputation techniques have greatly accelerated discoveries in genome-wide association studies (GWAS) [2]. Various approaches that utilize genome-wide data in genetic risk prediction have been proposed, including machine-learning models trained on individual-level genotype and phenotype data [3–8], and polygenic risk scores (PRS) estimated using GWAS summary statistics [9, 10]. Despite the potential information loss in summary data, PRS-based approaches have been widely adopted in practice since the summary statistics for large-scale association studies are often easily accessible [11, 12] while individual-level data are more difficult to acquire, deposit, and process. However, prediction accuracies for most complex diseases remain moderate, which is largely due to the challenges in both identifying all the functionally relevant variants and accurately estimating their effect sizes in the presence of linkage disequilibrium (LD) [13].

Explicit modeling and incorporation of external information, e.g. pleiotropy [7, 8] and LD [10], has been shown to effectively improve risk prediction accuracy. Recent advancements in integrative genomic functional annotation, coupled with the rich collection of summary statistics from GWAS, have enabled increase of statistical power in several different settings [14–16]. To our knowledge, the impact of functional annotations on performance of genetic risk prediction has not been systematically studied. Here, we introduce AnnoPred (available at https://github.com/yiminghu/AnnoPred), a principled framework that integrates GWAS summary statistics with various types of annotation data to improve risk prediction accuracy. We compare AnnoPred with state-of-the-art PRS-based approaches and demonstrate its consistent improvement in risk prediction performance using both simulations and real data of multiple complex diseases.

AnnoPred risk prediction framework has three main stages (Methods). First, we estimate GWAS signal enrichment in 61 different annotation categories, including functional genome predicted by GenoCanyon scores [17], GenoSkyline tissue-specific functionality scores of 7 tissue types [14], and 53 baseline annotations for diverse genomic features [18] for each trait analyzed. Second, we propose an empirical prior of SNP effect size based on annotation assignment and signal enrichment. In general, SNPs located in annotation categories that are highly enriched for GWAS signals receive a higher effect size prior. Finally, the empirical prior is adopted in a Bayesian framework in which marginal summary statistics and LD matrix estimated from a reference panel are jointly modeled to infer the posterior effect size of each SNP. AnnoPred PRS is defined by

$$PRS=\sum _{j=1}^M{X}_j{E}_A\left({\beta }_j\right|\widehat{\beta },\widehat{D})$$

where X_j and β_j are the standardized genotype and effect size of the j^th SNP, respectively, $\widehat{\beta }$ is the marginal estimate of β, $\widehat{D}$ is the sample LD matrix, and $E_A\left({\beta }_j\right|\widehat{\beta },\widehat{D})$ denotes the posterior expectation of effect sizes under an empirical prior based on annotation assignment for all SNPs when adjusting for LD matrix estimated from a reference panel (Methods).

Results

We first performed simulations to demonstrate AnnoPred’s ability to improve risk prediction accuracy. We compared AnnoPred with four popular PRS approaches (Methods), including PRS based on genome-wide significant SNPs (PRS_sig), PRS based on all SNPs in the dataset (PRS_all), PRS based on tuned cutoffs for p-values and LD pruning (PRS_P+T), and recently proposed LDpred [10]. Mean correlations between simulated and predicted traits were calculated from 100 replicates under different simulation settings (Methods). AnnoPred showed the best prediction performance in all settings when the causal SNPs are highly enriched in annotated regions (Table 1, S2 Table and S2 Fig). In general, performance of PRS_sig, PRS_P+T, LDpred, and AnnoPred all improved under a sparser genetic model and higher trait heritability. PRS_all showed comparable performance between sparse and polygenic models but its prediction accuracy was consistently worse than other methods. Sample size in the training set was also crucial for risk prediction accuracy. Increasing sample size could lead to continuous improvement in prediction accuracy under different settings (Fig 1).

Table 1. Mean correlation between simulated and predicted traits calculated from 100 replicates under different simulation settings.

The highest mean correlations are highlighted in boldface. Standard deviations are shown in parentheses. Traits were simulated from WTCCC genotype data, which contain 15,918 individuals genotyped for 393,273 SNPs. In each setting, we used 70% of the data to calculate the training summary statistics and randomly divided the rest 30% into two parts for parameter tuning.

Training samples	Heritability	#Causal	PRSsig	PRSall	PRSP+T	LDpred	AnnoPred
Half (~5K)	0.25	300	0.149(.028)	0.08(.021)	0.25(.028)	0.279(.025)	0.286(.024)
		3000	NA*	0.082(.016)	0.073(.020)	0.087(.019)	0.096(.020)
	0.5	300	0.304(.04)	0.16(.022)	0.48(.026)	0.502(.033)	0.512(.026)
		3000	NA*	0.157(.019)	0.157(.024)	0.195(.021)	0.209(.019)
Full (~10K)	0.25	300	0.217(.031)	0.11(.02)	0.332(.023)	0.35(.033)	0.358(.022)
		3000	NA*	0.11(.014)	0.107(.018)	0.136(.017)	0.145(.017)
	0.5	300	0.373(.036)	0.213(.023)	0.548(.024)	0.557(.047)	0.566(.034)
		3000	0.078(.023)	0.21(.019)	0.243(.021)	0.309(.021)	0.324(.019)

* NA means no SNP achieves genome-wide significance level (5e-8).

Fig 1. Evaluating the effect of sample size on prediction accuracy in simulation.

Traits were simulated using SNPs of chromosome 1, chromosome 1 and 2, chromosome 1 to 4 and the whole genome while keeping the same proportion of causal variants and heritability to mimic the situation of increasing sample size. In the figure, logNr = $\log N\frac{M}{M_s}$, where N is the number of individuals, M is the total number of variants and M_s is the number of variants used in simulation. In total four settings were simulated for each effective sample size: h² = 0.25, p = 0.001; h² = 0.25, p = 0.01; h² = 0.5, p = 0.001; h² = 0.5, p = 0.01, where p represents the proportion of causal variants. Each dot represent the mean COR of 50 replicates in one simulation setting and error bar represents the standard error.

To illustrate the improved risk prediction performance in real data, we applied AnnoPred to five human complex diseases—Crohn’s disease (CD), breast cancer (BC), rheumatoid arthritis (RA), type-II diabetes (T2D), and celiac disease (CEL). We first estimated GWAS signal enrichment in different annotation categories (Methods). Enrichment pattern varies greatly across diseases (Fig 2A; S1 Table), reflecting the genetic basis of these complex phenotypes. Functional genome predicted by GenoCanyon was consistently and significantly enriched for all five diseases. Blood was strongly enriched for three immune diseases, namely CD (P = 8.9×10⁻¹²), CEL (P = 7.0×10⁻¹⁵), and RA (P = 9.9×10⁻⁶), while gastrointestinal (GI) tract was enriched in CD (P = 2.6×10⁻⁵) and CEL (P = 1.4×10⁻⁴), both of which have a known GI component. For BC, epithelium (P = 7.4×10⁻⁴), GI (P = 5.9×10⁻³), and muscle (P = 6.1×10⁻³) were significantly enriched. A few studies have shown that breast cancer could arise from epithelial cells [19, 20]. The connections between breast cancer and muscle as well as GI tract have also been previously suggested [21, 22]. In addition, studies have suggested that GI can be used as diagnostic and treatment target for type-II diabetes, Crohn’s disease, and celiac disease [23–25]. Furthermore, the connection between immune system and Crohn’s disease, celiac disease and rheumatoid arthritis have been extensively studied in literature [26–28]. Next, we evaluated the effectiveness of proposed empirical effect size prior in three diseases (i.e. CD, CEL, and RA) with well-powered testing cohorts (N>2,000). Interestingly, despite the highly variable enrichment results in training datasets, integrative effect size prior could effectively identify SNPs with large effect sizes and consistent effect directions in independent validation cohorts (Fig 2B and 2C).

Fig 2. Evaluating effectiveness of annotations and empirical effect size prior.

(A) GWAS signal enrichment across GenoCanyon and tissue-specific GenoSkyline annotations. The horizontal lines mark p-value cutoffs of 0.05 and Bonferroni corrected significance level. (B) Comparing signal strength of SNPs with high priors and low priors in independent validation cohorts. SNPs with higher priors have significantly stronger associations across three independent and well-powered testing datasets (N>2,000). P-values were calculated using one-sided Kolmogorov-Smirnov test. (C) Comparing consistency of SNPs’ effect direction between training and testing datasets. Each bar quantifies the proportion of SNPs with consistent effect directions. P-values were calculated using one-sided two-sample binomial test.

Correlations between the calculated PRS and disease status (COR) for different approaches are summarized in Table 2. AnnoPred showed consistently improved prediction accuracy compared with all other methods across five diseases. Notably, PRS_sig and PRS_all showed suboptimal performance in these datasets, reaffirming the importance of modeling LD and other external information. A likelihood ratio test was used to test for the difference in the prediction accuracy between models comparing the likelihood of a logistic regression fitting PRS of one method to that of a logistic regression fitting PRS of two methods jointly (S11 Table). From the test, AnnoPred with 61 annotations performed significantly better than LDpred (p = 1.2E-22 for CD, p = 0.045 for BC, p = 4.2E-7 for RA, p = 3.3E-4 for T2D and p = 1.3E-3 for CEL). Reversing the order of test (that is, comparing the likelihood of model using annotations with model using and not using annotations jointly) results in non-significant p-values for most tests (S11 Table), which further demonstrates that PRS incorporating functional annotations mostly encompasses the information of PRS without annotations. To test different methods’ ability to stratify individuals with high risk, we compared the proportion of cases among testing samples with high PRS. AnnoPred outperformed all other methods in CD, CEL, RA, and T2D (S1 Fig). Next, we tested AnnoPred’s performance using only the 53 baseline annotations and observed a substantial drop in prediction accuracy for all diseases (S3 Table). AnnoPred with GenoCanyon and GenoSkyline annotations only (nine annotation tracks in total) yields better performance than the 53 baseline annotations (S10 Table). For CD and T2D, by using these 9 categories AnnoPred even achieved higher accuracy than the model with all 61 annotation tracks added. These results highlight the importance of annotation quality in genetic risk prediction, and also demonstrate GenoCanyon and GenoSkyline’s ability to accurately identify functionality in the human genome. Since different diseases have various enrichment patterns, we also run AnnoPred with significantly enriched annotations (enrichment test p value less than 0.05) for each disease (S10 Table). In general, using only the significantly enriched annotations indeed improved the performance in most diseases.

Table 2. CORs of different methods.

The highest CORs are highlighted in boldface.

Disease/Trait	PRSsig	PRSall	PRSP+T	LDpred	AnnoPred
Crohn's Disease	0.27	0.229	0.32	0.325	0.343
Breast Cancer	0.084	0.055	0.12	0.122	0.137
Rheumatoid Arthritis	0.204	0.114	0.248	0.282	0.287
Type-II Diabetes	0.165	0.156	0.204	0.202	0.22
Celiac Disease	0.11	0.136	0.18	0.197	0.213

Tissue specificity plays an important role in genetic risk prediction. Integrating more functional annotations with higher tissue and cell type specificity may further increase risk prediction accuracy, especially when the tissue type that is biologically relevant to the disease is not well characterized by the seven available tissue tracks in our current analyses. To explore how these factors will affect the AnnoPred model, we performed a few follow-up analyses. We have recently expanded our GenoSkyline annotations to more than 100 tissue and cell types from the Roadmap Epigenomics Project [29]. We investigated the performance of AnnoPred after integrating 66 annotation tracks representing a spectrum of adult tissue and cell types. As shown in S10 Table, incorporating more annotations into the model does not always further improve risk prediction accuracy compared with AnnoPred with fewer annotations in the model. This may be due to the overlap between functional regions (e.g. functional annotations for slightly different brain regions) when incorporating too many annotation tracks into the model, which will cause numerically unstable heritability estimates. This is because annotation-stratified LD score regression, the method we used to empirically estimate the informative prior for SNPs’ effect sizes, is a multiple linear regression model that regresses SNP-level summary statistics against annotation-stratified LD scores. When two functional annotation tracks are similar, the corresponding LD scores will also be correlated by definition. It is well understood that if multi-collinearity (i.e. correlation among covariates) in multiple regression leads to numerically unstable estimates for regression coefficients [30] (the heritability parameters in our case).

In order to study the effect of highly associated SNPs (e.g. SNPs in MHC regions for immune traits), we repeated the analysis on CD, RA, BC and T2D after removing the SNPs in MHC region (chr6: 28,477,797–33,448,354 bp). Re-analysis of CEL was unnecessary since the training summary statistics of CEL does not contain any SNP in the MHC region. After removing SNPs in MHC regions, the prediction accuracies for RA drops dramatically for all methods and AnnoPred remained to be the method with the best performance (S9 Table). For the rest diseases, results varied little from the original analysis. Besides COR, we also included AUCs for all the analysis performed (S2, S6, S9 and S10 Tables), all of which showed consistent patterns.

Due to distinct allele frequencies and LD structures across populations, risk prediction accuracy usually drops when the training and testing samples are from different populations. In order to investigate the robustness of AnnoPred against population heterogeneity, we applied AnnoPred to three non-European cohorts for breast cancer and type-II diabetes while training the model using summary statistics from European-based studies. The CORs and AUCs are summarized in S6 and S7 Tables. As expected, we observed a drop in prediction accuracy for all methods. However, AnnoPred still performed the best in all three trans-ethnic validation datasets.

Discussion

Our work demonstrates that functional annotations can effectively improve performance of genetic risk prediction. AnnoPred jointly analyzes diverse types of annotation data and GWAS summary statistics to upweight SNPs with a higher likelihood of functionality, which lead to consistently better prediction accuracy for multiple complex diseases. Our method is not without limitation. First, despite the consistent improvement compared with existing PRS-based methods, accuracies for most diseases remain moderate. In order to effectively stratify risk groups for clinical usage, our model remains to be further calibrated using large cohorts with measured environmental and clinical risk factors [1]. Second, accurate estimation of GWAS signal enrichment and SNP effect sizes requires a large sample size for the training dataset. This could potentially be improved by new estimators for annotation-stratified heritability [19]. A few Bayesian models combining GWAS summary statistics with functional annotations have been proposed for the purpose of fine-mapping functional variants [16, 20, 21]. Whether these models could be adapted to benefit risk prediction accuracy remains to be investigated in the future. Importantly, the rich collection of publicly available integrative annotation data, in conjunction with the increasing accessibility of GWAS summary statistics, makes AnnoPred a customizable and powerful tool. As GWAS sample size continues to grow, AnnoPred has the potential to achieve even better prediction accuracy and become widely adopted as a summary of genetic contribution in clinical applications of risk prediction.

Methods

Annotation data

GenoCanyon is a statistical framework to predict functional regions in the human genome through integrative analysis of ENCODE epigenomic data and multiple conservation metrics [17]. Later we have further extended the model and developed GenoSkyline, which aimed to predict tissue-specific functionality [14]. In the AnnoPred model, we incorporated GenoCanyon general functionality scores, GenoSkyline tissue-specific functionality scores for seven tissue types (brain, gastrointestinal tract, lung, heart, blood, muscle, and epithelium), and 53 LDSC baseline annotations that covered a variety of genomic features [18] (S1 Table). We smoothed GenoCanyon scores by a 10Kb window, a strategy previously shown to improve robustness of functionality prediction [22]. The smoothed GenoCanyon annotation and raw GenoSkyline annotations of seven tissue types were dichotomized based on a cutoff of 0.5. The regions with GenoCanyon or GenoSkyline scores greater than the cutoff were interpreted as non-tissue-specific or tissue-specific functional regions in the human genome. Such dichotomization has been previously shown to be robust against the cutoff choice [14]. Notably, the AnnoPred framework allows users to specify their own choice of annotations.

Heritability partition

We assume throughout the paper that both the phenotype Y_N×1 and the genotypes X_N×M are standardized with mean zero and variance one. We assume a linear model

$$Y_{N\times 1}=X_{N\times M}{\beta }_{M\times 1}+{\epsilon }_{N\times 1}$$

X, β and ε are mutually independent. We also assume that β is a random effect and effects of different SNPs are independent. A key idea in the AnnoPred framework is to utilize functional annotation information to accurately estimate SNPs’ effect sizes. In order to achieve that, we first partition trait heritability by annotations using LD score regression [18]. Since genotypes are standardized, per-SNP heritability is defined as the variance of β_i for the i^th SNP, and is used to quantify SNP effect sizes. More specifically, assume there are K + 1 pre-defined annotation categories, denoted as S₀, S₁, …, S_K with S₀ representing the entire genome. Under an additive assumption for heritability in overlapped annotations, we have ${\beta }_i\sim N(0,\sum _{j:i\in S_j}{\tau }_j)$, where τ₀, τ₁, …, τ_K, quantify the contribution to per-SNP heritability from each annotation category. Denote the estimated marginal effect size of the i^th SNP as ${\widehat{\beta }}_i=\frac{X_i^{T}Y}{N}$, then we have the following approximation

$$E\left(N{\widehat{\beta }}_i^2\right)\approx (N-1)\sum _k{\tau }_{k}l\left(i,k\right)+1$$

where l(i, k) is the annotation-stratified LD score and N denotes the total sample size. Regression coefficients τ_k are estimated through weighted least squares. The estimated heritability of the i^th SNP is then $\widehat{Var}\left({\beta }_i\right)=\sum _{j:i\in S_j}{\widehat{\tau }}_j$.

Empirical prior of effect size

Based on per-SNP heritability estimates, we propose two different priors for SNP effect sizes to add flexibility against different genetic architecture. For the first prior, we assume that SNP effect size follows a spike-and-slab distribution

$${\beta }_i\sim p_{0}N(0,\raisebox{1ex}{${\widehat{\sigma }}_i^2$}\!\left/ \!\raisebox{-1ex}{$p_0$}\right.)+(1-p_0){\delta }_0$$

where p₀ is the proportion of causal SNPs in the dataset, and δ₀ is a Dirac function representing a point mass at zero. The empirical variance of each SNP, i.e. ${\widehat{\sigma }}_i^2$, is determined by the annotation categories it falls in. More specifically, we assume ${\widehat{\sigma }}_i^2=c\left(\sum _{j:i\in S_j}{\widehat{\tau }}_j\right)$, where c is a constant calculated from the following equation

$${\sum }_i{\widehat{\sigma }}_i^2={\widehat{H}}^2.$$

We do not directly use $\sum _{j:i\in S_j}{\widehat{\tau }}_j$ as the empirical variance prior because it is estimated in the context where all SNPs in the 1000 Genomes Project database are included in the model [18]. Such per-SNP heritability estimates cannot be extrapolated to the risk prediction context where many fewer SNPs are analyzed [23]. Therefore, we rescale the heritability estimates to better quantify each SNP’s contribution toward chip heritability. Following [24], we use a summary statistics-based heritability estimator that approximates the Haseman-Elston estimator:

$${\widehat{H}}^2=\frac{({\overline{\chi }}^2-1)}{N\stackrel{-}{l}}$$

where ${\overline{\chi }}^2$ and $\overline{l}$ denote the mean $N{\widehat{\beta }}_i^2$ and mean non-stratified LD score, respectively.

In the first prior, we assumed the same proportion of causal SNPs but different effect sizes across annotation categories. We now describe the second prior that assumes different proportions of causal SNPs but the same effect size across annotation categories. To be specific, we assume the causal effect size to be Var(β_causal) = V, the total number of SNPs to be M_0, and the overall proportion of causal SNPs to be p_0. The total heritability $H_0^2$ can then be written as $H_0^2=p_0{M}_{0}V$. For the i^th SNP, use $T_i=\left(\bigcap _{j:i\in S_j}{S}_j\right)\cap \left(\bigcap _{k:i\notin S_k}{S}_k^c\right)$ to denote the collection of SNPs that share the same annotation assignment with the i^th SNP, and let $M_{T_i}=\left|T_i\right|$, i.e. the number of SNPs in the set. Then, the total heritability of SNPs in T_i is $H_{T_i}^2=p_{T_i}{M}_{T_i}V$, with $p_{T_i}$ denoting the proportion of causal SNPs in T_i. Following these notations, we have

$${\beta }_i\sim p_{T_i}N\left(0,V\right)+(1-p_{T_i}){\delta }_0$$

where $V=\frac{H_0}{p_0{N}_0}$ and $p_{T_i}=p_0\frac{M_0{H}_{T_i}^2}{M_{T_i}{H}_0^2}$. We use ${\widehat{H}}^2$ to estimate $H_0^2$, and the following formula to estimate $H_{T_i}^2$.

$${\widehat{H}}_{T_i}^2=\frac{{\sum }_{k\in T_i}\sum _{j:k\in S_j}{\widehat{\tau }}_j}{{\sum }_{k=1}^{M_0}\sum _{j:k\in S_j}{\widehat{\tau }}_j}{\widehat{H}}^2$$

Finally, p₀ is treated as a tuning parameter for both prior functions in our analysis.

Calculation of posterior effect sizes

By Bayes’ rule, the posterior distribution of β is:

$$f\left(\beta \right|\widehat{\beta },\widehat{D})\propto f(\widehat{\beta }\left|\beta ,\widehat{D}\right)f\left(\beta \right)$$

where $\widehat{D}=\frac{1}{N}{X}^{T}X$ is the sample correlation matrix and $\widehat{\beta }=\frac{1}{N}{X}^{T}Y$ is the marginal effect size estimates. Given β and $\widehat{D}$, $\widehat{\beta }$ follows a multivariate normal distribution asymptotically with the following mean and variance

$$E\left(\widehat{\beta }\right|\beta ,\widehat{D})=\frac{1}{N}[E(X^{T}X\beta |\beta ,\widehat{D})+E(X^T\epsilon |\beta ,\widehat{D})]=\widehat{D}\beta$$

$$Var\left(\widehat{\beta }\right|\beta ,\widehat{D})=Var(\frac{1}{N}{X}^T\epsilon |\beta ,\widehat{D})=\frac{1}{N}(1-h_g^2)\widehat{D}.$$

However, $\widehat{D}$ is usually non-invertible and has very high dimensions. We thus study the posterior distribution of a small chunk of $\widehat{\beta }$ instead. Let ${\widehat{\beta }}_b$ be the estimated marginal effect size of SNPs in a region b (e.g. a LD block) and the corresponding genotype matrix is X_b and sample correlation matrix is ${\widehat{D}}_b$. Then the conditional mean and variance of ${\widehat{\beta }}_b$ are

$$E\left({\widehat{\beta }}_b\right|{\beta }_b,{\widehat{D}}_b)=\frac{1}{N}[E(X_b^{T}X\beta |{\beta }_b,{\widehat{D}}_b)+E(X_b^T\epsilon |{\beta }_b,{\widehat{D}}_b)]={\widehat{D}}_b{\beta }_b$$

$$\begin{array}{c}Var({\widehat{\beta }}_b\text{|}{\beta }_b,{\widehat{D}}_b)=\frac{1}{N^2}var(X_b^T{X}_b{\beta }_b+X_b^T(X_{-b}{\beta }_{-b}+\epsilon )|{\beta }_b,{\widehat{D}}_b)\\ =\frac{1}{N^2}var(X_b^T(X_{-b}{\beta }_{-b}+\epsilon )|{\beta }_b,{\widehat{D}}_b)\\ =\frac{1}{N^2}{X}_b^{T}var(X_{-b}{\beta }_{-b}+\epsilon \text{|}{\beta }_b,{\widehat{D}}_b)X_b\\ =\frac{1}{N}(1-h_b^2){\widehat{D}}_b\end{array}$$

where $h_b^2=\sum _{i\in b}{\sigma }_i^2$ is the heritability of SNPs in region b, and X_−b and β_−b denote the genotype matrix and effect sizes of SNPs not in region b. The conditional distribution of β_b is:

$$\begin{array}{c}f\left({\beta }_b\text{|}{\widehat{\beta }}_b,{\widehat{D}}_b\right)\propto N\left({\widehat{D}}_b{\beta }_b,\frac{1}{N}\left(1-h_b^2\right){\widehat{D}}_b\right)\prod _{i\in b}f\left({\beta }_i\right)\\ \propto \{\begin{array}{l}N\left({\widehat{D}}_b{\beta }_b,\frac{1}{N}\left(1-h_b^2\right){\widehat{D}}_b\right)\prod _{i\in b}\left[p_{0}N\left(0,\raisebox{1ex}{${\sigma }_i^2$}\!\left/ \!\raisebox{-1ex}{$p_0$}\right.\right)+\left(1-p_0\right){\delta }_0\right],\mathrm{under}\mathrm{the}\mathrm{first}\mathrm{prior}\\ N\left({\widehat{D}}_b{\beta }_b,\frac{1}{N}\left(1-h_b^2\right){\widehat{D}}_b\right)\prod _{i\in b}\left[p_{T_i}N\left(0,V\right)+\left(1-p_{T_i}\right){\delta }_0\right],\mathrm{under}\mathrm{the}\mathrm{second}\mathrm{prior}\end{array}\end{array}$$

Although it is difficult to derive $E\left({\beta }_b|{\widehat{\beta }}_b,{\widehat{D}}_b\right)$ from the joint conditional distribution of β_b, each element of β_b follows a mixed normal distribution conditioning on ${\widehat{\beta }}_b$, ${\widehat{D}}_b$, and all other elements in β_b. Therefore, we apply a Gibbs sampler to draw samples from $f\left({\beta }_b\right|{\widehat{\beta }}_b,{\widehat{D}}_b)$ and use the sample mean as an approximation for $E\left({\beta }_b\right|{\widehat{\beta }}_b,{\widehat{D}}_b)$. We further performed a sensitivity analysis on the choice of the size of block b (S6 Fig). Specifically, we ran AnnoPred on the data of Crohn’s disease with different sizes of block and found that the results were robust to the sizes. In practice, the size of block b is specified by the total number of variants divided by 3,000.

Calculation of PRS

PRS is calculated using the following formula

$$PRS={\sum }_{j=1}^M{X}_j{E}_A\left({\beta }_j\right|\widehat{\beta },\widehat{D}),$$

where E_A denotes the posterior expectation as described above. In practice, the individual-level genotype matrix is not available and we use the LD matrix estimated from a reference panel or the validation samples to substitute $\widehat{D}$. We apply the same standard of choosing the size of b as described in [10]. Choices of prior and p₀ can be tuned in an independent cohort. For the data analysis described in this work, we adopted a cross-validation scheme to select tuning parameter due to the challenge in finding multiple independent cohorts without overlapping with the training GWAS summary statistics. The training datasets in our real data analyses and simulations are always fixed, i.e. GWAS summary statistics. We did not perform a classical cross-validation by using different subsets of the complete data to train and test our prediction model. The purpose of cross-validation in our study is purely parameter tuning. To select a suitable tuning parameter, we divide the independent testing dataset (individual level genotype and phenotype data) into two equal parts (A and B), and select the tuning parameters by optimizing prediction accuracy on dataset A. We then evaluate prediction accuracy using the remaining half of testing data, i.e. dataset B. Finally, we repeat the analysis one more time by choosing the tuning parameter on dataset B while evaluating the prediction accuracy on dataset A. Results from these two separate analyses are averaged to quantify model performance. For T2D where multiple independent cohorts are available (phs000237 and phs000388), we used an independent cohort for parameter tuning and the other for evaluating performance (S12 Table). The results are consistent with the cross-validation.

Comparison with existing methods

We compared AnnoPred with several commonly used risk prediction methods based on summary data of association studies. PRS_sig and PRS_all were both calculated as the inner product of marginal effect size estimates and the corresponding genotypes. PRS_all used all the SNPs that are shared between training and testing datasets while PRS_sig only used SNPs with p-values below 5 × 10⁻⁸ in the training set. PRS_P+T used SNPs passing both LD pruning and p-value thresholding. The thresholds are tuned in an independent dataset over a grid (0, 0.1, 0.2, … 0.9 for LD; 1, 0.3, 0.1, 0.03, 0.01, 3E-3, 1E-3, 3E-4, 1E-4, 3E-5, 1E-5, 1E-6, 1E-7, 5E-8, 1E-8 for p-value). LDpred can be viewed as a special case of AnnoPred, assuming the whole genome as the only functional annotation. This is because when enrichment is constant (i.e. causal variants are uniformly distributed across the genome), per-SNP heritability estimates would be nearly constant and therefore results in similar performance to LDpred. We have performed an additional simulation to demonstrate this using WTCCC genotype data with ~15K individuals and ~330K variants. We randomly divided the genome into two parts (two annotations) and uniformly selected causal SNPs. Then the traits were simulated in a similar way as other simulations in this paper. We estimated per-SNP heritability using LDSC in the two annotation categories, respectively. We ran the procedure for 100 times and the distributions of estimated per-SNP heritability in both regions are summarized in the figure below (the dashed line denotes the true per-SNP heritability, added as S4 Fig in the manuscript), which indicates that the per-SNP heritability estimates are uniform across the genome under constant enrichment. Therefore, AnnoPred would be mathematically equivalent with LDpred with enrichment is constant. We downloaded python code for PRS_P+T and LDpred from Github (https://github.com/bvilhjal/ldpred). All the tuning parameters were tuned through cross-validation as we did for AnnoPred. Besides all these PRSs, we also compared AnnoPred with a evaluating method used in [5], which uses 1E-1, 1E-2,…, 1E-5 as p-value threshold to select SNPs and report the accuracy for the best performed threshold (S4 and S5 Tables).

Given that many large-scale GWAS summary statistics have included almost all available cohorts for a disease of interest, it is challenging to find independent datasets with individual-level genotype and phenotype information and sufficient sample sizes. We were able to identify ideal validation datasets for the five diseases we analyzed in this paper. The performance of different methods on more traits shall be evaluated when we get access to more data in the future.

Simulation settings

We simulated traits from WTCCC genotype data, which contain 15,918 individuals genotyped for 393,273 SNPs after filtering variants with missing rate above 1% and individuals with genetic relatedness above 0.05. We first generated two annotations and each annotation was simulated by randomly selecting 10% of the genome, denoted as A₁ and A₂, which we assume are known when applying AnnoPred. Denote the heritability of the trait as $h_g^2$ (25% or 50%) and the number of causal variants as m (300 or 3,000). Causal variants were generated as follows: $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$ causal variants were selected from A₁, $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$ from A₂ and the rest from (A₁UA₂)^C corresponding to a high enrichment of signals in A₁ and A₂. Effect sizes of causal variants were sampled from $N(0,\frac{h_g^2}{m})$. For each simulation, we used 70% of the data to calculate the training summary statistics and randomly divided the rest 30% into two parts for parameter tuning. We also randomly selected half of the training data to calculate summary statistics in order to study the effect of sample size on prediction accuracy.

In order to evaluate the improvement in accuracy, we performed a permutation test to compare the CORs of AnnoPred and LDpred. Suppose the CORs of LDpred and AnnoPred in simulations are x₁, x₂, …, x_n and y₁, y₂, …, y_n, respectively. And the hypothesis we want to test is

$$H_0:{\mu }_x={\mu }_y\leftrightarrow H_1:{\mu }_x\ne {\mu }_y$$

where μ_x and μ_y represent the population mean of accuracies of LDpred and AnnoPred. We used $|\overline{x}-\overline{y}|$ as the test statistics and the p value can be calculated as $p=Pr\left(\left|\overline{x}-\overline{y}\right)>|{\overline{x}}_{obs}-{\overline{y}}_{obs}||H_0\right)$, in which $\overline{x}-\overline{y}$ represents the random variable and ${\overline{x}}_{obs}-{\overline{y}}_{obs}$ represents the actually observed values. We used permutation to approximate the distribution of $\left(\overline{x}-\overline{y}\right)$ when H₀ is true. Specifically, we first pooled x_i′s and y_i′s together. Then ${\tilde{x}}_1,{\tilde{x}}_2,\dots ,{\tilde{x}}_n$ and ${\tilde{y}}_1,{\tilde{y}}_2,\dots ,{\tilde{y}}_n$ were sampled from the pooled data for N = 10⁶ times and we calculated $\left(\overline{\tilde{x}}-\stackrel{-}{\tilde{y}}\right)$ for each ${\tilde{x}}_i\prime s$ and ${\tilde{y}}_i\prime s$ sampled, which formed the empirical distribution of $\left(\overline{x}-\overline{y}\right)$ under H₀. And the p value could be approximated by $\widehat{p}=\frac{\sum _{k=1}^{N}I\left\{\right|{\overline{\tilde{x}}}_k-{\overline{\tilde{y}}}_k|>|{\overline{x}}_{obs}-{\overline{y}}_{obs}\left|\right\}}{N}$, in which ${\overline{\tilde{x}}}_k-{\overline{\tilde{y}}}_k$ represents the sampled test statistic of the kth permutation.

To further study the effect of sample size on prediction performance, we simulated traits using SNPs of chromosome 1, chromosomes 1 and 2, chromosomes 1 to 4 and the whole genome while keeping the same proportion of causal variants and heritability to mimic the situation of increasing sample size. The corresponding relative sample sizes ($N\frac{M}{M_s}$, where N is the number of individuals, M is the total number of variants and M_s is the number of variants used in simulation) for the four scenarios are ~135K, ~67K, 37K and ~11K. For each effective sample size, we simulated traits under four settings: h² = 0.25, p = 0.001; h² = 0.25, p = 0.01; h² = 0.5, p = 0.001; h² = 0.5, p = 0.01, where p represents the proportion of causal variants (Fig 1).

Ethics statement

The study was approved by YALE UNIVERSITY HUMAN INVESTIGATION COMMITTEE with approval number 100 FR1 and 100 FR27.

Data access

GWAS summary statistics and validation data

We trained AnnoPred using publicly accessible GWAS summary statistics and evaluated risk prediction performance using individual-level genotype and phenotype data from cohorts independent from the training samples. Only SNPs shared between training and testing datasets were kept in our analyses. Details for each training and testing dataset are provided in S1 Text and S8 Table.

For Crohn’s disease, we trained the model using summary statistics from International Inflammatory Bowel Disease Genetics Consortium (IIBDGC; N_case = 6,333 and N_control = 15,056) [25]. Samples from the Wellcome Trust Case Control Consortium (WTCCC) were removed from the meta-analysis and used as the validation dataset (N_case = 1,689 and N_control = 2,891) [26]. For breast cancer, we trained the model using summary statistics from Genetic Associations and Mechanisms in Oncology (GAME-ON) study (N_case = 16,003 and N_control = 41,335) [27], and tested the performance using samples from the Cancer Genetic Markers of Susceptibility (CGEMS) study (N_case = 966 and N_control = 70) [28]. Shared samples between CGEMS and GAME-ON were removed. We used samples from the CIDR-GWAS of breast cancer for trans-ethnic analysis (N_case = 1,666 and N_control = 2,038) [29]. For rheumatoid arthritis, we used summary statistics from a meta-analysis with 5,539 cases and 20,169 controls to train the model [30]. WTCCC samples were removed from the meta-analysis and used for validation (N_case = 1,829 and N_control = 2,892) [26]. For type-II diabetes, the training dataset is Diabetes Genetics Replication and Meta-analysis (DIAGRAM) consortium GWAS with 12,171 cases and 56,862 controls [31]. We used samples from Northwestern NUgene Project for validation (N_case = 662 and N_control = 517) [32]. Samples from Institute for Personalized Medicine (IPM) eMERGE project are used for trans-ethnic analysis (African American: N_case = 517 and N_control = 213; Hispanic: N_case = 477 and N_control = 102) [33]. The training dataset for celiac disease is from a GWAS with 4,533 cases and 10,750 controls [34]. Samples in the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) celiac disease study were used for validation (N_case = 1,716 and N_control = 530) [35].

Software availability

AnnoPred software and source code are freely available online at https://github.com/yiminghu/AnnoPred.

Supporting information

S1 Fig. Enrichment of proportion of cases in the top 5% testing samples with high PRS.

(TIFF)

S2 Fig. Boxplots of the simulation results in Table 1, p-values of the permutation tests (Methods) quantify the improvement of AnnoPred over PRS without incorporating functional annotations.

(TIFF)

S3 Fig. Heritability enrichment across GenoCanyon and tissue-specific GenoSkyline annotations.

The horizontal line marks no enrichment.

(TIFF)

S4 Fig. Per-SNP heritability estimation under constant enrichment in simulation.

Dashed line marks the true per-SNP heritability.

(TIFF)

S5 Fig. Proportion of SNPs in GenoCanyon and tissue-specific GenoSkyline annotations.

(TIFF)

S6 Fig. Prediction accuracy of AnnoPred on Crohn’s disease data using different LD radiuses.

(TIFF)

S7 Fig. Comparing signal strength of SNPs with high priors and low priors in independent validation cohorts with underpowered sample size (<2000).

(A) Breast cancer (B) Type-II diabetes (C) Comparing consistency of SNPs’ effect direction between training and testing datasets. Each bar quantifies the proportion of SNPs with consistent effect directions. The association tests and effect size estimation on the testing data are underpowered due to the limited sample size.

(TIFF)

S1 Table. GWAS signal enrichment across 61 annotation categories.

(XLSX)

S2 Table. AUCs of different methods.

The highest AUCs are highlighted in boldface.

(XLSX)

S3 Table. Comparison of the complete model and AnnoPred with baseline annotations.

The highest AUCs are highlighted in boldface.

(XLSX)

S4 Table. Comparison of the AnnoPred with method used in (Speed and Balding 2014) for evaluation in real data analysis.

The highest AUCs are highlighted in boldface.

(XLSX)

S5 Table. Comparison of the AnnoPred with method used in (Speed and Balding 2014) for evaluation in simulation.

The highest correlations are highlighted in boldface.

(XLSX)

S6 Table. AUCs for trans-ethnic analyses.

The highest AUCs are highlighted in boldface.

(XLSX)

S7 Table. CORs for trans-ethnic analyses.

The highest CORs are highlighted in boldface.

(XLSX)

S8 Table. URLs for training and testing datasets.

(XLSX)

S9 Table. Prediction accuracies after removing SNPs in MHC regions.

The highest CORs/AUCs are highlighted in boldface.

(XLSX)

S10 Table. Prediction accuracies of AnnoPred when different annotations used.

The highest CORs/AUCs are highlighted in boldface.

(XLSX)

S11 Table. p-values from the likelihood ratio tests comparing different models.

(XLSX)

S12 Table. Prediction accuracies on T2D when tuning the parameter in an independent cohort.

(XLSX)

S1 Text. Details on GWAS summary statistics and validation data.

(DOCX)

Data Availability

All the GWAS summary statistics are available online and can be accessed through: http://www.ibdgenetics.org, http://gameon.dfci.harvard.edu, http://www.broadinstitute.org/ftp/pub/rheumatoid_arthritis/Stahl_etal_2010NG/, http://diagram-consortium.org/downloads.html and https://www.immunobase.org/downloads/protected_data/GWAS_Data/. Individual level genotype data are available from dbGaP (accession numbers: phs000147, phs000383, phs000237 and phs000274) and WTCCC on EGA https://www.ebi.ac.uk/ega/ (accession numbers: EGAD00000000001, EGAD00000000002, EGAD00000000007 and EGAD00001000401).

Funding Statement

This study was supported in part by the National Institutes of Health (https://www.nih.gov/) grants R01 GM59507, the VA Cooperative Studies Program of the Department of Veterans Affairs, Office of Research and Development (http://www.research.va.gov/programs/csp/), and the Yale World Scholars Program (http://bbs.yale.edu/training/initiatives/csc.aspx) sponsored by the China Scholarship Council. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.