A principal component approach to improve association testing with polygenic risk scores

Abstract

Polygenic risk scores (PRSs) have become an increasingly popular approach for demonstrating polygenic influences on complex traits and for establishing common polygenic signals between different traits. PRSs are typically constructed using pruning and thresholding (P+T), but the best choice of parameters is uncertain; thus multiple settings are used and the best is chosen. Optimization can lead to inflated type I error. Permutation procedures can correct this, but they can be computationally intensive. Alternatively, a single parameter setting can be chosen a priori for the PRS, but choosing suboptimal settings results in loss of power. We propose computing PRSs under a range of parameter settings, performing principal component analysis (PCA) on the resulting set of PRSs, and using the first PRS-PC in association tests. The first PC reweights the variants included in the PRS to achieve maximum variation over all PRS settings used. Using simulations and a real data application to study PRS association with bipolar disorder and psychosis in bipolar disorder, we compare the performance of the proposed PRS-PCA approach with a permutation test and an a priori selected p-value threshold. The PRS-PCA approach is simple to implement, outperforms the other strategies in most scenarios, and provides an unbiased estimate of prediction performance.

Introduction

Polygenic risk scores (PRSs) have become an increasingly popular tool in genetics research. PRSs leverage summary statistics from previous genome-wide association studies (GWASs) to predict risk for individuals in a new population. If the individuals’ predicted risk is associated with their phenotype, this approach provides evidence of polygenic effects even when no genome-wide significant variants exist. When a PRS for one trait is associated with another trait, this approach can be used to establish common polygenic signals between two different traits (Torkamani et al., 2018).

A PRS is a weighted sum of an individual’s alleles, where allele weights are estimated based on their effects in a GWAS in a different sample (International Schizophrenia Consortium et al., 2009). A simple summation across single nucleotide polymorphisms (SNPs) while ignoring the linkage disequilibrium (LD) among them would not be appropriate because trait-associated regions with high LD would be over-weighted. There are several approaches that account for LD in PRS construction. The most common approach, the so-called “pruning-and-thresholding” (P+T) method, constructs the PRS by first removing SNPs in high LD to obtain a set of roughly independent SNPs (pruning) and then including only SNPs that have a p-value below a certain value (thresholding) (Choi & O’Reilly, 2019; International Schizophrenia Consortium et al., 2009). Other methods use penalization to shrink most of the SNP effects to zero (Mak et al., 2017) or use a Bayesian prior that incorporates the LD structure to place downward bias on all of the SNP effects (Ge et al., 2019; Vilhjálmsson et al., 2015).

Regardless of the method, construction of a PRS requires specification of tuning parameters, such as the pruning and thresholding parameters in the P+T method. Typically, PRS analysis involves constructing multiple PRSs across a range of the tuning parameters, followed by selection of the optimal PRS for prediction (i.e. the one that gives the strongest evidence for association). For example in P+T, researchers construct the PRS using around ten different thresholds, and then use all generated PRSs in the analysis. This optimization can inflate the probability of a type I error, if the multiple testing inherent in choosing the best PRS is not accounted for. Inflated type 1 error can be guarded against by using permutations to evaluate significance of the selected PRS; we refer to this approach as Opt-Perm. Here, while the p-value would be corrected for multiple testing, the optimized PRS may still be over-fit and thus the corresponding R² value, which measures the proportion of variation in the trait explained by the PRS, would be inflated. Moreover, it should be noted that permutation procedures can become quite computationally intensive. It has also been proposed to use external or internal validation to choose tuning parameters and avoid permutations. However, external validation datasets are often not available, especially for rarely-studied phenotypes (Mak et al., 2018), and in smaller samples, splitting the data into training and validation sets can decrease power. As an alternative to the optimization approach, one could a priori choose a single tuning parameter setting (e.g. fixing the p-value threshold and LD pruning level) to construct a single PRS. This approach has been used recently in investigations to test for associations of one PRS with many phenotypes (Richardson et al., 2019; Zheutlin et al., 2019). By not optimizing over a set of tuning parameters for each test of association, this strategy avoids further increasing the multiple testing and computation time. However, a sub-optimal PRS may be selected, leading to poor prediction and power to test for association of the PRS with the trait.

Here, we instead compute PRSs over a range of tuning parameter settings, perform principal component analysis (PCA) on the set of PRSs, and use only the first PRS-PC for association testing. The first PC captures the largest amount of variation in the computed PRSs and thus could have better discrimination of the phenotype we are testing. This strategy was recently implemented in studies of polygenic effects on schizophrenia as well as brain imaging (Alnæs et al., 2019; Bergen et al., 2019; De Lange et al., 2019; Maglanoc et al., 2019). This unsupervised approach incorporates all computed scores across a range of tuning parameters and, importantly, is agnostic regarding the outcome of interest and thereby maintains correct type I error. Additionally, the PRS-PCA approach produces a score that is not overfit, which can be used to assess predictive performance of the PRS using measures such as R² or area under the receiver operating characteristic curve (AUC).

Here, we assess the statistical properties of the proposed method in the context of P+T PRS analysis. We begin by constructing PRSs using the P+T approach across a range of p-value thresholds. We then compare the performance of the PRS-PCA approach with the Opt-Perm approach and a priori selection of the p-value threshold tuning parameter. Using simulations and analysis of the Mayo Clinic Bipolar Disorder (BD) Biobank data, we show that the PRS-PCA approach maintains correct type I error and outperforms the other PRS strategies in most scenarios. With the increasing availability of GWAS summary statistics, application of PRS strategies to test for association between many different PRSs and multiple phenotypes is becoming common (B. Coombes et al., 2020; Grigoroiu-Serbanescu et al., 2020; Zheutlin et al., 2019). The PRS-PCA approach can substantially reduce multiple testing without suffering loss of power and is easy to implement, making it especially beneficial in this context.

Methods

Polygenic risk scores

Let $G_{ij}$ denote the number of copies of the reference allele for the $j^{th}$ SNP for the $i^{th}$ individual, possibly estimated via imputation. Let ${\widehat{\beta }}_j$ and $p_j$ be the estimated effect and p-value, respectively, for the $j^{th}$ SNP. The PRS for the $i^{th}$ individual is then ${\sum }_{j=1}^J{G}_{ij}{\widehat{\beta }}_j$ over a set of $J$ markers. Using GWAS summary statistics from a prior analysis of a trait of interest and LD structure estimated either from a reference panel or the target data, the set of J markers to include in the sum and their estimated effects are usually chosen using a P+T strategy (Choi & O’Reilly, 2019). Briefly, P+T “prunes” markers throughout the genome to obtain approximately independent SNPs and only uses SNPs with p-values below a certain threshold to estimate the PRS. This approach can be optimized over different p-value thresholds, clump sizes, and LD measures to determine approximate independence. Typically, researchers a priori select the level of pruning and optimize the p-value threshold over a set of $K$ constructed PRSs, $PR{S}_1,\dots ,PR{S}_K$ based on thresholds: $t_1<t_2<\cdots <t_K$. Each PRS can be written as

$$PR{S}_k=\sum _{j=1}^J{G}_{ij}{\beta }_j^{\left(k\right)}$$

where ${\beta }_j^{\left(k\right)}={\widehat{\beta }}_J{I}_{\left[0,t_k\right]}\left(p_j\right)$ is a hard-threshold version of the effect size ${\widehat{\beta }}_J$ where $I_{\left[0,t_k\right]}\left(p_j\right)$ is the indicator of whether $p_j<t_k$. However, by searching for the best tuning parameter setting, the PRS can be overfit to the target data and the test of association of the PRS with a trait can have inflated type-I error. This can be corrected by using permutations to generate empirical p-values for association with the optimized PRS.

PRS-PCA approach

Instead of using the target data to choose the best PRS, we propose using an unsupervised approach to construct a single PRS from the set of PRSs $\left(PR{S}_1,\dots ,PR{S}_K\right)$ computed over a range of tuning parameter values (e.g. p-value thresholds). Because we are not interested in the actual scale of the PRS, we perform a PCA on the correlation matrix rather than the covariance matrix. This is equivalent to performing PCA on the matrix of standardized PRSs

$$\overline{PR{S}_k}=\frac{PR{S}_k-m_k}{s_k}$$

Where $m_k$ and $s_k$ are the mean and standard deviation of the $k^{th}$ PRS. The PCs are simply weighted sums of the underlying variables $\overline{PR{S}_1},\dots ,\overline{PR{S}_K}$. Specifically, we can write the first PC as $P{C}_1=l_1\overline{PR{S}_1}+\cdots +l_k\overline{PR{S}_K}$ where $l_k$ are the loadings for the first PC extracted by the PCA.

To demonstrate how PRS-PCA reweights the original effect sizes ${\beta }_j$, we first can re-write this as

$$\begin{gathered} P{C}_1=\sum _{k=1}^K{l}_k\overline{PR{S}_k}=\sum _{k=1}^K{l}_k\frac{PR{S}_k-m_k}{s_k} \\ =\sum _{k=1}^K\frac{l_k}{s_k}PR{S}_k-\sum _{k=1}^K\frac{l_k{m}_k}{s_k} \\ =\sum _{k=1}^K\widetilde{l_k}PR{S}_k-C \end{gathered}$$

Where $\widetilde{l_k}$ are the standardized loadings $\frac{l_k}{s_k}$ and $C$ refers to a constant term depending only on terms that do not vary between subjects. Without loss of generality, we drop the constant and define the PRS-PCA as

$$\begin{gathered} PRSPCA=P{C}_1+C=\sum _{k=1}^K\widetilde{l_k}PR{S}_k \\ =\sum _{k=1}^K\widetilde{l_k}\sum _{j=1}^J{G}_{ij}{\beta }_j^{\left(k\right)}=\sum _{j=1}^J{G}_{ij}\sum _{k=1}^K\widetilde{l_k}{\beta }_j^{\left(k\right)} \\ =\sum _{j=1}^J{G}_{ij}{\widehat{\beta }}_{j}w\left(p_j\right) \end{gathered}$$

where $w\left(p_j\right)={\sum }_{k=1}^K\widetilde{l_k}{I}_{\left[0,t_k\right]}\left(p_j\right)$ is the PRS-PCA weight on ${\widehat{\beta }}_j$; which depends on how many thresholds the p-value is below and the loadings of the PRSs corresponding to those p-value thresholds. Assuming $\widetilde{l_k}>0$ for all $k$, we divide $w\left(p_j\right)$ by ${\sum }_{k=1}^K\widetilde{l_k}$ to ensure the weights are between zero and one. Thus, we can see that the weight function corresponds to progressive shrinkage of the original effect sizes ${\widehat{\beta }}_j$. The most significant SNPs retain their original effect sizes while SNPs with larger p-values are multiplied by weights that are smaller than one and decrease as a function of the p-value. SNPs with $p_j>t_K$ have their effect size shrunk to zero. Because the loadings used in the weights are from PCA, PRS-PCA captures the greatest variation of the PRSs computed under the different thresholds used. We use this reweighted PRS (i.e. PRS-PCA) to test for association with the phenotype rather than performing $K$ different association tests with PRSs at different thresholds and then choosing the best one.

Simulations

To estimate empirical type I error and power of the previous and newly proposed methods under different scenarios, we simulated data with and without genotype-phenotype associations. We generated genotypes by sampling without replacement from the Mayo Clinic Bipolar Disorder Biobank sample, followed by generating phenotypes conditional on (or independent of) the genotypes. The Mayo Clinic Bipolar Disorder Biobank collection, genotyping, and genetic data quality control has been described in previous publications (Frye et al., 2015; Markota et al., 2018). Briefly, the Illumina HumanOmniExpress platform was used to genotype cases and controls. For quality control purposes, we excluded subjects with <98% call rate and related subjects. SNPs with call rate <98%, MAF < 0.01, and those not in Hardy-Weinberg Equilibrium (HWE; P<1e-06) were removed. After these steps 643 011 SNPs, 968 cases, and 777 controls remained.

We explored the performance of the methods using samples sizes of N = 500 and 1500 with a balanced case-control design. To avoid assigning “causal” effects to SNPs in LD, we first clumped the summary statistics using PLINKv1.90 (--clump-kb 250 --clump-p=1 --clump-r2=0.1) to obtain 93 802 approximately independent SNPs. To choose realistic effect sizes for SNPs across the genome, we randomly chose the effect size of each independent SNP from a normal distribution with mean equal to $\log \left(\widehat{OR}\right)$ and standard deviation $\widehat{SE}$ of the corresponding SNP in the summary statistics from the Psychiatric Genomics Consortium (PGC) Schizophrenia (SCZ) GWAS (Consortium et al., 2014). We previously showed this PRS was associated with psychosis during mania in bipolar disorder (Markota et al., 2018). Finally, we varied the level of polygenicity of the trait by selecting the ‘causal’ SNPs from the pruned set with absolute value of the $\log \left(\widehat{OR}\right)$ greater than 0.01 (high; 71694 SNPs), 0.07 (medium; 1493 SNPs), and 0.15 (low; 31 SNPs), respectively, in the PGC-SZ GWAS. Using GCTA (Yang et al., 2011), we simulated the liability of a trait and varied the heritability of the liability to be 0, 0.2, 0.4, 0.6, or 0.8. The final simulated liability was then dichotomized at the median to create a balance of cases and controls. For each scenario, data was simulated 3000 and 1000 times to estimate empirical type I error (heritability = 0) and power (heritability > 0), respectively.

We used PRSice2 (Choi & O’Reilly, 2019) to compute PRSs under different p-value thresholds in the simulated datasets using the PGC-SCZ summary statistics. The simulated datasets were then analyzed using PRS-PCA as well as Opt-Perm, and a priori selection of a p-value threshold ($p_T=5x{10}^{-8},0.01$, or 1). To explore the effect of the set of p-value thresholds searched on the performance of PRS-PCA and Opt-Perm, we computed PRSs at either K = 5 ($\left(p_T=5x{10}^{-8},{10}^{-6},{10}^{-4},0.01,1\right)$, 11 ($p_T=5x{10}^{-8},{10}^{-7},{10}^{-6},{10}^{-5},{10}^{-4},0.001,0.01,0.005,0.1,0.5$. 1), or 100 $(\text{thresholds}\hspace{0.17em}\text{chosen}\hspace{0.17em}\text{using}\hspace{0.17em}\text{an}\hspace{0.17em}\text{evenly}\hspace{0.17em}\text{spaced}\hspace{0.17em}\text{grid}\hspace{0.17em}\text{on}\hspace{0.17em}\text{the}\hspace{0.17em}\text{negative}\hspace{0.17em}\text{log}10\hspace{0.17em}\text{scale}\hspace{0.17em}from 0\hspace{0.17em}(p_T=1)\hspace{0.17em}\text{to}\hspace{0.17em}7.3\hspace{0.17em}(p_T=5x{10}^{-8})$ p-value thresholds. It should be noted that the default search implemented in PRSice2 searches multiple hundreds of p-value thresholds over a grid from $5x{10}^{-8}$ to 1. We chose a smaller grid to reduce computational expense in our simulations. Logistic regression models were fit to estimate the p-value of association (H₀: PRS not associated with the trait) of the PRS approaches with the simulated trait using R (R Core Team, 2018) version 3.5.2. We also estimated the percent of variation of the binary phenotype explained by each PRS using Nagelkerke’s R² for each PRS approach using the R package rsq (Zhang, 2018). For each scenario in the simulations, power and type I error were calculated using an α = 0.05 significance level.

Application to Mayo Clinic Bipolar Biobank Data

To compare performance across PRS approaches, we used publicly available GWAS summary statistics to calculate PRSs for a variety of traits for subjects in the Mayo Bipolar Biobank dataset, including: SCZ (Consortium et al., 2014), BD (Stahl et al., 2019), major depressive disorder (MDD) (Wray et al., 2018), attention deficit and hyperactivity disorder (ADHD) (Demontis et al., 2019), anxiety disorders (Otowa et al., 2016), post-traumatic stress disorder (PTSD) (Duncan et al., 2018), obsessive compulsive disorder (OCD)(International Obsessive Compulsive Disorder Foundation Genetics Collaborative (IOCDF-GC) and OCD Collaborative Genetics Association Studies (OCGAS), 2018), anorexia nervosa (AN) (Bulik et al., 2017), insomnia (Lane et al., 2019), and educational attainment (EA) (J. J. Lee et al., 2018). We used PRSice2 (Choi & O’Reilly, 2019) to compute the PRSs using the same settings described for the simulations. Some smaller p-value thresholds were not applicable for GWAS without genome-wide significant variants. We used the various PRS approaches to test for association of each PRS with BD case-control status (N cases = 968; N controls = 777) to compare the performances of the methods in a well-studied phenotype. We additionally repeated these analyses using the history of psychosis during mania in BD cases (N with manic psychosis = 336; N without psychosis = 309) as the phenotype. We recently demonstrated that psychosis during mania is associated with polygenic risk of schizophrenia (Markota et al., 2018). No large GWAS exists for this phenotype, thus, PRS approaches can be quite useful here to elucidate potential differences in genetic background between bipolar cases with and without psychosis, and the genetic overlap of this phenotype with other psychiatric traits in addition to SCZ. We used logistic regression to test for association of each PRS with BD or psychosis status after controlling for the first four principal components of the genotype data to adjust for population stratification. P-values for the Opt-Perm method were calculated using up to 100,000 permutations. We estimated the percent of variation of the binary phenotypes explained by each PRS using Nagelkerke’s R². For the Opt-Perm approach, we followed the standard approach of reporting the Nagelkerke’s R² estimate for the best p-value threshold, which is a biased overestimate of the true R².

Results

Type I error

Table 1 shows the empirical type I error and median R² for each method with varying set of p-value thresholds used, corresponding to setting the heritability of the liability equal to zero. A total of 3000 simulations were performed for each scenario (row) in Table 1. The PRS-PCA approach maintains correct type I error in all scenarios. As expected, optimization of p-value thresholds without correction for multiple testing results in inflated type I error, which worsens as the number of thresholds searched increases. Permutations correct the type I error. However, it should be noted that while both PRS-PCA and Opt-Perm maintain correct type I error, the median R² for Opt-Perm is greater than zero, demonstrating that the median estimate of variance explained by the PRS is biased upward for Opt-Perm. Fig 1 shows that this bias can become quite large under the null hypothesis and is always larger than the bias of the PRS-PCA approach. This was especially true for the smaller sample size (N = 500).

Table 1.

Empirical Type I error and median R² in each simulation setting for each method with sample size N and number of parameters searched K. PCA = first PC of search, Opt = Select best parameter, Opt-Perm = Permutation of Opt

N	K	PCA α (median R2)	Opt α (median R2)	Opt-Perm α (median R2)
500	5	0.054 (0)	0.151 (0.2%)	0.051 (0.2%)
500	11	0.052 (0)	0.194 (0.3%)	0.053 (0.3%)
500	100	0.049 (0)	0.255 (0.5%)	0.050 (0.5%)
1500	5	0.047 (0)	0.145 (0.1%)	0.051 (0.1%)
1500	11	0.046 (0)	0.188 (0.1%)	0.047 (0.1%)
1500	100	0.049 (0)	0.250 (0.1%)	0.044 (0.1%)

Fig 1.

Comparison of Nagelkerke’s R² for PRS-PCA and Opt-Perm under the null hypothesis of no polygenic effect with varying the set of p-value thresholds used (K) and sample size (N). R² asymptotically goes to zero under the null.

Investigation of SNP weights in the PRS-PCA approach

For the set of 93 802 independent SNPs obtained after pruning, we performed PCA on the K PRSs (K = 5, 11, or 100) and plotted the stepwise weight curves for $w\left(p_j\right)$ described in the Methods (Fig 2). These curves show that as p-value increases, the SNP effect is down-weighted more depending on the choice of K thresholds. For example, with our set of K = 5 thresholds, variants with p-value in [1, 0.01), [0.01, 0.0001), [0.0001, 1e-6), [1e-6, 5e-8), [5e-8, 0] were down-weighted by 22%, 6%, 2%, 0.7%, and 0%, respectively. With our K = 11 thresholds, SNPs with p-value in [1, 0.1) and [0.1, 0.05) were severely down-weighted by 61% and 41%, respectively. With K = 100, a smoother form of the down-weighting of SNPs based on their p-values occurs.

Fig 2.

Scaled weights on effect sizes of SNPs (betas) for SCZ-PRS assigned by PRS-PCA in the Mayo Clinic sample. Weights depend on the p-value thresholds used in the analysis (K = 5, 11, or 106), PCA loadings, and the p-value of the SNP from the training sample.

Power

We assessed the power to detect association using the various PRS approaches for a trait with high, medium, or low polygenicity using 1000 simulations for a given sample size and heritability. Empirical power for these methods is shown in Fig 3. All methods had more power using a sample size of 1500, but the relative performance of the methods remained unchanged between sample sizes. The PRS-PCA approach, regardless of the set of thresholds used (K = 5, 11, or 100), had greater power than Opt-Perm in almost all scenarios and achieved nearly the same power as the PRS constructed with fixed threshold matching the simulation setting ($p_T=5\text{e-8}$ and 1 for low and high polygenicity, respectively). The PRS-PCA approach performed best with K = 11 when the trait was highly polygenic. PRS-PCA with the larger set of thresholds (K = 100) did not perform as well in this scenario, but otherwise had generally similar performance as PRS-PCA with K = 11. When the trait had few causal variants, PRS-PCA with the set of K=5 thresholds performed marginally better. While a similar pattern of performance was seen for Opt-Perm, the Opt-Perm performance varied less than PRS-PCA with respect to the choice of thresholds used in the optimization.

Fig 3.

Empirical power of each method given a trait with high (left; |log(OR)| > 0.01), medium (center; |log(OR)| > 0.07), or low (right; |log(OR)| > 0.15) polygenicity with sample size of N = 500 (top) or 1500 (bottom). GWS = genome-wide significant p-value threshold (5e-8). Line thickness corresponds to PRS-PCA and Opt-Perm with varying K (thicker = more thresholds used).

Illustration of Approach: Application to Mayo Clinic Bipolar Biobank Data

For the psychiatric traits considered in the real data analysis, the PRS-PCAs were weakly correlated with each other, with most correlations between 0 and 0.15 and the highest correlation (r = 0.43) between PTSD and AN (Supplementary Figure 1). We also estimated the variation in the PRSs explained by the first PC as well as the PCA loadings for the PRSs of each trait from Table 2 (Supplementary Figure 2). For PRSs with the same set of p-value thresholds, the pattern of PC-loadings was very similar. The first PRS-PC explained between 35% and 75% of the variation in PRSs computed at different p-value thresholds regardless of K and the amount of variation explained by the first PRS-PC was higher for traits where the PRSs calculated under different p-value thresholds were more correlated (EA and SCZ in Supplementary Figure 3).

Table 2.

Comparison of PRS approaches testing for association of each PRS with BD case-control status. The traits are sorted by the PCA approach (K = 11) p-value. Prediction performance was measured by Nagelkerke’s R².

PRS	PCA (K = 5) p-value (R2)	PCA (K = 11) p-value (R2)	PCA (K = 100) p-value (R2)	Opt p-value (R2)*	Best Threshold	Opt-Perm p-value
BD	8e-14 (4.4%)	6e-17 (5.6%)	3e-14 (4.6%)	4e-14 (4.6%)	1	< 1e-05
SZ	5e-06 (1.6%)	5e-08 (2.3%)	5e-07 (1.9%)	2e-11 (3.5%)	0.05	< 1e-05
MDD	0.008 (0.5%)	7e-05 (1.2%)	0.002 (0.7%)	5e-06 (1.6%)	0.05	4e-05
ADHD	0.021 (0.4%)	0.003 (0.6%)	0.023 (0.3%)	3e-04 (1.0%)	1	0.003
Insomnia	0.015 (0.4%)	0.004 (0.6%)	0.006 (0.5%)	5e-04 (0.9%)	0.01	0.005
Anxiety	0.129 (0.1%)	0.023 (0.3%)	0.037 (0.3%)	0.013 (0.4%)	0.1	0.095
PTSD	0.075 (0.2%)	0.043 (0.3%)	0.034 (0.3%)	0.021 (0.4%)	1	0.131
OCD	0.641 (0.0%)	0.072 (0.2%)	0.325 (0.0%)	0.022 (0.4%)	0.2	0.137
AN	0.119 (0.1%)	0.102 (0.1%)	0.065 (0.2%)	0.184 (0.1%)	1	0.656
EA	0.306 (0.0%)	0.278 (0.0%)	0.379 (0.0%)	0.153 (0.1%)	1	0.527

^*Nagelkerke’s R² for the Opt approach is estimated from the best performing p-value threshold searched (K = 11). The best p-value threshold from the K = 11 thresholds searched is reported.

We first considered PRS analyses of BD case-control phenotype, which has been extensively studied (Ruderfer et al., 2018; Stahl et al., 2019). Overall, the PRS-PCA method performed best with our proposed set of K = 11 thresholds. The PRS-PCA method with K = 11 and K = 100 resulted in p-values < 0.05 for 7 of the 10 traits considered. Meanwhile, only 5 PRSs were significantly associated with BD based on Opt-Perm with K = 11. Additionally, a precise Opt-Perm p-value could not be estimated for the strongest two PRS associations (BD and SCZ), due to the computational burden of running more than 100,000 permutations. While the PRS-PCA method took an average of 1.9 seconds to run all three settings of K (5, 11, and 100), the Opt-Perm approach took about 0.09 seconds per permutation (Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz), which means that 100,000 permutations took around 2.5 hours without parallelization.

We also analyzed a case-only phenotype, history of manic psychosis (Table 3), for which no large samples exist. No method performed uniformly best. The PRS-PCA method with K = 11 showed that the PRSs for EA, BD, and SCZ were higher in cases with psychosis than those without. With K = 5, the PRS-PCA analysis failed to detect association with SCZ PRS. The Opt-Perm method (with K = 11) provided weaker evidence of association with the BD-PRS, but stronger evidence of a PTSD PRS association than the PRS-PCA method.

Table 3.

Comparison of PRS approaches testing for association of each PRS with presence of psychosis among cases of bipolar disorder (N = 645). The traits are sorted by the PCA approach (K = 11) p-value. Prediction performance was measured by Nagelkerke’s R².

PRS	PCA (K = 5) p-value (R2)	PCA (K = 11) p-value (R2)	PCA (K = 100) p-value (R2)	Opt p-value (R2)*	Best Threshold	Opt-Perm p-value
EA	6e-04 (2.0%)	9e-04 (2.0%)	0.001 (2.0%)	9e-04 (2.0%)	0.0001	0.006
BD	0.014 (1.1%)	0.020 (1.0%)	0.013 (1.1%)	0.019 (1.0%)	0.0001	0.132
SCZ	0.141 (0.3%)	0.045 (0.7%)	0.071 (0.5%)	0.002 (1.8%)	0.05	0.013
PTSD	0.629 (0.0%)	0.115 (0.4%)	0.319 (0.1%)	0.006 (1.4%)	0.05	0.041
Anxiety	0.459 (0.0%)	0.270 (0.1%)	0.339 (0.0%)	0.219 (0.2%)	0.2	0.829
AN	0.732 (0.0%)	0.551 (0.0%)	0.972 (0.0%)	0.013 (1.1%)	1	0.075
OCD	0.674 (0.0%)	0.584 (0.0%)	0.863 (0.0%)	0.294 (0.1%)	1e-5	0.888
ADHD	0.806 (0.0%)	0.671 (0.0%)	0.865 (0.0%)	0.153 (0.3%)	1	0.664
MDD	0.890 (0.0%)	0.718 (0.0%)	0.955 (0.0%)	0.087 (0.5%)	1	0.438
Insomnia	0.618 (0.0%)	0.884 (0.0%)	0.432 (0.0%)	0.059 (0.6%)	0.0001	0.350

^*Nagelkerke’s R² for the Opt approach is estimated from the best performing p-value threshold searched (K = 11). The best p-value threshold from the K = 11 thresholds searched is reported.

Discussion

In this paper we proposed a method of PRS analysis that uses PCA to concentrate the maximum variation in a set of PRSs in a single PC, and then tests for association of the phenotype with only the first PRS-PC. This method avoids optimizing the parameters to construct the PRS, which inflates the probability of a type I error if unaccounted for, and is computationally faster than using permutations to correct for the inflation (Supplementary Table 1). Through simulations, we showed that the PRS-PCA approach can be as or more powerful than the Opt-Perm approach that relies on permutations to compute the p-value. We showed how the PRS-PCA uses PC loadings to reweight the original SNP effects such that SNPs with larger p-values are down-weighted more.

The choice of thresholds explored performed similarly in most scenarios for both PRS-PCA and even more so for Opt-Perm. Our set of K = 11 thresholds is a very typical choice of analysts, and similar searches have been used previously (Bergen et al., 2019). This setting for PRS-PCA performed best in most of the simulated scenarios (Figure 3), as well as in the real data application (Table 2). Also, in addition to being computationally faster than the Opt-Perm approach, because PRS-PCA tests a single PRS rather than selecting the most predictive PRS in a particular dataset, the PRS-PCA approach does not overestimate PRS performance (e.g. area under the curve or proportion of variation explained) – and in fact, it is expected to underestimate it if only the first PC is used for prediction. On the other hand, the upward bias in prediction performance can be substantial for Opt-Perm when applied to small sample sizes (Table 1).

In our real data application, all of the methods identified five PRSs – BD, SCZ, MDD, ADHD, and insomnia - significantly higher in cases with BD than controls (Table 2). However, only PRS-PCA with K = 11 and K = 100 identified an additional two traits – PTSD and anxiety – with higher genetic load in cases. All of these traits have been shown to have significant genetic correlation or PRS association with BD, in larger samples (Di Florio et al., 2020; Grigoroiu-Serbanescu et al., 2020; P. H. Lee et al., 2019; Nievergelt et al., 2019). In our analysis of the case-only phenotype of manic psychosis, both the PRS-PCA (K = 11) and the Opt-Perm approaches reproduced our previous finding that the PRS for SCZ is higher in cases with a history of manic psychosis (N = 336) than those without a history of psychosis (N = 309) (Markota et al., 2018). Both methods also showed cases with manic psychosis had higher genetic load for EA. While psychosis in the context of BD has been less studied, prior studies have shown small positive genetic correlation between EA and SCZ, and a PRS for EA has been found to be higher in people with SCZ (Power et al., 2015). Finally, only the PRS-PCA approach found evidence that the PRS for BD was higher in cases with manic psychosis. This may suggest that a higher genetic load for BD may cause more severe symptoms of BD.

The PRS-PCA approach controls type I error while maintaining good power. This approach is well-suited to hypothesis testing with many PRSs, because it prevents overfitting each PRS to the outcome and does not require choosing one p-value threshold for all PRSs (Mullins et al., 2019; Richardson et al., 2019; Zheutlin et al., 2019), which can reduce power as seen in our simulations. In this study, we explored how the PRS-PCA approach can improve PRS analyses that implement P+T, by using the first PC PRSs across different thresholds. While more than one PC could be used for association testing in this approach, others have found that subsequent PCs explained relatively little of the variation in the phenotype (Bergen et al., 2019). A similar observation was made in the context of PCA-based SNP-set tests (Ballard et al., 2010). Furthermore, representing polygenic risk with one score allows for ease of interpretation. Here, we have focused on the most popular implementation of PRS: P+T. Future investigation is needed to test if the same PCA approach can be used to avoid optimizing PRSs over other tuning parameters with non-P+T PRS approaches, such as lassosum (Mak et al., 2017), LDpred (Vilhjálmsson et al., 2015), or PRS-CS (Ge et al., 2019). Furthermore, PRSs constructed with different methods could easily be combined using the PCA approach. Here, the PRSs are summed using the loadings for PC1 as weights. However, other weights might be more optimal in certain situations. The PRS-PCA approach could be easily incorporated into the adaptive Sum of Powered Score (aSPU) method, which performs well in many different contexts (B. J. Coombes et al., 2018; Pan, Chen, et al., 2015; Pan, Kwak, et al., 2015). This topic warrants future investigation.

In this paper, we advocate for use of a powerful method of efficiently consolidating information harnessed by PRSs, which avoids the multiple testing inherent in the popular optimization approach. In studies that aim to test for association of PRSs with more than one phenotype such as a PRS PheWAS (Zheutlin et al., 2019) or more than one PRS (B. Coombes et al., 2020), the PRS-PCA approach would substantially reduce the multiple testing that would occur with the optimization approach. With the growing use of PRSs, the PRS-PCA approach gives researchers an unbiased and powerful approach to index polygenic risk.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request. R Code to implement PRS-PCA has been made available in the Supplementary Material.