Performance of Polygenic Risk Scores for Atherosclerotic Cardiovascular Disease in the All of Us Program

Abstract

BACKGROUND:

Performance and transferability of contemporary polygenic risk scores (PRS) for atherosclerotic cardiovascular disease phenotypes may vary across PRS methods, training data, and trait ascertainment.

METHODS:

We aimed to investigate the performance and transferability of contemporary PRS for atherosclerotic cardiovascular disease subtypes: coronary heart disease (CHD), abdominal aortic aneurysm (AAA), ischemic stroke (IS), and peripheral artery disease (PAD), using the All of Us Workbench, which consists of a large, diverse cohort with whole-genome sequence data. We also developed and evaluated a multi-trait PRS for each subtype. Performance of PRS for 4 atherosclerotic cardiovascular disease subtypes was compared across genetic similarity groups in 245 388 All of Us participants. Groups genetically similar to European, African, admixed American, and remaining groups (combined as other) were used to assess PRS for CHD, IS, AAA, PAD, and multi-trait.

RESULTS:

PRS for CHD and AAA performed better than IS and PAD. For CHD, CHD_PGS003725 performed the best (hazard ratio per SD increase [95% CI]), across genetic ancestry groups, European, and African (1.72 [1.67–1.78], 1.23 [1.17–1.29]), with CHD_PGS004696 being best for admixed American (1.91 [1.70–2.15]), and CHD_PGS003356 for other (1.75 [1.58–1.95]). The best performing PRS for AAA was AAA_Multi for European, other, and admixed American (1.71 [1.52–1.92], 1.59 [1.07–2.37], 1.50 [0.90–2.52]) and AAA_PGS003972 for African (1.39 [1.19–1.63]). For IS, IS_Multi performed best for other and European (1.49 [1.17–1.89], 1.33 [1.25–1.42]), and IS_PGS000039 performed best in admixed American and African (1.17 [1.07–1.27], 1.09 [1.04–1.15]). For PAD, PAD_Multi performed best for all groups (other, 1.51 [1.19–1.92]; European, 1.32 [1.24–1.41]; admixed American, 1.23 [1.05–1.45]; and African, 1.18 [1.04–1.34]).

CONCLUSIONS:

Multi-trait and multi-ancestry PRS performed better than individual trait and/or single ancestry PRS for each atherosclerotic cardiovascular disease phenotype across ancestrally diverse and admixed individuals, with minimal change including adjustment for conventional risk factors.

Atherosclerotic cardiovascular disease (ASCVD) is the leading cause of mortality globally^1,2 and includes 4 related subtypes: coronary heart disease (CHD), ischemic stroke (IS), peripheral artery disease (PAD), and abdominal aortic aneurysm (AAA).³ Since the hallmark of each subtype is atherosclerotic plaque in the respective arterial bed, the subtypes share clinical risk factors and genetic susceptibility variants.⁴ Due to a historical bias, polygenic risk scores (PRS) for ASCVD subtypes^3,5 are derived from genome-wide association studies (GWAS) of primarily European genetic ancestry (EUR) individuals. The performance of a PRS depends on the genetic architecture of the phenotype of interest (eg, frequency of alleles, linkage disequilibrium between measured and causal alleles, effect size of alleles) and heterogeneity of development cohorts, as well as quality of summary statistics that are used to create a PRS (eg, high imputation accuracy and sample sizes, similar linkage disequilibrium structure across grouped individuals).^6–9 Newer PRS that include larger GWAS data sets from multiple ancestries perform better in independent and diverse cohorts. In addition, recent studies show that linearly combining PRS for multiple related traits capitalizes on genetic pleiotropy and results in better predictive performance for complex disorders.^10,11

Therefore, we hypothesized that multi-ancestry and multi-trait PRS would perform better for ASCVD subtypes across ancestry groups given the shared genetic architecture of the phenotypes.¹² We compared performance and transferability of available PRS for ASCVD phenotypes (ie, PRS for CHD (PRS_CHD), PRS for AAA (PRS_AAA), PRS for IS (PRS_IS), and PRS for PAD (PRS_PAD) across major genetic ancestry groups. Additionally, we explored the performance of newly developed multi-trait and multi-ancestry PRS for each subtype by leveraging data available in the All of Us (AoU) cohort. The AoU Researcher Workbench includes short-read whole-genome sequence data for 245 388 individuals¹³ and corresponding electronic health record (EHR) data.¹⁴ We ascertained ASCVD phenotypes and risk factors in individuals genetically similar to EUR, Middle Eastern, AFR, admixed American (AMR), East Asian, and South Asian ancestry.

Methods

The general workflow of the study is displayed in Figure 1. Additional information is provided in Supplemental Materials 1 and 2 (Tables S1 through S6; Figures S1 through S15). This study was approved by Mayo Clinic IRB application 10-002758. Due to the sensitive nature of the cohort used for training and validation of the scores, requests for access to the individual-level data sets from qualified researchers trained in human subject confidentiality protocols should be sent to the All of Us Researcher Workbench. All PRS are or will be readily available on https://www.pgscatalog.org/ on publication.

Figure 1.

Polygenic risk score (PRS) validation workflow. Demonstrates the creation of cohorts for each atherosclerotic cardiovascular disease (ASCVD) phenotype, validation of each chosen PRS with the All of Us (AoU) disease-specific cohorts, and statistical analyses that were used to quantify the performance of each PRS across all phenotypes. AAA indicates abdominal aortic aneurysm; AFR, African; AMR, Admixed American; CHD, coronary heart disease; EUR, European; HR, hazard ratio; IS, ischemic stroke; OTH, other; and PAD, peripheral artery disease.

Genotype, Phenotype, and Genetic Ancestry Ascertainment

The All of Us Researcher Workbench (v7) genomic data released in March 2023 includes 245,388 individuals.¹³ We used EHR data to create a cohort of incident cases for ASCVD subtypes, including censored subjects who did not experience an event, as well as demographics and conventional risk factors for ASCVD. Incident cases for ASCVD subtypes were defined as individuals diagnosed after 6 months since entry into the EHR, using algorithms published on pheKB and validated by the eMERGE Network¹⁵ (https://phekb.org/network-associations/emerge). For case-control analyses with logistic regression, controls for ASCVD phenotypes had no ASCVD diagnosis at the time of analysis (Table 1).

Table 1.

ASCVD Phenotype Definitions for CHD, AAA, IS, and PAD

Analyses were restricted to adults (18 years of age or older at first EHR record) and individuals with differing sex at birth, and gender designations were excluded (Figure S1). Type 2 diabetes (T2D), systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), antihypertensives, and statin use were determined at the most recent data entry before the median time to disease diagnosis for each ancestry group (Table S1) in the control group, or at the time of disease diagnosis for cases.¹⁶

Genetic ancestry was predetermined by AoU based on principal component (PC) analysis and random forest classification using the 1000 Genomes,¹⁷ the Human Genome Diversity Project,¹⁸ and gnomAD¹⁹ data. These PC-based groups, including AFR, AMR, East Asian, EUR, Middle Eastern, South Asian, and other, were defined in the ancestry_preds_other file based on genetic similarity. We analyzed genetically classified AFR, AMR, and EUR individuals, with the remaining individuals classified as other (OTH) (ie, highly admixed individuals or those belonging to genetic ancestry groups not large enough to analyze independently).¹³ Hereafter, we use the term “ancestry” to indicate derived genetic ancestry.

PRS Performance in AoU

PRS for each subtype were downloaded from the polygenic score (PGS) catalog (www.pgscatalog.org, November 2023), prioritizing recent PRS trained on multi-ancestry populations when available and limiting PRS chosen to up to 4 PRS for each subtype with the best performance of HR or odds ratio (OR) per SD, with consideration for AUC as previously published unless otherwise noted. We included scores developed by our group (IS_PGS004939 and PAD_PGS004940) due to the limited availability of IS and PAD PRS in the PGS catalog. PRS evaluated in this study are referred to as TRAIT_{PGSCatalogReference} (ie, PRS_CHD by Patel et al²⁰ is CHD_PGS003725). Four PRS_CHD were identified for testing, as more multi-ancestry studies have been published with this phenotype. Two PRS_CHD were chosen from 1 publication²¹ as one of the PRS was implemented in the eMERGE clinical trial.²² Table 2 outlines the PRS from the PGS catalog included in these analyses, with descriptions of development methods and PRS training cohort genetic diversity.^11,20–31 PRS for ASCVD risk factors (ie, T2D, SBP, BMI, LDL-C) were also tested in the AoU cohort. One PRS for each risk factor (chosen based on recently reported performance and inclusion of large, multi-ancestry cohorts) was used in multi-trait PRS development, with sample sizes outlined in Table 2.

Table 2.

Details of Polygenic Risk Scores From PGS Catalog That Were Chosen for Testing

We used the snp_match function in the bigsnpR package to match and flip alleles as needed.³² The score function in Plink2 was then used to create a PRS for all individuals.³³ Testing was performed within genetic ancestry groups (EUR, AFR, AMR) and OTH. We normalized all PRS to zero-mean and unit-variance within each ancestry group.

Multi-trait PRS

To consider the potential for genetic pleiotropy, we computed PRS for T2D, SBP, BMI, and LDL-C independently in the AoU cohort and evaluated how these multiple PRS jointly influence the ASCVD events, creating an adjusted weight multi-trait PRS following previously described methods.¹¹ This method involves combining multiple PRS with other known risk factors to predict a trait of interest.¹¹ We used one PRS for each ASCVD subtype based on performance (HR per SD): CHD_PGS003356, AAA_PGS003972, IS_PGS000039, and PAD_PGS004940, excluding PRS that had included risk factors (ie, CHD_PGS003725²⁰) to minimize bias. Samples were divided into training (75%, n=170,938) and testing (25%, n=56,979) cohorts using the caret package in R, considering any ASCVD case when randomly splitting the cohorts. Case and control sample sizes for training and testing groups used in the calculation of multi-trait PRS for each trait and ancestry are presented in Table S2. The training cohort was used to assess coefficients to weight multi-trait PRS components, which were determined by 10-fold cross-validation in the glmnet and glmnetUtils R packages³⁴ for each ASCVD phenotype and ancestry group (Figure S3). Testing cohorts used the weights derived from the training cohort to combine the traits for the multi-trait PRS.

PRS Reporting

For all models, we followed the PRS reporting guidelines outlined by Wand et al^3,35 when reporting the associations of PRS with ASCVD phenotypes. We performed 2 types of analyses. The primary analyses were based on time to incident events, allowing for censored data, using the Cox proportional hazards model. Time was the time from the first EHR record. Sex, age, 10 PCs, risk factors, and PRS were included as covariates for analyses within each ancestry group (EUR, AFR, AMR, and OTH) for CHD, AAA, IS, PAD, and all ASCVD (Table S3). Secondary analyses were based on logistic regression models within each genetic ancestry group, adjusting for age, sex, and PCs, and we compared these models to those that also included PRS. We additionally estimated the OR per 1 SD and the OR for each subtype for the top 5% of PRS distribution compared with the rest (Table S4). Each PRS was standardized to a mean of 0 and an SD of 1 within each outcome model for each PC-based ancestry group being evaluated to account for population stratification.

C-statistics were assessed for a model that included age, sex, and PCs, as well as a model including age, sex, PCs, and PRS (Table S4). Pearson correlations between PRS within each ASCVD phenotype were assessed (Figure S2). In addition, Pearson correlations were assessed between PRS and the conventional risk factors used in pooled cohort equations (PCE; Figures S4 through S7). PRS results were examined using calibrate and validate functions in the rms package using the bootstrap method with 40 repetitions (Figures S8 through S11). Nagelkerke and calibration R² values were obtained to assess the performance of the PRS across ancestry groups, and Brier Scores were calculated as a measure of accuracy of the predictions (Table S4). Calibration plots are also assessed (Figures S8 through S11). Hazard ratios (HR) from the Cox model per 1 SD increase adjusted for time interval (age at EHR entry to age of diagnosis) and sex were derived from Cox proportional hazards regression. Schoenfeld residuals were examined for deviations from the proportional-hazards assumption. We created additional models for comparison that adjusted for conventional ASCVD risk factors (ie, T2D, SBP, LDL-C), as well as statin and antihypertensive use. Summary information for each validated PRS is in Tables S3 and S4.

Integrated Risk Scores

We also estimated the 10-year risk of ASCVD events for each individual using PCE.³⁶ Variables required for the calculation of the PCE include age, sex, race, total cholesterol, HDL cholesterol, SBP, consumption of antihypertensive medications, diabetes status, and smoking status. These variables were extracted from the data collected for each participant within the AoU cohort.

Ten-year clinical risk of cardiovascular event based on PCE is:

$$\begin{array}{c}\begin{array}{c}PC{E}_{10y}=1-Surviva{l}^{e^{(\sum Ind{X}^{\prime }\beta -\sum Mean{X}^{\prime }\beta )}}\end{array}\end{array}$$     (1)

Where $\sum Ind{X}^{\prime }\beta$ is the sum of individual’s values of covariates multiplied by their coefficients, $\sum Mean{X}^{\prime }\beta$ is the sum of population’s mean of covariates multiplied by their coefficients, and $Survival$ is the sex- ancestry-specific survival rate of the population.³⁷

We then combined the 10-year risk of cardiovascular event (PCE) with the PRS tested for CHD, AAA, IS, and PAD to create an integrated risk score (IRS) that incorporates polygenic risk score for each trait as follows:

$$\begin{array}{c}\begin{array}{c}IR{S}_{10y}=1-Surviva{l}^{e^{((\sum Ind{X}^{\prime }\beta -\sum Mean{X}^{\prime }\beta )+(PRS*\ln \left(H{R}_{per1SDofPRS}\right)))}}\end{array}\end{array}$$    (2)

Where PRS is the standardized PRS score based on the distribution of PRS in the ancestry and ${\beta }_{PRS}$ is equal to $\ln \left(H{R}_{per1SDofPRS}\right)$, which is the natural logarithm of the HR for each subtype for 1 SD of each PRS. The HR per SD is specific to the PRS and subtype being analyzed.

Net reclassification indices (NRI) were evaluated for all PRS using the nricens package with the function nribin for categorical and continuous NRI. Sensitivity and specificity were calculated for all clinical and integrated risk scores (Table S5). Risk category reclassifications were also examined based on a threshold of 10-year ASCVD risk of 10% (Figures S12 through S15).

Results

Overall, among PRS for ASCVD subtypes, PRS_CHD performed best (Figure 2). Of the PRS_CHD, CHD_PGS003725 had the strongest associations with CHD in EUR and AFR ancestry groups of the AoU cohort (HR [95% CI]), with the highest performance in EUR(1.642 [1.590–1.696]) and African (1.178 [1.121–1.238]). PRS testing in the AMR group showed CHD_PGS004696 to be the best performing (1.623 [1.479–1.782]), and CHD_PGS003356 performing best in other (1.754 [1.575–1.953]; Figure 2; Table S3). The CHD_Multi PRS showed similar results to the best performing ancestry-specific PRS, with the closest comparisons for EUR(1.668 [1.566–1.777]) and AFR(1.183 [1.058–1.323]), and yet attenuated performance for AMR (1.470 [1.252–1.727]) and OTH populations (1.592 [1.255–2.020]; Figure 2; Tables S3 and S4). The performance of PRS_CHD in AFR individuals was much lower across all groups (Figure 2; Table S3). Based on Nagelkerke R² values, the model used to develop CHD_PGS003725 that integrates other related traits provides better predictive accuracy than CHD_Multi (Figure 3; Table S4).

Figure 2.

Polygenic risk score (PRS) validation for atherosclerotic cardiovascular disease (ASCVD) subtypes in groups by genetic similarity. PRS validation in African (AFR), European (EUR), admixed American (AMR), and other (OTH) genetic ancestry populations for coronary heart disease (CHD), ischemic stroke (IS), abdominal aortic aneurysm (AAA), and peripheral artery disease (PAD). Plots show hazards ratio per SD (HR per SD) with CIs for each PRS tested in each phenotype.

Figure 3.

Nagelkerke R² of polygenic risk score (PRS) for atherosclerotic cardiovascular disease subtypes in genetic ancestry groups. Nagelkerke R² values with 95% CI of PRS for coronary heart disease (CHD), ischemic stroke (IS), abdominal aortic aneurysm (AAA), and peripheral artery disease (PAD) for single trait PRS and multi-trait PRS for African (AFR), European (EUR), admixed American (AMR), and other (OTH) genetic ancestry groups.

For AAA, we compared 3 PRS_AAA (ie, AAA_PGS002054, AAA_PGS001784, and AAA_PGS003972) in the 4 ancestry groups in AoU. AAA_PGS002054 was developed using LDPred2, and the remaining 2 PRS were developed with PRS-CS.^24–26 AAA_PGS001784 and AAA_PGS003972 were developed using EUR cohorts, and both used multi-ancestry GWAS summary statistics (Table 2).^25,26 PRS-CS scores outperformed the LDPred2 score in all ancestry groups, though it is noteworthy that GWAS summary statistics used in training data differed (Figures 2 and 3, Table S3). Out of previously published PRS for AAA, AAA_PGS003972 had the largest GWAS training data for AAA and was the best performing PRS_AAA, across ancestry groups with HR per SD of 1.685 (1.590–1.786) for EUR, 1.448 (1.157–1.812) for AMR, 1.436 (1.194–1.728) for OTH, and 1.393 (1.193–1.626) for AFR despite limited diversity in training cohorts. Comparing with the newly developed multi-trait score, AAA_Multi had slightly better performance in EUR (1.706 [1.517–1.919]), OTH (1.590 [1.068–2.366]), and AMR (1.504 [0.900–2.515]), yet with larger CIs. AAA_Multi also had calibration, and Nagelkerke R² value greater than the single trait PRS by an average of 60.5% across ancestry groups (Figure 3; Table S4).

For IS, 3 PRS_IS^38,39 varying in development methods were compared.^27,40–43 The PRS_IS were trained on European populations, limiting the transferability of these scores between ancestry groups. However, IS_PGS000039 and IS_PGS004939 additionally used multi-ancestry GWAS summary statistics,^11,27 and improved the cross-ancestry performance compared with IS_PGS000053, which did not use external GWAS summary statistics and reflected a primarily European study. There was also a difference in PRS development method, allowing comparison of metaGRS as a multi-trait method and PRS-CS as a single-trait continuous shrinkage method (IS_PGS000039 and IS_PGS004939).^27,42,44 Considering scores from the PGS catalog, IS_PGS004939 performed best for EUR (1.161 [1.124–1.199]), whereas IS_PGS000039 performed best for OTH (1.253[1.073–1.464]), AMR (1.167 [1.072–1.270]), and AFR (1.091 [1.037–1.148]; Figures 2 and 3; Table S3). IS_Multi performed better in 2 out of 4 ancestry groups (EUR: 1.334 [1.250–1.423], OTH: 1.487 [1.172–1.886]), while performing similarly for AMR (1.140 [0.973–1.336]) and AFR (1.088 [0.973–1.216]), resulting in an average increase of HR per SD 1.14-fold.

Transferability of the selected available PRS_PAD was limited overall, with little predictive value for AFR (HR per 1 SD, ≈1). PAD_PGS004940 consistently outperformed PAD_PGS002055 for EUR (1.256 [1.218–1.295]), AMR (1.134 [1.051–1.224]), OTH(1.118 [1.035–1.208]), and AFR (1.091 [1.021–1.166]; Figure 2; Table S3). However, PAD_Multi performed better overall with an average HR per SD increase of 16.6% for EUR (1.374 [1.292–1.461]), 11.4% for AFR (1.185 [1.045–1.343]), 12.0% for AMR (1.242 [1.063–1.451]), and 46.8% for OTH (1.562 [1.241–1.966]) from the best performing PRS.

Adjustment for conventional risk factors (T2D, SBP, LDL-C), as well as antihypertensive and statin use¹⁶ modestly attenuated the HR per SD for the PRS (≈0.1 HR per SD decrease), while increasing the C-statistic (≈0.1 C-statistic increase). Overall, the multi-trait PRS for each ASCVD phenotype generally performed better than the best single trait PRS, supporting the inclusion of ASCVD subtypes and risk factors in PRS development. It should be noted that the multi-trait PRS for each trait was trained and tested using independent groups of AoU participants, and results may be inflated, thus requiring additional external validation before downstream uses, such as clinical application. For others, PRS performance varied across ASCVD subtypes, as this group includes a wide variety of admixed and nonadmixed individuals with limited sample sizes and resembled the performance in the AMR group. Calibrations and correlations for each PRS, trait, and ancestry group showed variable results relative to phenotype and ancestry group (Figures S4 through S11). Calibration plots showed better calibration where there was sufficient data and deviated more in the tails of the distribution, likely due to sparse data (Figures S8 through S11). Correlation coefficients of PCE-based risk factors with PRS values are comparable to a recent study using PRS in the AoU Researcher Workbench, considering the difference in study populations being compared.⁴⁵ R² statistics from calibration are shown in Table S4, and Nagelkerke R² with 95% CI are displayed in Figure 3. Expectedly, PRS performed better in EUR than in other ancestries for all ASCVD subtypes, and EUR-trained PRS performed poorly in AFR.⁴⁶ Sample sizes for training PRS for subtypes are noted in Table S5 (Supplemental Material 2), and training sample sizes for multi-trait PRS are in Table S2 (Supplemental Material 1).

We also examined performance of the IRS that included PRS and PCE to estimate the 10-year ASCVD risk in the AoU cohort. The IRS for all ASCVD subtypes had reasonable discrimination (C statistics: 0.682–0.778; Table S6). We identified improvement in risk categorization using continuous NRI and categorical NRI with a 10% 10-year ASCVD risk threshold (Figures S12 through S15; Table S6). Categorical NRI showed improvement in classification ranging from 0.5% to 5.1% for CHD, 0.3% to 2.6% for AAA, −0.2% to 0.6% for IS, and −0.2% to 0.5% for PAD. However, continuous NRI had all positive NRI ranging from 25.0% to 32.3% for CHD, 5.3% to 29.6% for AAA, 3.2% to 4.8% for IS, and 2.0% to 11.6% for PAD. IRS showed improvement in prediction for 10-year ASCVD risk scores compared with clinical risk scores.

Discussion

Performance of contemporary PRS for ASCVD subtypes varied by trait, ancestry group, and method of PRS development. PRS for CHD and AAA subtypes performed better across genetic ancestry groups than IS and PAD (Figures 2 and 3). This is consistent with higher heritability estimates for CHD (40%–60%)^47,48 and AAA (70%–77%)^49,50 than those for IS (37%–38%)⁵¹ and PAD (11%–55%).^52,53 Expectedly, performance was best in those of EUR ancestry and the least in those of AFR ancestry. Due to the splitting of the AoU cohort into training and testing cohorts for the development of multi-trait PRS, it should be noted that sample sizes are smaller for testing in multi-trait PRS compared with the other PRS. Despite this, multi-ancestry and multi-trait PRS tended to perform better for AAA, IS, and PAD.

PRS_CHD has evolved and improved over the last decade⁵⁴ but remains suboptimal for non-EUR ancestries. For example, the LDPred2-based method used to develop CHD_PGS003725²⁰ combined GWAS summary statistics for CHD in addition to other ASCVD phenotypes and risk factors, and was then trained on primarily EUR populations in the UK Biobank.²⁰ This method had comparable performance to the multi-trait PRS developed in this study for most ancestry groups. There was a marked increase in performance for CHD_PGS004696 in AMR. This can likely be attributed to the multi-ancestry training of this PRS using PRS-CSx, resulting in better prediction for admixed populations such as AMR when utilizing adjustments for PCs as well. However, PRS applied to AMR and OTH for all ASCVD subtypes showed wider CIs for HR statistics, suggesting high uncertainty, and should be used with caution. Performance of PRS_AAA has improved substantially, as the most recent PRS_AAA was the best performing (AAA_PGS0003972) and leveraged 24 known genomic risk loci associated with AAA,^55–58 using larger sample sizes than previously available.²⁶

PRS_PAD and PRS_IS performed less well, implying the need for more data for these phenotypes before implementation into clinical practice (Figure 3). PRS_PAD performed similarly for AMR and OTH, but had a lower average R² compared with EUR than PRS_CHD and PRS_AAA with broader CIs overall (Figure 3). HR per SD of up to 1.69 (IS) and OR per SD of 1.44 (PAD) have been reported for cohort-based studies; however, the scores were not externally tested and were not available publicly or on the PGS catalog.^52,59 As many publications reporting metrics for PRS preceded or did not adhere to the current guidelines,^3,35, direct comparison of PRS performance is difficult. Additionally, PAD is a heterogeneous phenotype, and case ascertainment may vary across cohorts. Environmental and lifestyle factors, such as smoking, may have a larger effect on PAD susceptibility than genetic factors.

Consistently lower performances of PRS for ASCVD subtypes in AFR^12,60 is likely due to the greater genetic distance from EUR, who compose most training data available for PRS development.^20,21 Whether differences in causal variants and subsequent effect sizes contribute to performance is unclear.^61,62 There is need for inclusion of more individuals genetically similar to African and Latino genetic ancestries, and improving portability of scores through methods,⁶³ as PRS derived from diverse, multi-ancestry training data generally performed better than single-ancestry derived PRS (Figures 2 and 3; Table S3).²¹ Differential performance of PRS for ASCVD subtypes across genetic ancestry groups could be due to varying size of training data sets, case ascertainment algorithms within GWAS and PRS development, PRS development methods, and overall diversity of training populations (Tables S2 and S5).

In addition to advances in single-trait PRS, leveraging genetic correlations between ASCVD traits further improved performance for each ASCVD trait.^10,64–66 Reported multi-trait studies for similar phenotypes, including CHD, have found that going beyond correlated traits and risk factors for inclusion criteria can also improve PRS.^11,64 Similar to our findings, analyses with multi-trait methods have improved performance for correlated psychiatric and behavioral disorders,⁶⁷ such as the iPSYCH cohort that combined 937 PRS and improved performance in all disorders compared with single trait PRS.¹⁰ Despite the performance gains using multi-trait methods, PRS for non-EUR groups remain suboptimal with decreased performance in IS and PAD. Sample sizes in training and testing cohorts for the multi-trait PRS for IS and PAD were comparable to the other traits, with most cases in EUR, followed by AFR, AMR, and OTH.

The process of benchmarking in a highly diverse cohort enabled the identification of best-performing PRS for potential integration with clinical risk scores. IRS used for clinical risk evaluation showed an increase in performance when integrating PRS_CHD (Table S6). IRS, with the best performing PRS, will provide the most accurate risk estimates for patients. The All of Us cohort is useful for validation of contemporary PRS as it includes a large, diverse group of individuals. It is currently standard to assume that there is no sample overlap between AoU and other cohorts used in this analysis; however, individual-level overlap cannot be entirely ruled out, given the case that an individual did not document participation in another study. PRS_AAA and PRS_CHD performed robustly, suggesting potential for clinical utility. However, PRS_IS and PRS_PAD performed less well. Previous use of PRS for ASCVD phenotypes in clinical settings has shown promising results. For example, in the MI-GENES trial, patients chose to implement changes based on PRS to reduce the risk of CHD by starting statins.⁶⁸ The Electronic Medical Records and Genomics (eMERGE) Network is testing clinical implementation²² of PRS for 9 common conditions in adults, including CHD.⁶⁰

PRS for ASCVD developed from multi-ancestry cohorts and multiple related traits performed best across ancestrally diverse and admixed individuals. PRS for CHD and AAA performed better than those for IS and PAD. Limited sample sizes and diversity within development cohorts restrict the portability of these PRS. As PRS continues to evolve, periodic assessment of performance may be needed. One such evolution includes the evaluation of new PRS using continuous ancestry adjustment methods.⁶⁹ The anticipated development of additional multi-ancestry and multi-trait PRS introduces challenges for benchmarking, given the need to track multiple inputs, genetic variant sets, and weights. This study highlights the need for diverse cohorts for GWAS and PRS development for all ASCVD subtypes and supports the inclusion of correlated traits in PRS development.

Limitations

The AoU cohort defined genetic ancestry based on the projection of samples onto the PC space, rather than alternative methods of determining genetic ancestry, such as ADMIXTURE and SCOPE, which infer individual ancestry proportions from multiple ancestral populations using clustering based approaches. Therefore, ancestry assignments may have differed for some individuals, particularly those with admixed ancestry.^70,71 Additionally, while the AoU cohort was not used in developing the PRS for the ASCVD phenotypes, a few individuals present in the AoU cohort could have been subjects in other study cohorts used in the development of the PRS weights.

ARTICLE INFORMATION

Acknowledgments

The authors acknowledge the All of Us participants for their contributions to the All of Us Researcher Workbench for the opportunity to contribute to this research. We thank the National Institutes of Health’s All of Us Research Program for making the data available for use in this study. Data and Code Availability: polygenic risk scores (PRS) chosen for validation were publicly available through the polygenic score (PGS) catalog (https://www.pgscatalog.org/), except for 2 internally developed PRS, which will be published on PGS Catalog upon publication (ie, PGS004939, PGS004940). We used data from the Researcher Workbench for All of Us Research Programs Controlled Tier data set version 7.1, which requires access to data directly with the All of Us Research Program, as it includes genomic data in the form of whole genome sequencing and genotyping arrays, previously suppressed demographic data fields from electronic health records and surveys, and unshifted dates of events. Multi-trait PRS have not been validated externally at this time, but may be available on request.

Disclosures

None.

Supplemental Material

Tables S1–S6

Figures S1–S15

Supplementary Material

hcg-19-e005382-s001.xlsx ^{(19.3KB, xlsx)}

Funding Statement

This study was done as part of the Polygenic Risk Methods in Diverse Populations (PRIMED) Consortium funded by the National Human Genome Research Institute (NHGRI) grant U01HG11710, as well as T32 grant HL007111-45, National Heart, Lung, and Blood grant K24 HL137010, and R35 GM140487.