Implementing Reporting Standards for Polygenic Risk Scores for Atherosclerotic Cardiovascular Disease

Abstract

Purpose of Review

There is considerable interest in using polygenic risk scores (PRSs) for assessing risk of atherosclerotic cardiovascular disease (ASCVD). A barrier to the clinical use of PRSs is heterogeneity in how PRS studies are reported. In this review, we summarize approaches to establish a uniform reporting framework for PRSs for coronary heart disease (CHD), the most common form of ASCVD.

Recent Findings

Reporting standards for PRSs need to be contextualized for disease specific applications.

Summary

In addition to metrics of predictive performance, reporting standards for PRSs for CHD should include how cases/ control were ascertained, degree of adjustment for conventional CHD risk factors, portability to diverse genetic ancestry groups and admixed individuals, and quality control measures for clinical deployment. Such a framework will enable PRSs to be optimized and benchmarked for clinical use.

Introduction

The Framingham Heart Study (FHS), initiated in the late 1940s, introduced the concept of risk factors for coronary heart disease (CHD) [1, 2] and led to the development of an equation to estimate an individual’s 10-year risk of CHD based on such factors [3]. Subsequently data from several prospective cardiovascular cohort studies was pooled to create Pooled Cohort Equations, which are currently used for estimating 10-year risk of atherosclerotic cardiovascular disease (ASCVD) in the clinical setting [4]. However, the performance of risk algorithms for ASCVD, the leading cause of death worldwide, is suboptimal, motivating a search for new biomarkers [5].

There is considerable interest in using polygenic risk scores (PRSs) to improve risk stratification for ASCVD and target interventions to those at highest risk [6••]. Such scores were made possible by genome-wide association studies (GWAS) which identified single nucleotide variants (SNVs) associated with ASCVD [7]. A PRS summates additive genetic susceptibility to a disease by combining variants and effect sizes derived from GWAS and is calculated as

$$PR{S}_j={\sum }_i^N{\beta }_i{\times }_{ij},$$

where $N$ is the number of SNVs in the score, ${\beta }_i$ is the effect size of variant $i$ of individual $j$, and ${ }_{ij}$ is the number of risk alleles [8]. PRSs for CHD initially were constructed from genome wide significant SNVs [9, 10], but evolved to include millions of SNVs across the genome [11–13]. The latter are more strongly associated with CHD, a reflection of the highly polygenic basis of CHD [14].

The heterogeneity in the reporting of PRS studies make it challenging to compare and benchmark PRSs [15, 16••], which is essential to ensure validity and reproducibility [16••]. Recently, a reporting framework for PRS studies was proposed [16••]. In this paper we contexualize this framework for ASCVD, which includes four main subtypes: CHD, cerebrovascular disease, peripheral artery disease, and abdominal aortic aneurysm. We focus on PRSs to estimate risk of incident CHD, the most common form of ASCVD.

Reporting Standards for PRS Studies

To reproduce results of a PRS study, one needs access to SNV weights from GWAS summary statistics, information on testing and validation cohorts used to train and develop the PRS, and the parameters used to calibrate the PRS to independent data. Validation is typically conducted in independent cohorts representative of the target populations, which must also undergo quality control (QC) before utilizing the weights derived from the PRS to identify individuals at high risk in clinical settings (Fig. 1). A reporting framework for PRS studies requires inclusion of gender distribution, genetic ancestry, number of cases and controls, definitions of disease phenotypes and covariates, QC procedures employed, software algorithms, statistical methods, genotyping methods, genotyping arrays, genome build, imputation methods, alternate and reference allele designations [17].

Fig. 1.

An example of a PRS workflow. Created with BioRender.com

A guideline for reporting genetic risk prediction studies, the Genetic Risk Prediction Studies (GRIPS) framework, published in 2011, was based on a checklist developed by the Human Genome Epidemiology Network [18]. This is similar to what had been created for GWAS (STREGA) [19], observational studies (STROBE) [20], diagnostic studies (STARD) [21], and TRIPOD guidelines for multivariable prediction models [22]. The goal of GRIPS was to increase transparency, quality of reporting genetic risk, and completeness of reporting methodology [18]. It included a checklist of 25 items for reporting, including definition of phenotypes, key elements, sources of data, genetic variants and QC, procedures, limitations, how to access data, and writing for a broad audience [18]. Adherence to these standards has been inconsistent and, importantly, GRIPS did not fully address PRSs or specify what should be reported in PRS studies [16••].

To address this gap, in 2021, the Polygenic Risk Score Reporting Statement (PRS-RS) was proposed by the ClinGen Complex Disease Working Group in collaboration with the Polygenic Score (PGS) Catalog [16••]. The PRS-RS framework included background, rationale for PRS development, study population definitions, variable and parameter definitions, details of risk models, risk model evaluation metrics, and clinical translation considerations [16••]. An example of how the 33 PRS-RS requirements can be adopted for a PRS for CHD is shown in Supplemental Table S1 [23].

The PGS Catalog (https://www.pgscatalog.org/), an open database of published PRSs, currently lists 3400 polygenic scores for 596 traits, including 191 cardiovascular disease-related traits [24•]. The catalog provides phenotype definitions, covariate information, statistical models used, demographics for the study populations, study design, PRS performance, limitations, and intended uses [16••, 24•]. To be added to the PGS Catalog, PRS publications should adhere to the PRS-RS [16••]. Together with the GWAS Catalog (https://www.ebi.ac.uk/gwas/), which is resource for GWAS summary statistics [25], the PGS Catalog is meant to facilitate adherence to standards needed for benchmarking and replicating PRSs and the eventual use of PRSs in clinical settings [24•, 25]. However, these standards need to be contextualized for disease specific applications and we discuss below how standards could be adapted for CHD (Table 1).

Table 1.

Contextualizing reporting standards of PRSs for CHD

Reporting framework for a CHD PRS
Metric	Contextual factors
• Training data set • Tuning data set • Validation data set • Heritability • SNV heritability • HR for 1 SD increase in PRS • HR for top 5th percentile vs rest • AUC and R2 for PRS • AUC and R2 for clinical risk factors +PRS • Net reclassification index • Portability to non-European ancestry individuals	• Definition of CHD • Case control ascertainment • Age dependent performance of PRS • Adjustment for relevant covariates • Inclusion of environmental factors/social determinants of health in risk models • Performance of PRS in different genetic ancestry groups and admixed individuals • QC procedures to enable PRS reporting for use in the clinical setting

HR, hazard ratio; SD, standard deviation; AUC, area under the curve; R2, liability

Adapting PRS Reporting Standards for CHD

Harmonizing CHD Phenotype Definitions

It is crucial to report definitions and/or phenotyping algorithms used to ascertain CHD cases and controls in the training, tuning, and validation datasets. Definitions of CHD used for inclusion in GWAS may differ from clinical CHD definitions. For example, GWAS for CHD include participants with myocardial infarction (MI), revascularization, an abnormal stress test, or even angiographic disease. In prospective cohorts, the definition of CHD often includes nonfatal or fatal MI and coronary revascularization. Additional CHD phenotypes include quantitative traits such as coronary calcium, extent of coronary angiographic disease, as well as recurrent CHD events.

Adjustment for Relevant Covariates

Typically, PRSs are adjusted for age, sex, and principal components (PCs) [14, 17, 26]. Additional covariates can include conventional risk factors for CHD include total cholesterol, high-density lipoprotein cholesterol, systolic blood pressure, diabetes status, hypertension, family history, and smoking status [27]. However, in most studies, adjustment for these covariates results in only modest change in the strength of association of PRS with CHD [12, 28].

Predictive Performance and Calibration

Hazard ratio/odds ratio (OR) per one standard deviation (SD) increase in PRS is a commonly used metric. Area under the curve (AUC) is used for as a measure of discrimination, as outlined in the PRS-RS [16••]. Additional measures include net reclassification index and associated integrated discrimination improvement (IDI) [29]. Metrics such as positive predictive value (PPV) and negative predictive value (NPV), sensitivity, and specificity are appropriate for screening tests, but less so for probabilistic risk variables such as PRS [6••, 30, 31]. Calibration of risk models that includes PRSs for CHD should be described.

PRS Performance in the Context of Age

The population for which CHD risk estimation is most useful is young to middle aged individuals [32]. Beyond late middle age, disease prevalence increases and the association of PRS with case status may decrease [33]. When developing and validating a PRS, one should evaluate whether the relative risk or OR, associated with a PRS is variable across age. In reporting PRS results, information regarding age in training and validation sets along with any age-mediated effect modification should be mentioned [8].

Integration Into Existing Risk Frameworks

The most informative measure of risk is the absolute risk of disease over a defined period. There is a need to standardize how PRSs are integrated into existing clinical risk frameworks to estimate absolute risk of disease [27, 31, 34] based on which screening and treatment decisions are made [31, 35–37]. Such integration assumes that the regression coefficients for conventional risk factors are unchanged after adding PRS, which may not be necessarily true.

Validity of PRSs for Different Genetic Ancestry Groups

Transferability of PRSs across populations is variable because of eurocentric bias in GWAS [14]. Furthermore, CHD incidence/prevalence rate varies among genetic ancestry groups [38, 39]. Generalizability of a given PRS across ancestral populations is influenced by the genetic architecture of CHD, linkage disequilibrium (LD) between causal and measured tagging variants, and SNV frequencies that vary across such groups [40]. Additional variation among diverse genetic ancestry populations could be due to differences in how the phenotype is ascertained, genetic drift, selection, environmental differences and gene-environment interactions, uncorrected population stratification, unequal representation of LD and variant frequency across populations, and random error in GWAS effect size estimates [41]. Typically, PRSs are scaled to account for allele frequency differences using population means and SDs within controls of each ancestry to achieve comparable distributions across ancestries [42]. Efforts are ongoing to develop PRSs in diverse genetic ancestry populations to close gaps in predictive accuracy and prevent worsening of health disparities [43]. Guidelines for reporting race and ethnicity have been recently published by the National Academy of Sciences, Engineering and Medicine [44].

Validity of PRSs for Admixed Individuals

Global genetic ancestry is defined as the relative proportion of ancestral blocks from different populations across the genome, representing admixture fractions for an individual, and local ancestry is genetic ancestry of an individual at specific regions of the chromosomes [45, 46]. Global and local ancestry can be ascertained using tools such as ADMIXTURE and RFMIX, respectively (Fig. 2) [47–49]. Many individuals are a mosaic of admixture fractions [46]. Local ancestry inference (LAI) tools such as RFMIX and FLARE can be used to deduce genetic composition of admixed individuals [48, 50]. Development of PRSs for admixed populations is an evolving area. Initial approaches have incorporated ancestry-specific effect sizes, linear combinations of PRSs, and local ancestry [48, 51–53].

Fig. 2.

Global (A) and local (B) genetic ancestry in a Latino individual. Genetic ancestry population proportions are represented as blue for European, green for African, and red for Native Admixed American. Created with BioRender.com

Integrating Environmental Variables and SDOH in Risk Prediction Models

Social determinants of health (SDOH) associated with CHD include low educational attainment, low income, social isolation, structural racism, poor environments, and barriers to accessing high-quality health care [54]. Such factors contribute to differing prevalence of disease in population groups, e.g., black men and women have twice the age-standardized rate of CHD compared to white men and women [54, 55]. Incorporating SDOH into a PRS model could improve prediction of risk for specific health outcomes, including CHD [56]. However, until recently, SDOH have not been documented in electronic health records or in prospective cohorts [57]. It is likely that SDOH will be eventually incorporated risk models for CHD which include PRSs [58, 59].

The Analytic Validity, Clinical Validity, Clinical Utility, and Associated Ethical, Legal and Social Implications (ACCE) Framework

The ACCE framework is used to evaluate genetic tests [60, 61]. It addresses disorder, testing, and clinical scenarios using a set of 44 questions to assess reliability of the information required for decision making [60, 61]. The Center for Disease Control (CDC), Genetic Testing Network Steering Group, and the Public Health Genetic Unit in the United Kingdom have either adopted or been influenced by the ACCE framework [62, 63]. Additional certifications and frameworks relevant to genetic testing include Clinical Laboratory Improvement Amendments of 1988 (CLIA) and College of American Pathologists (CAP) certifications or amendments signify that test results are meeting industry standards for clinical laboratory testing [16••, 64]. CLIA standards cover how tests are performed and the quality control measures to ensure the analytical validity of genetic testing [64]. CAP is a higher quality accreditation to minimize the gap between research and clinical settings by rigorous documentation practices, training protocols, good lab practices, sharing and monitoring of results, appropriate methodology, and standardizing operating procedures throughout research pipelines [65].

Current Clinical Use of PRSs

PRSs are already being used in the clinical setting, for example a PRS for breast cancer is offered by Myriad genetics [66, 67] and a PRS for CHD has been available at Mayo Clinic since 2018, based on the Myocardial Infarction Genes (MI-GENES) randomized controlled trial, in which use of a PRS for CHD led to lower LDL-cholesterol levels [37]. The eMERGE Network is conducting an implementation study of PRSs for 10 common conditions including CHD, atrial fibrillation, asthma, breast cancer, chronic kidney disease, hypercholesterolemia, obesity, prostate cancer, type I diabetes, and type II diabetes. Results will be returned as part of a Genome Informed Risk Assessment (GIRA) that also includes family history and monogenic etiology [68]. The National Health Service in the UK, working with the company Genomics plc, has launched a trial integrating PRS into a cardiovascular risk prediction tool (QRisk) in 1,000 patients in Northeastern England aged 45 to 64 [69]. The study will assess clinical benefit of such integration and is the initial step in a wider rollout of integrated risk scores for a range of common complex diseases, such as breast and bowel cancers, diabetes, and osteoporosis [70].

Outside of the clinical setting, direct-to-consumer (DTC) platforms estimate PRSs for CHD using varying methods. For instance, PRSs from 23andMe are calculated using generalized linear models and are trained on individual level data [71]. Many DTC platforms utilize the Polygenic Index Repository [72], which has its own guidelines for inclusion, similar to those in the PGS catalog. The methods specify all trait/disease PRS reports offered and includes information about variables, characteristics, and performance of the PRSs. 23andMe uses self-reported phenotypes with additional information regarding trait/disease prevalence and consistent demographic trends (i.e., phenotypes in the 23andMe research cohort that are more prevalent in one sex or certain age categories should follow existing trends in other cohorts) [71]. Performance metrics are reported as AUC, risk levels, ORs, AUC in each decade of age, and calibration plots after adjustment [71]. Weighting techniques, ancestral population data, and scaling methods are used to increase transferability across populations [71].

Conclusions

PRSs can improve accuracy of risk estimates for ASCVD and thereby inform preventive strategies. We have attempted to outline how PRS reporting standards can be contextualized for CHD, the most common form of ASCVD. In addition to metrics of predictive performance, factors to be considered include phenotype definition, adjustment for known risk factors, and portability to diverse genetic ancestry and admixed populations. Such standardization will facilitate the use of PRSs in the clinical setting to refine CHD risk estimates.