Genetic Risk Scores for the Clinical Rheumatologist

Abstract

Background/Historical Perspective:

The advent of genome-wide sequencing and large-scale genetic epidemiological studies has led to numerous opportunities for the application of genetics in clinical medicine. Leveraging this information towards the formation of clinically useful tools has been an ongoing research goal in this area. A genetic risk score (GRS) is a measure that attempts to estimate the cumulative contribution of established genetic risk factors towards an outcome of interest, taking into account the cumulative risk that each of these individual genetic risk factors conveys. The purpose of this perspective is to provide a systemic framework to evaluate a GRS for clinical application.

Summary of Current Literature:

Since the initial polygenic risk score methodology in 2007, there has been increasing GRS application across the medical literature. In rheumatology, this has included application to rheumatoid arthritis, gout, spondyloarthritis, lupus, and inflammatory arthritis.

Major conclusions:

GRSs are particularly relevant to rheumatology, where common diseases have many complex genetic factors contributing to risk. Despite this, there is no widely accepted method for the critical application of a GRS, which can be a particular challenge for the clinical rheumatologist seeking to clinically apply GRSs. This review provides a framework by which the clinician may systemically evaluate a GRS.

Future research directions:

As genotyping becomes more accessible and cost-effective, it will become increasingly important to recognize the clinical applicability of GRSs and identify those of the highest utility for patient care. This framework for the evaluation of a GRS will also help ensure reliability among GRS research in rheumatology, thereby helping to advance the field.

Introduction

Many clinicians are familiar with the concept of Mendelian inheritance of traits (i.e. autosomal dominant, autosomal recessive, X-linked). However, in adult rheumatology, the most common diseases have many genetic risk factors that play a role in development along with environmental exposures, leading to complex genetic risk that goes beyond individual single nucleotide polymorphisms (SNPs) and follows non-Mendelian inheritance patterns. This has led to the importance of genome-wide association studies (GWAS) and further research into more complex measures of genetic risk (1–6). The advent of genome-wide sequencing and genetic epidemiology has led to opportunities for the application of genetics in clinical medicine, including diagnosis, prognosis, risk stratification, and targeted therapeutic intervention.

A genetic risk score (GRS) is a measure that attempts to estimate the cumulative contribution of established genetic risk factors towards an outcome of interest, taking into account the amount of risk that each of these individual genetic risk factors conveys (Table 1) (7). While there has been some interchange between the terms GRS and polygenic risk score (PRS) in the literature, these two entities are technically different, with PRS being a genome-wide extension to large numbers of possibly correlated markers, aiming to capture all heritable variation. While both approaches have been actively studied for clinical application, GRS has been predominant.

Table 1.

Glossary.

Single nucleotide polymorphism (SNP) A variation of a single DNA base pair, which account for the majority of sequence variation between individuals.

Genetic risk score (GRS) A score which attempts to estimate cumulative genetic risk towards an outcome of interest by accounting for the risk conveyed by multiple SNPs established as associated with the phenotype.

Polygenic risk score (PRS) A score which attempts to account for all heritable SNP variation in prediction of an outcome, often identified via GWAS summary statistics.

Genome-wide association study (GWAS) A study which aims to identify SNPs associated with an outcome of interest by comparing large numbers of genomes between those with and without the outcome.

Genetic variance Between-individual genetic differences in deoxyribonucleic acid (DNA) sequence.

Allele An alternative DNA sequence located at a specific genetic location.

Population stratification The state where populations are distinguishable by observing genotypes, reflective of genetic ancestry.

Despite wide applications across the medical literature, there is no gold standard approach for generating or interpreting a GRS, and this can make understanding and evaluating a GRS challenging for the rheumatologist without formal genetics training. As our understanding of genetic risk factors and GRS applications continues to improve, it will be important to develop methods by which rheumatologists can critically evaluate GRS literature, particularly with the increasing availability and decreasing cost of clinical genetic testing.

History and application of the genetic risk score

The first PRS was published in 2007 in a methodologic paper using hypothetical data, showing that ~75 loci could explain >50% of genetic variance for a disease (8). The authors suggested using either the accuracy of risk prediction (the correlation between true and predicted genetic risk) or the area under the receiver operating characteristic curve (AUC) to evaluate the utility of genetic profiling. The AUC remains the most common method of showing overall GRS utility, however, there has been criticism as to how this metric applies towards population-based screening (9).

Since that initial publication, there have been many GRS applications across various subspecialties. In cardiology, this has included identification of coronary heart disease risk, as well as atherosclerosis burden and implications towards personalized medicine showing differential benefit from statin therapy (10–17). In oncology, where there is a clear movement towards personalized tumor-directed therapy, GRS-based cancer risk stratification has continued to advance (18–20). GRS applications have also extended into psychiatry and endocrinology with intriguing results (21–25). However, there has not yet been widespread clinical use, and despite these successes, there has also been concern regarding the usefulness of applying such findings in the clinical setting and whether genotyping is justified by additional risk prediction (9).

As GRSs have made their way into the rheumatology literature, they have extended to a diverse range of rheumatic diseases and disease manifestations. In rheumatoid arthritis (RA), there have been initial applications of GRSs towards overall RA risk, as well as specific features such as seropositivity and prevalent interstitial lung disease (ILD) (26–30). There have also been successful GRS applications towards gout, spondyloarthritis, and lupus disease manifestations (32–34). Despite being relatively new to rheumatology, there are already GRS implications in the realms of diagnosis (27,32), prognosis (26,29,33,34), and risk stratification (27,28,30). Another exciting development in GRS research is the use of genetic probability for assisting diagnostic evaluation, as has been recently applied in undifferentiated early inflammatory arthritis (35). With the expanding applications of GRS research in rheumatology, it is becoming increasingly important for rheumatologists to be familiar with how to interpret such findings.

Critical evaluation of genetic risk scores

The general concept of a GRS is to estimate the cumulative contribution of genetic risk factors towards an outcome of interest. In generating a GRS there are five essential steps that must be undertaken (Figure 1).

Figure 1. Five essential steps in GRS formation.

This figure shows the five essential steps required to form a genetic risk score. These include 1) selection of pre-defined SNPs for the outcome of interest; 2) individually weight SNP effect; 3) create a summative GRS with weighted SNPs; 4) evaluate score performance; and 5) validate the score and assess calibration. These five essential steps are used to guide the systemic evaluation detailed throughout the manuscript.

Abbreviations: single nucleotide polymorphism (SNP), genetic risk score (GRS)

Given the lack of a standardized method for generation and reporting, combined with the wide variety of applications, it can be difficult to evaluate the quality of research using GRSs. To remedy this, we have outlined a system to assist clinicians in the critical evaluation of a GRS article, focusing on identification of key aspects of strong methodology, reliable results, and applicability (Table 2). We have separated this into nine distinct components that contain critical questions to be asked while evaluating the GRS study. The reader should be able to identify the answer to these questions if the study is well-designed and clearly presented, even without a genetics background. If all of these questions have been addressed in the manuscript, then this provides reassurance that the study has been carefully and reliably designed. If none of these are clear or a large number are missing, then the reader should be skeptical. If some of these components are missing, then the study should be interpreted cautiously, although the overall impact on its credibility depends on the exact deficiency. For this purpose, we will assume that other aspects of the study are appropriate and will focus solely on the GRS evaluation. Examples of phrases from a manuscript are provided from a study which evaluated the use of a GRS for identification of RA-ILD among patients with RA (30). Key methods for each step are summarized in Table 3.

Table 2.

Critical evaluation checklist for a GRS research article.

1. SNPs included
How were the SNPs initially selected for inclusion in the GRS?
What SNPs were included in the GRS?
2. Genetic model
How was genetic risk modeled for the effect allele?
3. Weighting
Are the SNPs weighted within the score?
How are the SNPs weighted within the score?
How was the effect size estimated?
4. Selection process
Were all identified SNPs included or were any removed from the final score?
5. Population stratification
Was there adjustment for population stratification?
6. Performance
What method was used to evaluate performance of the GRS?
Was this compared to a model with clinical factors that did not include the GRS?
7. Validation
How was the GRS validated?
Was calibration performed/reported on the model?
8. Application
Are there clear instructions on how to calculate the score?
Are there recommended cut-points for the GRS or the regression model?
9. Generalizability
Are these findings generalizable to the desired clinical setting?

Abbreviations: single nucleotide polymorphism (SNP), genetic risk score (GRS)

Table 3.

Key methodological approaches and statistical analyses for each step in the development of a genetic risk score (GRS).

Step in Genetic Risk Score Development	Key Methodological Approach and/or Statistical Analyses
SNP Selection	A priori selection, selection algorithm (ie, LASSO, Bayesian, filtering, pruning, backward selection, predictor selection)
SNP Genetic Model	Additive, genotype/codominant, autosomal dominant, autosomal recessive
SNP Weighting	Weighting by a log odds ratio of effect
Population Stratification - Determination	Principal component analysis of genetic data to estimate population structure; mapping to known reference population for labeling genetic ancestry
Population Stratification - Adjustment	Account for population structure as a confounder using regression, stratified analysis, or restriction
GRS Performance	Regression model for the strength of association with outcome of interest; percentile-based comparison. reporting area under the curve for predictive performance; R2 or pseudo-R for the proportion of variation explained by model; comparison of nested models with clinical variables using likelihood ratio testing, information criteria comparison
GRS Validation	Internal validation methods (cross-validation (k-fold), boot-strapping); External validation in a separate cohort
GRS Calibration	Calculation of expected: observed ratios, use of Hosmer-Lemeshow goodness of fit testing, adjustment for overfitting with shrinkage factor.
Generalizability	Calculation of detection rate, false positive rate, likelihood ratio for test, number needed to genotype, sensitivity, specificity

Abbreviations: single nucleotide polymorphism (SNP), least absolute shrinkage and selection operator (LASSO)

How were the SNPs initially selected for inclusion in the GRS?

Authors will have pre-identified the SNPs to be included in their score. Classically, SNPs are identified from the prior literature, either genome-wide association studies (GWAS) or single variant studies (candidate gene studies), however, this is not always the case. Currently, GWAS are the predominant source of SNP predictors; GWAS offers systematic identification with allele effect sizes derived from the same dataset. If GWAS summary statistics are used, then the reader should ask themselves 1) “What was the outcome of interest for the GWAS?” and 2) “What was the study population?”. These are important in assessing generalizability and appropriateness of the SNPs that are included. If the outcomes used in GWAS studies are different from the outcome identified by the GRS, or if study populations are substantially different, then the underlying assumption that SNPs included in the GRS contribute to risk of the outcome may not be true. For example, differences in genetic ancestry may lead to genetic association in one ancestry but not another (31). Due to these differences, an association that was identified in one population may not be applicable in a different population with different genetic ancestry. For example, the PTPN22 R620W SNP (rs2476601) has been identified as a risk factor for multiple autoimmune diseases. This SNP has an allele frequency of 10.1% in populations of European ancestry, but < 0.01% in populations of East Asian ancestry (1,5,25,31,35,36). This PTPN22 SNP may then be selected for inclusion in a GRS in a European population but, due to rarity, may not be selected in a GRS in an East Asian population.

There may also be situations where no prior studies exist, in which case the study may include an initial GWAS to identify associated SNPs. SNP identification via GWAS within the same cohort raises potential for overestimation of GRS performance, though this can be mitigated by partitioning and specialized statistical approaches. SNPs may also be selected based on functional and biological considerations or by using a validated algorithm to filter SNPs based on the statistical strength of association and degree of genetic correlation (linkage disequilibrium) with each other. Regardless, the criteria and methods for selection should be clearly defined.

Example: “[SNPs] were selected based on prior associations with idiopathic pulmonary fibrosis in GWAS studies and with RA-ILD in a large international cohort…” (30)

What SNPs were included in the GRS?

While this may seem self-explanatory, it is not consistently reported. Specific SNPs which were included should be listed and described in a table (often supplementary material), detailing all relevant paraments such as effect size, source, and allele frequencies.

Example: “We extracted MUC5B rs35705950, TOLLIP rs5743890, DSP rs2076295, OBFC1 rs11191865, DPP9 rs12610495, EHMT2 rs7887, FAM13A rs2609255, LRRC34 rs6793295, IVD rs2034650, ATP11A rs1278769, intergenic rs4727443, and TERT rs2736100, which were directly genotyped.” (30)

How was genetic risk modeled for the effect allele?

There are four general models used to convey genetic risk. An additive model is the most commonly used in GWAS and GRS research. It is generally felt to be the most appropriate model for complex genetic risk as it reduces misclassification bias. It is the widely accepted model of polygenic inheritance, both of alleles at a given locus and additively across large numbers of loci, with height being a classic example (37, 38). If not specifically stated, phrases which may clue the reader to the use of an additive model include “summing the total number of risk alleles”, “adding number of reference alleles”, or “0,1, or 2 alleles”. The alternative are models that follow Mendelian inheritance patterns – dominant, recessive, or co-dominant models. Phrases that may indicate the use of these include “presence of risk allele”, “major or minor allele”, or “wildtype or variant allele”. While these alternative models may be used in certain settings based on the specific SNPs under consideration, they should always be compared to an additive model to confirm that they are appropriate.

Example: “Autosomal dominant inheritance was assumed, as this was the best-fitting genetic model for MUC5B rs35705950 in prior work by Juge et al. and enabled us to pool results via a meta-analytic approach…we also tested whether an additive model outperformed a dominant model⋯” (30)

Are the SNPs weighted within the score?

Weighting should be performed for the SNPs in a GRS, as this accounts for the differential risk that each individual SNP contributes. In modeling complex genetic risk, it would be inaccurate to assume that any specific disease-associated SNP conveys the same amount of risk as another. An unweighted GRS should raise serious questions of validity.

How are the SNPs weighted within the score?

Methods of weighting may vary, but the most common is to weight the number of risk alleles using the allelic effect size of the outcome of interest. The selection of a specific weighting method may be outside the scope of a clinical reader, but methods should clearly state how this was done.

Example: “…weighted by the natural log of the pooled OR.” (30)

How was the effect size estimated?

The effect size used in weighting may be derived from a variety of sources. If GWAS was used to identify SNPs, then often the effect size will be derived from the same GWAS; both single population GWAS and meta-analysis can be used. Alternatively, there may be univariate analyses to derive effect sizes either in a separate (discovery) cohort or in the same cohort. If a separate cohort is used to derive effect sizes, characteristics of that cohort should be evaluated to ensure these are applicable to the GRS study cohort. Differences in disease characteristics and/or genetic ancestry may affect the magnitude of effect of the individual genetic variants (42). This is an active area of genetic epidemiology research. For example, in patients with gout, while some markers demonstrated consistent effects, differential effect sizes were observed for the same genetic variant across populations; this was particularly notable in those with Indigenous Aotearoa New Zealand Māori and Pacific Islander ancestry (43). While the same cohort for GRS evaluation can be used to derive effect sizes, there is potential to overestimate GRS performance with this approach.

Example: “…we calculated pooled ORs and 95% confidence intervals (CI) weighted by variance utilizing a fixed effects model via a published meta-analytic method…” (30)

Were all identified SNPs included or were any removed from the final score?

While not required, some studies will use a negative selection approach to simplify the score by removing SNPs that don’t significantly contribute. Additional terms that may be seen include “backward selection” or “predictor selection”. Exact methods vary, but should be some form of backward selection technique. Forward selection is less preferable because it doesn’t simultaneously assess all candidate variables. If performed, the study may report p values from a significance test during the selection steps, in which case smaller significance level cut-offs (0.05, 0.01) will eliminate more predictors, potentially missing important ones, whereas larger cut-offs (0.20) will increase risk of selecting less important predictors (39). While selection can simplify the score, it can lead to model instability or overfitting, specifically in small datasets where the selected predictors may vary depending on the data used.

Example: “We utilized a backward elimination predictor selection approach…using a cut-off p-value of ≥0.20 by Wald test.” (30)

Was there adjustment for population stratification?

Population stratification is the state where populations are distinguishable by observing genotypes (40). As different subgroups of humans have experienced different stresses and environmental pressures over time, there are detectable differences in populations that provide information about their genetic ancestry. In simplified terms, if one is performing a study looking at genetic risk for a trait that is more common in a certain ancestral group, then one will find that genetic profiles of that group are associated with that trait, rather than the markers which are biologically relevant. In that case, there is the risk of a GRS overpredicting the trait in that group and underperforming in others (40, 41). This must be accounted for in genetic case-control studies. Ideally, a GRS would be selected that was developed and validated in a population of ancestry most similar to that of a given patient. In practicality, the resources required to establish ethnically diverse genetic databases have historically limited large genetic studies to those of European ancestry (42). The techniques by which to do this are beyond the required knowledge base of a clinical rheumatologist. However, there should be a clear description indicating that the authors have addressed this. Adjustment of models by principal components of ancestry derived from genome-wide data is the most common method used, but other terms which would indicate that population stratification was considered include “global ancestry”, “local ancestry”, “population structure”, “genetic similarity”, “admixture”, “multidimensional scaling (MDS)”, or “genomic control”. It is discouraged to use race/ethnicity for proxies of population stratification, as the definition of these social constructs has regional variation and fails to capture underlying genetic similarity (44).

Example: “Using the imputed genotypes from the full cohort and the EIGENSTRAT software program, principal component (PC) analysis showed 10 PCs that explained 70% of variation. These PCs, as well as age and sex, were used to correct the association of the SNPs with RA-ILD for population stratification.” (30)

What method was used to evaluate performance of the GRS?

There are a variety of methods to evaluate performance. Score results can be split into tertiles, quartiles, quintiles, etc., and hazard ratios used to compare the different groups. Another common method is to utilize a regression model containing the GRS to generate a receiver operating characteristic (ROC) curve and subsequent AUC.

Example: “Multivariable logistic regression was used to generate a receiver operating characteristic (ROC) curve with calculation of the area under the curve (AUC).” (30)

Was this compared to a model with clinical factors that did not include the GRS?

In most cases, the main question of interest for the clinician is whether the GRS improves prediction over known risk factors. This should be addressed by comparing the GRS in a model with clinical predictors to a model using the predictors without the GRS. This is commonly done using regression models compared by a nested likelihood ratio test. If this question is not addressed, it becomes difficult to understand relevance of the GRS to the clinical problem. A proposed minimum AUC for clinical applicability is 0.96 (9); if the AUC of the combined model is above this and p value is significant via nested likelihood ratio test against the clinical model, then the GRS should be considered to have clinical potential. The information criteria of the model may also be used for qualitative comparison.

Example: “We performed multivariable logistic regression for RA-ILD with a combined risk score that included the clinical risk factors and final GRS…calculated the AUC…then compared it to the clinical score alone via a nested likelihood ratio test.” (30)

How was the GRS validated?

Validation is crucial in understanding the generalizability of a score. External validation (use of a separate cohort) is the most comprehensive and preferred method. In some circumstances, external validation will not be possible and in that case internal validation should be utilized. Phrases indicating an internal validation approach may include “split-sample”, “cross-validation (k-fold)”, or “bootstrapping”. For the clinician, it should be evident that some type of validation approach was used and whether this was external or internal.

Example: “Internal validation was performed via bootstrapping using 50 different random samples.” (30)

Was calibration performed/reported on the model?

Calibration determines whether the absolute predicted risks by the model are similar to the actual observed risks (45). If calibration is performed, this most commonly will be reported as either “calibration-in-the-large”, “expected:observed ratio”, “calibration slope”, or there may be a “calibration plot” provided with interpretation. Hosmer-Lemeshow goodness of fit testing may be reported. Ideal values and interpretations are shown in Table 4. Following calibration there should also be an indication by the authors that the model was corrected for overfitting if necessary.

Table 4.

Expected values and interpretations for common calibration metrics.

	Ideal Value	Below	Above
Calibration-in-the-large	0	Underestimation of true events	Overestimation of true events
Calibration slope	1	Overfitting of model	Underfitting of model
Expected:observed (E:O) ratio	1	Under-prediction of true events	Over-prediction of true events

Example: “…the bootstrap-calculated shrinkage factor (slope of the calibration curve) was 0.948⋯The expected:observed (E:O) ratio was 1.005…and the calibration-in-the-large (CITL) was −0.006…Following correction via the shrinkage coefficient to account for overfitting…” (30)

Are there clear instructions on how to calculate the score?

A reader should be able to calculate and apply the GRS if genetic data were available. Instructions should be provided both for the GRS and for the regression equation. If available, a GRS calculator can facilitate calculation of results.

Example: “A risk calculator is provided in the Supplementary Material…To calculate ILD risk among RA patients using the VARA-ILD Combined Risk Score model, first calculate the beta-coefficient product for each variable as described…” (30)

Are there recommended cut-points for the GRS or the regression model?

Cut-points, if provided, can give a useful idea of the potential clinical usefulness of a GRS. Ideally, the authors may show a variety of possible cut-points to demonstrate performance (i.e. sensitivity, specificity) at different levels. This may help the clinician envision how a GRS could be applied in the clinical setting.

Example: “With a sensitivity of 91.5% at a cut-point of 0.05, the VARA-ILD Combined Risk Score would have excluded ~500 individuals (~20%) from unnecessary testing⋯” (30)

Are these findings generalizable to the desired clinical setting?

By answering the prior questions, the reader should have a sense for whether the study population is similar to their own patient population for application of the GRS. This includes the evaluation of cohorts used for SNP identification and GRS derivation, and the validation methods performed. If considering applying a GRS to a population with different ancestry or disease characteristics than the original cohort, it would be important to externally validate this score in the group of interest to ensure the GRS accurately describes risk in settings with different linkage disequilibrium patterns and allele frequencies, effect sizes, or disease burden. As demonstrated in RA, PRSs based on multi-ancestry GWAS perform superiorly when applied to separate populations as compared to single-ancestry GWAS (31). Historically, GRS studies have not specifically commented on generalizability of findings, but it has been suggested that studies should report metrics that reflect how a GRS would perform in the population of interest (9). This includes the detection rate, false positive rate, likelihood ratio for the test, and number needed to genotype (analogous to number needed to test [NNT]); the inclusion of these allows for the comparison with other diagnostic modalities using statistical concepts with which clinicians may be more familiar.

Conclusion

By using the systematic approach described above, a clinical rheumatologist will be able to critically appraise a GRS research article and understand the general validity of the methods. If the methods are sound, the reader must then ask themselves a final question: “Would being able to identify/predict this with genetics change clinical care?”. As genetic analysis becomes more accessible and cost-effective, it will become increasingly important to recognize the clinical applicability of GRSs and identify those scores of highest utility for patient care. GRSs have already shown exciting possibilities for assisting in diagnostic evaluation (35), particularly as we enter the era of artificial intelligence-aided decision-making where the opportunity to incorporate personalized GRS predictions is primed. GRSs will likely be augmented by parallel inclusion of other ‘omic data (e.g. transcriptomics, proteomics, and epigenomics from appropriate tissues). As the scope of GRS applications continues to expand, new predictive techniques based on GRSs emerge, and artificial intelligence-based tools begin to enter the clinical workspace, we as a rheumatology community need to be prepared to employ these techniques carefully and effectively, supported by a sound understanding of the techniques. Future directions in critical evaluation could include the validation of a scoring system for the methodology and performance of a GRS.