The appropriate use of risk scores in the prediction of atrial fibrillation

Atrial fibrillation (AF) is the most common arrhythmia encountered in clinical practice (1). It is associated with a significant risk for several adverse cardiovascular outcomes, including stroke (2), myocardial infarction (3), heart failure (4), and mortality (5). Additionally, AF is associated with significant cost to the health care system, with annual projected costs between $6 and $26 billion dollars (6). The aforementioned complications and financial burden associated with this arrhythmia underscore the importance of accurate AF risk assessment, as this will allow for the development of targeted preventive strategies.

This need for the accurate prediction of AF has given rise to the development of several scoring systems from population-based cohort studies (7). Risk scores have been developed in the Framingham Heart Study (FHS) (8), the Atherosclerosis Risk In Communities (ARIC) study (9), and the Women’s Health Study (WHS) (10). However, the risk scores developed from these individual cohorts were limited in their predictive ability, as each cohort varied widely in the diversity (e.g., age, sex, and race/ethnicity) of recruited participants. Accordingly, the Cohorts for Aging and Research in Genomic Epidemiology (CHARGE)-AF consortium derived a 5-year predictive model to address some of these limitations (11). This score used pooled data from 18,556 participants of the FHS, the Cardiovascular Health Study, and ARIC, and included the following characteristics: age, race, height, weight, systolic and diastolic blood pressure, current smoking, treatment of hypertension, diabetes, and history of myocardial infarction and heart failure. The score was then validated in a sample from the Age, Gene and Environment-Reykjavik study (AGES) and the Rotterdam Study (RS), and it demonstrated acceptable discrimination in these cohorts. Additionally, the CHARGE-AF model has been validated in the EPIC-Norfolk cohort (12), in a large multi-ethnic patient population in New York City (13), and in the Multi-Ethnic Study of Atherosclerosis (MESA) (14), providing evidence that this risk prediction tool performs well in diverse populations. The above models and risk scores were derived with the specific aim of predicting AF incidence, and the decision to include, or exclude, predictors was largely based on prior knowledge of well-known AF risk factors and the association of those predictors with AF. Furthermore, each score, particularly CHARGE-AF, has been validated in external cohorts, confirming its ability to accurately predict AF across diverse settings.

The ability of models originally derived to predict AF-related complications, particularly stroke, to predict the occurrence of AF also has been explored. The CHADS₂ (congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, stroke/transient ischemic attack) score (15), and its later version, CHA₂DS₂-VASc (Congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, stroke/transient ischemic attack, vascular disease, age 65–75 years, and sex category) (16), were originally developed to predict stroke among patients with AF. These scores aid clinicians in the selection of appropriate anticoagulation strategies. Recent reports have suggested that the CHADS₂ and CHA₂DS₂-VASc risk scores also are able to predict incident AF (17,18). Notably, many of the risk factors included in CHADS₂ and CHA₂DS₂-VASc (older age, diabetes, hypertension, heart failure, vascular disease) are well-known AF risk factors, likely explaining their ability to predict AF. If CHADS₂ and CHA₂DS₂-VASc adequately identify individuals who are high risk of developing AF, this would obviate the need to use models specifically derived to predict AF.

A recent report published in the American Heart Journal aimed to directly compare the predictive ability and calibration of the CHARGE-AF and CHA₂DS₂-VASc risk scores for the prediction of incident AF (19). For this analysis, Christophersen et al. used data from 4,548 (mean age, 63.9±10.6 years, 56% women) participants form the original FHS and Framingham Offspring Cohort applying a pooled-examination approach and standard statistical techniques (Wald χ² statistic to assess model fit, the C-statistic to assess model discrimination, and the Hosmer-Lemeshow (HL) χ² statistic to assess model calibration). The authors hypothesized that the CHARGE-AF risk score would have better model performance in AF prediction than CHA₂DS₂-VASc in a community-based cohort. The results confirmed their initial hypothesis: compared with CHA₂DS₂-VASc, the CHARGE-AF model demonstrated better fit (Wald χ²=403 vs. 209, both with 1 df), improved discrimination (C-statistic =0.757; 95% CI, 0.741–0.762 vs. C-statistic =0.712; 95% CI, 0.693–0.731), and better calibration (HL χ²=5.6; P= 0.69 vs. HL χ²=28.5; P<0.0001) in the prediction of AF. Due to the fact that women <65 years of age with lone AF have a low risk of stroke, a secondary analysis was performed assigning a CHA₂DS₂-VASc of 0 to all women <65 years who scored 0 on all other categories. When scoring these women with lone AF as CHA₂DS₂-VASc =0, the model fit (Wald χ²=288) and discrimination (C-statistic =0.730; 95% CI, 0.713–0.747) improved, yet calibration was reduced (HL χ²=35.5; P<0.0001). A secondary analysis also was performed in which sex was excluded from the CHA₂DS₂-VASc score, as women have been suggested to have the same or lower risk of AF compared with men (20). This resulted in improved discrimination (C-statistic =0.741; 95% CI, 0.724–0.758), but the model fit (Wald χ²=360) and calibration (HL χ²=28.5; P<0.0001) remained inferior to values reported for the CHARGE-AF score. Interactions were not detected by age or sex, and similar results were observed in sex-stratified models.

The results from Christophersen et al. (19) are consistent with those from a recent publication from the MESA cohort (14). In the MESA analysis, which included a multi-ethnic sample of 6,663 adults in the United States without prior cardiovascular disease, the C-statistic for the CHARGE-AF score was 0.779 (95% CI, 0.744–0.814), compared with a C-statistic of 0.695 (95% CI, 0.654–0.735) for the CHA₂DS₂-VASc score.

Overall, the findings from the FHS and MESA studies confirm that the CHARGE-AF risk score is superior to the CHA₂D₂-VASc risk score in the prediction of incident AF in community-based cohorts. The FHS analysis also offers insight into the use of the CHA₂DS₂-VASc score to predict AF, as the discriminative ability of the CHA₂DS₂-VASc risk score improved when sex category was removed from the model. Female sex is associated with a higher risk for stroke among patients with AF (16), yet women are less likely to develop the arrhythmia compared with men (though this sex difference disappears once differences in AF risk factors between men and women, including height, are considered) (21). Accordingly, female sex was not included in the CHARGE-AF model, as this tool was developed with the intention of predicting incident AF and not its complications. Additionally, the model fit (measured by Wald χ²) and calibration (measured by HL χ²) for CHA₂DS₂-VASc without sex category remained inferior to that of CHARGE-AF, highlighting the perils of using risk scores for the prediction of outcomes other than for what the score was originally intended.

An additional limitation of the CHA₂DS₂-VASc score for the prediction of AF compared with the CHARGE-AF model is the absence of information on the actual risk of AF associated with a particular value of the CHA₂DS₂-VASc score. For example, we know that, based on the original study in which the CHA₂DS₂-VASc score was developed, a CHA₂DS₂-VASc of 2 equates to a 2% annual risk of stroke (16). However, the risk of AF associated with a comparable CHA₂DS₂-VASc score is unknown, and we are unable to derive this information from the C-statistic. In contrast, the CHARGE-AF model (as well as the other AF-specific models) provides an actual estimate of AF risk over a 5- to 10-year period.

The findings of Christophersen and colleagues have relevant clinical implications, as the burden that AF places on the health care system will increase with the expected growth in individuals 65 years and older (1,22). These projections expose the urgent need for the development of AF preventive strategies. However, before targeted screening measures or the identification of high-risk patients for clinical trial enrollment are feasible, we must be able to appropriately select those who are more likely to benefit from such efforts. The success of future research aiming to prevent AF will ultimately rely on the appropriate selection of participants who are deemed high risk. Therefore, risk scores such as CHARGE-AF that were originally developed to identify persons who are high risk for AF development are of paramount importance to aid current and future preventive research endeavors. Using the CHADS₂ or CHA₂DS₂-VASc scores for this purpose, though an attractive alternative due to its simplicity should be avoided since these scores have suboptimal performance in the prediction of AF. Finally, although the CHARGE-AF score has demonstrated its predictive value across a wide range of populations, additional work is needed to determine the role that other clinical factors, blood biomarkers, and genetic information have in predicting AF.