Predicted Versus Observed Major Adverse Cardiac Event Risk in Women With Evidence of Ischemia and No Obstructive Coronary Artery Disease: A Report From WISE (Women's Ischemia Syndrome Evaluation)

Abstract

Background

Primary prevention risk scores are commonly used to predict cardiovascular (CVD) outcomes. The applicability of these scores in patients with evidence of myocardial ischemia but no obstructive coronary artery disease is unclear.

Methods and Results

Among 935 women with signs and symptoms of ischemia enrolled in WISE (Women's Ischemia Syndrome Evaluation), 567 had no obstructive coronary artery disease on angiography. Of these, 433 had had available risk data for 6 commonly used scores: Framingham Risk Score, Reynolds Risk Score, Adult Treatment Panel III, Atherosclerotic Cardiovascular Disease, Systematic Coronary Risk Evaluation, Cardiovascular Risk Score 2. Score‐specific CVD rates were assessed. For each score, we evaluated predicted versus observed event rates at 10‐year follow‐up using c statistic. Recalibration was done for 3 of the 6 scores. The 433 women had a mean age of 56.9±9.4 years, 82.5% were white, 52.7% had hypertension, 43.6% had dyslipidemia, and 16.9% had diabetes mellitus. The observed 10‐year score‐specific CVD rates varied between 5.54% (Systematic Coronary Risk Evaluation) to 28.87% (Framingham Risk Score), whereas predicted event rates varied from 1.86% (Systematic Coronary Risk Evaluation) to 6.99% (Cardiovascular Risk Score 2). The majority of scores showed moderate discrimination (c statistic 0.53 for Atherosclerotic Cardiovascular Disease and Systematic Coronary Risk Evaluation; 0.78 for Framingham Risk Score) and underestimated risk (statistical discordance −58% for Adult Treatment Panel III; −84% for Atherosclerotic Cardiovascular Disease). Recalibrated Reynolds Risk Score, Atherosclerotic Cardiovascular Disease, and Framingham Risk Score had improved performance, but significant underestimation remained.

Conclusions

Commonly used CVD risk scores fail to accurately predict CVD rates in women with ischemia and no obstructive coronary artery disease. These results emphasize the need for new risk assessment scores to reliably assess this population.

Nonstandard Abbreviations and Acronyms

CVD

cardiovascular disease

INOCA

ischemia and no obstructive coronary artery disease

FRS

Framingham risk score

RRS

Reynolds Risk score

ATP III‐FRS

Adult Treatment Panel III

ASCVD

Atherosclerotic Cardiovascular Disease

SCORE

Systematic Coronary Risk Evaluation

QRISK2

Cardiovascular Disease Risk Algorithm Version 2

MI

myocardial infarction

CAD

coronary artery disease

VA‐Cart

Veterans Administration Cardiovascular Assessment Reporting and Tracking System

NCDR

National Cardiac Data Registry

MESA

Multi‐Ethnic Study of Atherosclerosis

WISE

Women's Ischemia Syndrome Evaluation

REGARDS

Reasons for Geographic and Racial Differences in Stroke

CLINICAL PERSPECTIVE

What Is New?

• In women with myocardial ischemia but no obstructive coronary artery disease, commonly used primary prevention risk scores fail to accurately predict cardiovascular event rates.

What Are the Clinical Implications?

• Women with ischemia and no obstructive coronary artery disease should be considered higher risk, and primary prevention risk tools should not be applied.

• These results emphasize the need for new specifically tailored, stepwise comprehensive risk assessment tools addressing ischemia and no obstructive coronary artery disease and other novel risk factors in this increasingly prevalent population.

Introduction

Cardiovascular disease (CVD) remains the leading cause of death among women in the United States with the majority of these deaths attributable to ischemic heart disease.1 Patients with signs and symptoms of ischemia but no obstructive coronary artery disease, now termed INOCA, are increasingly prevalent.2, 3 INOCA is a separate condition from myocardial infarction (MI) with normal coronaries or from MI as defined in the Fourth Universal Definition in that it is a stable disorder without a discrete rise and/or fall of troponin levels. INOCA includes patients with “normal” appearing coronary arteries, defined as luminal stenosis 0% or <20% and those with nonobstructive coronary artery disease, defined as luminal stenosis ≥20% but <50%.4, 5, 6 The prevalence of INOCA is higher in women than in men.7

Strong evidence now indicates that INOCA patients are at elevated risk for major adverse cardiovascular events, including death.8, 9, 10, 11, 12, 13 In a study of 540 women with INOCA, heart failure was the most frequent event with an observed 10‐fold higher event rate compared with asymptomatic community‐based women.9 Moreover, patients with nonobstructive coronary artery disease (CAD) in 3 coronary arteries have an annual risk for MI and death similar to that of patients with single‐vessel obstructive CAD.11 Despite this evidence, INOCA patients are often discharged from specialty care and receive less intensive primary or secondary prevention guideline‐directed medical therapy after visualization of nonobstructive CAD on invasive angiography.14, 15, 16 Overall estimates in women and men from the VA‐CART (Veterans Administration Cardiovascular Assessment Reporting and Tracking System),17 the NCDR (National Cardiac Data Registry), and the National Heart, Lung, and Blood Institute–sponsored WISE (Women's Ischemia Syndrome Evaluation)8 databases indicate that there are at least 3 to 4 million American women and men with stable INOCA. Incurred healthcare costs are similar to obstructive CAD.

In both primary and secondary prevention, the assessment of CVD risk and prevention of future events have major public health implications. As a result of variability in clinician‐estimated likelihood of outcomes, a number of stepwise, multivariable risk models have been developed18 to classify risk and guide therapies. Currently at least 6 risk scores, recommended by different guidelines, are available for primary prevention CVD risk assessment. The Framingham risk score (FRS),19 the Reynolds Risk Score (RRS),20 the Adult Treatment Panel III (ATP III‐FRS) risk score (a modified FRS),21 the Atherosclerotic Cardiovascular Disease (ASCVD) risk score (also referred to as the Pooled Cohort Equations or PCE score),22 the Systematic Coronary Risk Evaluation (SCORE),23 and the QRISK2 (Cardiovascular Disease Risk Algorithm Version 2)24 address mainly asymptomatic patients. The newer scores were derived from more racially and geographically diverse cohorts but have been shown to overestimate or underestimate risk in independent cohorts.25, 26, 27 Both the overprediction and underprediction of events have relevance particularly when treatment decisions are made. Specifically, for ASCVD, a threshold of 7.5% in 10 years triggers consideration of pharmacotherapy (statin treatment) in patients above this risk level in the current guidelines.28 The 6 risk scores developed during the past 2 decades have a common core of variables; age, sex, systolic blood pressure, and cholesterol levels are almost uniformly part of the scores (Table 1).20, 24, 29 However, their outcome definitions vary significantly. This is of particular interest because stroke determines CVD risk to a greater degree in women than men so scores that focus solely on coronary heart disease will differ in risk assessment from those that include stroke as an outcome.

Table 1.

Primary Prevention Risk Scores Comparison

	FRS 2008	ATP III‐FRS 2002	SCORE 2003	RRS 2007	Q‐Risk2 2008	ASCVD 2013
Outcome	Angina, MI, death from CHD, stroke, TIA, peripheral vascular disease, and heart failure	MI, death from CHD	Death from CVD	MI, stroke, coronary revascularization, death from CHD	Angina hospitalization, MI, stroke	MI, stroke, death from CHD
Sex	+	+	+	+	+	+
Age	+	+	+	+	+	+
Ethnicity					+	+
Chronic disease					+ (CKD, RA)
BMI					+
Total cholesterol	+	+	+	+		+
HDL‐C	+	+				+
TC:HDL‐C ratio			+		+
LDL‐C
Systolic blood pressure	+	+	+	+	+	+
Smoking status	+	+	+	+	+	+
Diabetes mellitus	+				+	+
Hypertensive treatment	+	+			+	+
Family history of CAD				+
hsCRP				+
Hb A1C				+a

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; ATP III‐FRS, Adult Treatment Panel III risk score (a modified FRS); BMI, body mass index; CAD, coronary artery disease; CHD, congestive heart failure; CKD, chronic kidney disease; CVD, cardiovascular disease; FRS, Framingham Risk Score; Hb A1C, hemoglobin A1C; HDL‐C, high‐density lipoprotein cholesterol; hsCRP, high sensitive C reactive protein; LDL‐C, low‐density lipoprotein cholesterol; MI, myocardial infarction; QRISK2, Cardiovascular Disease Risk Algorithm Version 2; RA, rheumatoid arthritis; RRS, Reynolds Risk Score; SCORE, Systematic Coronary Risk Evaluation; TC, total cholesterol; and TIA, transient ischemic attack.

^aIncluded if patient has diabetes mellitus.

Our aim was to test the performance of commonly used primary prevention scores in women with INOCA. We chose specifically to examine primary prevention as most patients with no significant obstructive CAD on angiography have traditionally been dismissed from specialty care with low rates of therapy prescribed,14, 15, 16, 30 such that they are managed mainly for their CVD risk factors and not as a spectrum of ischemic heart disease.

Methods

A total of 935 women without a history of CAD and signs and symptoms of ischemia were enrolled in the WISE (NCT00000554) between September 1996 to March 2000 and were followed by site personnel through March 2006 for death, cardiovascular death, nonfatal MI, nonfatal stroke, hospitalization for angina,31 hospitalization for heart failure, and revascularization. The protocol was approved by the institutional review boards at each site (University of Alabama at Birmingham, University of Florida at Gainesville, University of Pittsburgh Medical Center, and Allegheny General Hospital in Pittsburgh), and all participants provided written informed consent. The data that support the findings of this study are available from the corresponding author upon reasonable request. A National Death Index search was initiated to assess cardiovascular death data through December 2007 with the goal of enhancing the precision of mortality estimates. WISE inclusion criteria required clinician‐determined signs and symptoms of ischemia for which a clinically indicated invasive coronary angiogram was conducted.32 Because of the anticipation of provocative testing in the general WISE cohort, women with a recent (6 weeks) history of acute coronary syndrome or MI were excluded. Among these 935 women, 433 (46.3%) had no history of CAD, no obstructive CAD on angiography, were aged 40 to 79, and had available data for risk assessment according to the majority of the commonly used scores (Table 1)20, 24, 29 and were included in our analysis. Inclusion and exclusion criteria were selected by clinicians broadly for all the models. They are not specific to each model. A total of 346 women had all variables available for RRS calculation. All variables used in risk score calculation were directly available except the level of glycated hemoglobin, which therefore was calculated from fasting glucose level as previously published ([glucose level+46.7]/28.7).33, 34 The published formula was applied for each of the risk models to calculate the risks used in this analysis. QRISK2 was computed from a calculator, and ATP III‐FRS used a published table instead of a formula. The models were not refit to this cohort, but there was a recalibration done for the models with available formulas.

In the recalibration, the original risk score was fit as a covariate in a Cox proportional hazards model for the outcomes of each model. The coefficient (calibration slope) and baseline hazard estimate from the new model were used to rescale the original risk scores. We used an exponential distribution for the calibration. The linear predictor from the new model (original linear predictor times the calibration slope, LP) and the baseline hazard function at 10 years (s10) were used in the formula 1−s10^exp (LP−mean [LP]) to get a recalibrated 10‐year risk. These were then scaled in the same way as the original scores to the actual follow‐up time as in DeFilippis et al.27

Individual score‐specific CVD event rates were assessed. When comparing observed to expected event rates calculated from an individual risk prediction tool (eg, RRS), the observed event rate is calculated specific to the outcome for which the risk score was designed to predict. Therefore, the observed event rates vary by individual risk score being evaluated. For the majority of scores, the outcome was a composite that included death with a follow‐up period of 10 years. The maximum follow‐up for WISE women was 8.16 years for nonfatal events and an additional 2 years using a National Death Index search. Because the risk scores are 10‐year risk, in any woman with >10 years of follow‐up data, mortality was truncated to 10 years to match. QRISK2 did not include death so the National Death Index follow‐up was not included, and the maximum follow‐up duration was 8.16 years.

For each subject with <10 years of follow‐up, including those who died, the 10‐year risk estimate was lowered to correspond to their length of follow‐up using an exponential survival function to scale the risk score. If A10 denotes the 10‐year proportion with events according to a risk score, then the 1‐year proportion is A1=−ln(1−A10)/10. So for a person with 8.5 years of follow‐up, A8.5=1−exp(−A1×8.5).27

The performance of the prediction models constructed was assessed with measures of discrimination and calibration. Predicted risks were compared with the observed outcomes using the published models for computation of each of the predicted risks.8, 16, 17, 18, 19, 20 The observed risk equaled 1 minus the Kaplan–Meier product limit estimator of the survival for the whole cohort. The data include not only counts of the number of events but also the times when events occurred, so the analysis does not simply tally the number of events. The product limit estimator is a step function with jumps at the observed event times, and the size of the jumps depends on the number of observed events and the number of censored observations prior to that time. For each time with observed events during follow‐up, the Kaplan–Meier estimator is updated to get a cumulative risk for the entire follow‐up period. Risk was evaluated separately for each risk prediction score. Different scores combined events in different combinations using the earliest event observed for a person. Therefore, if a risk score did not use a certain event type, that event would not have been considered for that prediction and that individual might be categorized differently than another risk score. For example, women with hospitalization for angina were considered to have an adverse event in the FRS, but did not have an adverse event in the ASCVD risk score (unless they had another event type that would qualify them for that ASCVD score). Thus, a woman at high risk for adverse events using the FRS may not necessarily be considered high risk in the ASCVD score because of these differences.

The primary measure of discrimination used a transformation of Somer's rank correlation for censored data, the c index, which is similar to the area‐under‐the curve measure from receiver operating characteristic curves. For individuals who had events, the times of the events were incorporated into these c statistics. For a given individual with an event at a given time, their predicted risk is compared to an individual with an event at a later time, and they ought to have a higher predicted risk for the pair to be in agreement. If the predicted risk is lower for the individual with an earlier event time, the pair of individuals are not in agreement. The individuals with events are compared across all possible pairs in the data to determine the c statistic for that particular risk score. Calibration plots were produced plotting the predicted survival probabilities (1 minus predicted risk) versus the observed Kaplan–Meier values at the observed times. Linear regression of the resultant scatter plots were used to estimate the calibration slope and intercepts. In addition to c statistics and calibration plots, the observed percentages of subjects with events were compared to the percentages of predicted events. The number of predicted events was calculated as the sum of predicted risk scores and was divided by the cohort size to get a proportion for predicted events. The difference between predicted minus observed event percentages were tabulated for all subjects under “signed difference,” and the signed differences divided by the observed percentage were the “discordances.” These additional discordance measures were also tabulated for the following 4 predicted risk categories: 0 to 5, 5 to 7.5, 7.5 to 19.9, and 20 or above. According to the 2019 American College of Cardiology/American Heart Association guideline on the primary prevention of cardiovascular disease, low risk is <5%, borderline risk is 5% to 7.5%, intermediate risk is 7.5% to 20%, and high risk is >20%.35 The RMS package in R (R Foundation for Statistical Computing, Vienna, Austria) was used for analysis.

Results

Baseline characteristics are described in Table 2. The 433 women had a mean age of 56.9±9.4 years, and a majority (82.5%) were white. Just more than half of the women had hypertension at baseline, and statins were prescribed for 15.9% of the patients at baseline. In all 433 women during the entire follow‐up, there were 24 CV deaths observed, 9 had MI events, 20 had heart failures, 17 had strokes, 19 had percutaneous transluminal coronary angioplasty (PTCA), 5 had coronary artery bypass graft (CABG), and 89 had hospitalizations for angina.

Table 2.

Baseline Demographic and Clinical Characteristics of INOCA Patients

Baseline Characteristics	WISE Subjects (n=433)
Age, mean±SD, y	56.9±9.4
BMI, mean±SD, kg/m2	29.7±6.5
Race, n (%) White African Americans Other	357 (82.5) 72 (16.6) 4 (0.9)
History of hypertension, n (%)	228 (52.7)
History of diabetes mellitus, n (%)	73 (16.9)
History of dyslipidemia, n (%)	175 (43.6)
Family history of CAD, n (%)	278 (65.4)
Smoking history, n (%) current former	81 (18.7) 134 (31)
History of chronic renal dysfunction (creatinine >1.5 mg/dL), n (%)	11 (2.6)
Systolic blood pressure, mean±SD, mm Hg	136.1±20.6
Diastolic blood pressure, mean±SD, mm Hg	80.3±17.3
hsCRP, mean±SD, mg/dL	0.72±1.3
Treatment with ACE‐I, n (%)	87 (20.1)
Treatment with ARB, n (%)	13 (3)
Treatment with CCB, n (%)	98 (22.6)
Treatment with BB, n (%)	133 (30.8)
Treatment with diuretics, n (%)	107 (24.7)
Treatment with statins, n (%)	69 (15.9)
Treatment with aspirin, n (%)	207 (48)
Ventriculography—ejection fraction, %±SD	66.3±9.5
Total cholesterol, mean±SD, mg/dL	196.8±44.5
HDL‐C, mean±SD, mg/dL	55.2±13
LDL‐C, mean±SD, mg/dL	115.9±40.7
Triglycerides, mean±SD, mg/dL	135.3±87.4

ACE‐I indicates angiotensin converting enzyme inhibitor; ARB, angiotensin receptor blockers; BB, beta blockers; BMI, body mass index; CAD, coronary artery disease; CCB, calcium channels blockers; HDL‐C, high‐density lipoprotein cholesterol; hsCRP, high sensitive C reactive protein; INOCA, ischemia and no obstructive coronary artery disease; LDL‐C, low‐density lipoprotein cholesterol; and WISE, Women's Ischemia Syndrome Evaluation.

Included if patient has diabetes mellitus.

Depending on each score‐specific outcome definition, the observed 10‐year risk of events in this INOCA population ranged from 5.54% to 28.87%, in accordance with previous data showing a higher risk than previously thought. Table 3 summarizes the risk score distribution and the observed risk in women with INOCA.

Table 3.

Summary of Risk Score Distributions and Observed Risk in INOCA Women

Model	No.	Risk Score, Median (Range)	Recalibrated Score, Median (Range)	Observed Event Count, n (%)	Follow‐Up Years, Median (Range)
RRS (MI, stroke, coronary revascularization, cardiovascular death)	346	2.24 (0—50.1)	8.06 (0, 59.01)	58 (16.76)	8.40 (0—10)
FRS CVD (angina hospitalization, MI, stroke, CHF, cardiovascular death)	433	4.99 (0—50.26)	21.38 (0.02, 56.95)	125 (28.87)	6.13 (0.01—10)
ASCVD (MI, stroke, cardiovascular death)	433	0.04 (0—34.13)	9.89 (0.01, 22.87)	44 (10.16)	8.45 (0.01—10)
SCORE (cardiovascular death)	433	0.89 (0—20.56)	···	24 (5.54)	8.57 (0.01—10)
ATP III‐FRS (MI, cardiovascular death)	433	1.63 (0—29.34)	···	29 (6.70)	8.53 (0.01—10)
QRISK2 (angina hospitalization, MI, stroke)	433	5.29 (0.01—41.43)	···	104 (24.02)	5.50 (0.01—9)

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; ATP III‐FRS, Adult Treatment Panel III risk score; CHF, congestive heart failure; CVD, cardiovascular disease; FRS, Framingham Risk Score; hsCRP, high sensitive C reactive protein; INOCA, ischemia and no obstructive coronary artery disease; MI, myocardial infarction; QRISK2, Cardiovascular Disease Risk Algorithm Version 2; RRS, Reynolds Risk Score; and SCORE, Systematic Coronary Risk Evaluation.

When comparing the predicted versus the observed event rates using individual score outcomes, FRS predicted an event rate of 6.87%, whereas the observed event rate was 4 times greater at 28.87%. Similarly, the predicted rate for RRS was 3.28%, whereas the observed rate was at 12.14%. The QRISK score, which is updated every year to include variables such as chronic diseases with cardiovascular implications (eg, rheumatoid arthritis, chronic kidney disease), also underestimated risk (6.99% as compared to the observed rate of events of 24.02%). The 2 major risk scores commonly used in Europe and the United States, SCORE and ASCVD, underestimated the risk by 66.43% and 83.96%, respectively. All of the scores showed moderate to low discrimination (c statistics as low as 0.53 for ASCVD and SCORE and as high as 0.78 for FRS) (Table 4). Recalibration was possible for 3 of the 6 available scores. When recalibrated scores were used, discrimination improved, but all 3 scores still underestimated risk by as much as 27.68% for FRS and 10.43% for ASCVD (Table 5). Systematic discordance and underestimation of predicted versus observed event rates are also reflected in calibration plots (Figure).

Table 4.

Predicted versus Observed Rate of Events for Primary Prevention Scores in Women With INOCA With the Original Risk Scores

Original Risk Score	Predicted Events, n (%)	Observed Events, n (%)	Signed Difference, %	Discordance, %	c Statistic
RRS, n=346	11.34 (3.28)	42 (12.14)	−8.86	−72.98	0.77
QRISK II	30.28 (6.99)	104 (24.02)	−17.03	−70.90	0.77
SCORE	8.04 (1.86)	24 (5.54)	−3.68	−66.43	0.53
ASCVD	7.06 (1.63)	44 (10.16)	−8.53	−83.96	0.53
FRS CVD	29.73 (6.87)	125 (28.87)	−22	−76.20	0.78
ATP III‐FRS	12.17 (2.81)	29 (6.7)	−3.89	−58.06	0.61

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; ATP III‐FRS, Adult Treatment Panel III risk score; CVD, cardiovascular disease; FRS, Framingham Risk Score; INOCA, ischemia and no obstructive coronary artery disease; QRISK2, Cardiovascular Disease Risk Algorithm Version 2; RRS, Reynolds Risk Score; and SCORE, Systemic Coronary Risk Evaluation.

Table 5.

Predicted versus Observed Rate of Events for Primary Prevention Scores in Women With INOCA With Recalibrated Risk Scores

Recalibrated Risk Score	Predicted Events, n (%)	Observed Events, n (%)	Signed Difference, %	Discordance, %	c Statistic
RRS, n=346	33.17 (9.59)	42 (12.14)	−2.55	−21	0.8
ASCVD	39.42 (9.1)	44 (10.16)	−1.06	−10.43	0.91
FRS CVD	90.43 (20.88)	125 (28.87)	−7.99	−27.68	0.91

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; CVD, cardiovascular disease; FRS, Framingham Risk Score; INOCA, ischemia and no obstructive coronary artery disease; and RRS, Reynolds Risk Score.

Figure 1. Risk score specific predicted vs observed rate of events, calibration plots.

Dotted line indicates reference line for equal predicted and observed risk. Solid line indicates observed risk. ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; ATP III‐FRS, Adult Treatment Panel III risk score (a modified FRS); CVD, cardiovascular disease; FRS, Framingham Risk Score; QRISK2, cardiovascular risk score; RRS, Reynolds Risk Score; and SCORE, Systematic Coronary Risk Evaluation.

Related to clinically relevant risk categories (0%–<5%, 5%–<7.5%, 7.5–20%, and >20%) that guide treatment decisions, 5 of the 6 scores classified the majority of patients as being at low risk for CVD events (Table 6). From 433 women, 217 (50%) were classified as low risk by FRS and as much as 389 (90%) and 395 (91%) by SCORE and ASCVD, respectively. In contrast, only 213 (49%) women were classified as low risk by QRISK2. Significant discordance was seen between predicted and observed rates of events in all risk categories. Although the predicted rate of events in low‐risk categories was 0.22% for ASCVD and 1.19% for SCORE, their actual observed risk was as high as 9.62% for ASCVD and 4.88% for SCORE. From the total observed event rates, a majority of events were seen in those women classified as low risk: 68% of observed events were seen in women classified as low risk (0–<5%) by FRS, 60.34% for RRS, 86.40% for ASCVD, 79% for SCORE, 96% for ATP III‐FRS, and 65.4% for QRISK2. Recalibration of the RRS, ASCVD, and FRS improved risk stratification, with a majority of women being reclassified as high risk (>7.5%) for all 3 scores (Table 7).

Table 6.

Predicted versus Observed Rate of Events in Clinically Relevant Risk Categories With the Original Risk Scores

Original Risk Score	Total, n	Observed Events, n (%)	Predicted Events, n (%)	Signed Difference (Absolute)	Signed Difference, %	Discordance, %
RRS
0 to <5	281	35 (12.46)	5.46 (1.94)	−29.54	−10.52	−84.43
5 to <7.5	37	6 (16.22)	2.18 (5.89)	−3.82	−10.33	−63.69
7.5 to 20	25	0 (0)	2.67 (10.68)	2.67	10.68	···
≥20	3	1 (33.33)	1.03 (34.33)	0.03	1	3
QRISK II
0 to <5	213	68 (31.92)	3.93 (1.85)	−64.07	−30.07	−94.20
5 to <7.5	61	10 (16.39)	3.77 (6.18)	−6.23	−10.21	−62.29
7.5 to 20	140	24 (17.14)	17.5 (12.5)	−6.5	−4.64	−27.07
≥20	19	2 (10.53)	5.07 (26.68)	3.07	16.15	153.37
SCORE
0 to <5	389	19 (4.88)	4.61 (1.19)	−14.39	−3.69	−75.61
5 to <7.5	29	2 (6.9)	1.73 (5.97)	−0.27	−0.93	−13.48
7.5 to 20	14	3 (21.43)	1.5 (10.71)	−1.5	−10.72	−50.02
≥20	1	0 (0)	0.21 (21)	0.21	21	···
ASCVD
0 to <5	395	38 (9.62)	0.85 (0.22)	−37.15	−9.40	−97.71
5 to <7.5	9	4 (44.44)	0.56 (6.22)	−3.44	−38.22	−86
7.5 to 20	15	2 (13.33)	2.1 (14)	0.1	0.67	5.03
≥20	14	0 (0)	3.56 (25.43)	3.56	25.43	···
FRS CVD
0 to <5	217	85 (39.17)	4.7 (2.17)	−80.3	−37	−94.46
5 to <7.5	59	14 (23.73)	3.58 (6.07)	−10.42	−17.66	−74.42
7.5 to 20	136	24 (17.65)	15.82 (11.63)	−8.18	−6.02	−34.11
≥20	21	2 (9.52)	5.63 (26.81)	3.63	17.29	181.62
ATP III‐FRS
0 to <5	360	23 (6.39)	5.84 (1.62)	−17.16	−4.77	−74.65
5 to <7.5	41	4 (9.76)	2.4 (5.85)	−1.6	−3.91	−40.06
7.5 to 20	29	2 (6.9)	3.2 (11.03)	1.2	4.13	59.86
≥20	3	0 (0)	0.73 (24.33)	0.73	24.33	···

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; ATP III‐FRS, Adult Treatment Panel III risk score; CVD, cardiovascular disease; FRS, Framingham Risk Score; QRISK2, Cardiovascular Disease Risk Algorithm Version 2; RRS, Reynolds Risk Score; and SCORE, Systematic Coronary Risk Evaluation.

Table 7.

Predicted versus Observed Rate of Events in Clinically Relevant Risk Categories With Recalibrated Risk Scores

Recalibrated Risk Score	Total n	Observed Events, n (%)	Predicted Events, n (%)	Signed Difference (Absolute)	Signed Difference, %	Discordance, %
RRS
0 to <5	107	24 (22.43)	2.27 (2.12)	−21.73	−20.31	−90.55
5 to <7.5	53	3 (5.66)	3.24 (6.11)	0.24	0.45	7.95
7.5 to 20	157	14 (8.92)	19.87 (12.66)	5.87	3.74	41.93
≥20	29	1 (3.45)	7.8 (26.9)	6.8	23.45	679.71
ASCVD
0 to <5	98	23 (23.47)	2.13 (2.17)	−20.87	−21.30	−90.75
5 to <7.5	47	8 (17.02)	2.95 (6.28)	−5.05	−10.74	−63.10
7.5 to 20	261	13 (4.98)	28.44 (10.9)	15.44	5.92	118.88%
≥20	27	0 (0)	5.89 (21.81)	5.89	21.81	···
FRS CVD
0 to <5	64	32 (50)	1.29 (2.02)	−30.71	−47.98	−95.96
5 to <7.5	26	14 (53.85)	1.6 (6.15)	−12.4	−47.70	−88.58
7.5 to 20	108	46 (42.59)	15.28 (14.15)	−30.72	−28.44	−66.78
≥20	235	33 (14.04)	72.27 (30.75)	39.27	16.71	119.02

ASCVD indicates Atherosclerotic Cardiovascular Disease risk score; CVD, cardiovascular disease; FRS, Framingham Risk Score; and RRS, Reynolds Risk Score.

Discussion

Our analysis showed that 5 of 6 commonly used primary prevention risk scores classified a large proportion of INOCA women as low risk for CVD events and all 6 scores failed to accurately predict risk in this population. Although recalibration improved risk score performance when applied to our population, significant underestimation of event rates was present across many of the clinically relevant risk categories with major implications regarding treatment decisions. The discrimination was moderate to low for all risk scores.

Contemporary with the WISE enrollment time period, little was known relative to risk in INOCA patients. All women enrolled underwent clinically indicated coronary angiography based on signs and/or symptoms of ischemia.32 Patients with no significant obstructive CAD on angiography have traditionally been dismissed from specialty care with low rates of therapy prescribed,14, 15, 16, 30 such that they were managed mainly for their CVD risk factors and not as a spectrum of ischemic heart disease.10 In addition, in primary care, the management of CVD risk factors is often suboptimal relative to evidence‐based primary prevention guidelines.36

Given the differences in methodology, recruitment time periods, and racial and geographical limitations, the applicability of the 6 scores analyzed has already been challenged in modern populations. FRS, in particular, generally overestimate risk in primary prevention women.19, 37 In a recent analysis using 4227 patients enrolled in the MESA (Multi‐Ethnic Study of Atherosclerosis)27 data set, the majority of primary prevention scores overestimated risk in men by 37% to 154% and in women by 8% to 67%.

The ASCVD score had poor calibration and overestimated risk for both women and men in the REGARDS (Reasons for Geographic and Racial Differences in Stroke) cohort.38 In the subgroups of participants for whom ASCVD risk may trigger a discussion about statin initiation, the calibration and discrimination were better among women, both in those from the “stroke belt” (North Carolina, South Carolina, Georgia, Alabama, Mississippi, Louisiana, Tennessee, and Arkansas) and in the remainder of the continental United States.38 The ASCVD score performance was also related to social deprivation status, demonstrating good calibration among individuals with social deprivation, but overestimated risk among those with less social deprivation.39 Among Rotterdam Study participants, the ATP III‐FRS, ASCVD, and SCORE showed poor calibration and moderate to good discrimination; significant differences in the proportion of individuals eligible for statins therapy were seen among the 3 scores.26 A recently published article showed that primary prevention scores mainly underestimated risk in an INOCA example patient, whereas great heterogeneity and either overestimation or underestimation of risk was seen when secondary prevention risk scores were used.13

There are multiple plausible explanations for the failure of primary prevention risk scores to accurately predict risk in this INOCA population. First, these scores were created for a primary prevention population that is asymptomatic and thus it may be inappropriate to use these scores to predict outcomes in symptomatic patients despite their lack of obstructive CAD. Although recalibration improved score performance, it is not always available or feasible in common practice. Second, the significance of traditional risk factors in these patients with a high prevalence of coronary vasomotor dysfunction40 may be less than in other traditional populations. The pathophysiology of chest pain and ischemia in the absence of obstructive CAD may be related to subclinical atherosclerosis and thus CVD risk factors are relevant, but other mechanisms may also contribute to INOCA. Although risk factors for coronary microvascular dysfunction are similar to traditional CVD risk factors,41 whether their presence has synergistic or additive value is uncertain as are the implications for therapeutic decisions. Both endothelial and nonendothelial dysfunction have prognostic implications41, 42, 43 and endothelial dysfunction is an independent predictor of major adverse cardiovascular events regardless of the presence of traditional risk factors.43, 44 Both endothelial‐dependent and non–endothelial‐dependent pathways provide possible targets for traditional prevention therapies including statins and angiotensin converting enzyme inhibitors that may alter the long‐term risk in INOCA patients, although this has not been proven and a large study is underway (Women's Ischemia Trial to Reduce Events in Nonobstructive CAD or WARRIOR trial). A substantial portion of variability in endothelial‐dependent and non–endothelial‐dependent coronary dysfunction remains unexplained by traditional risk factors45 and neither is included in the available scores. A recent study in INOCA patients undergoing invasive coronary reactivity testing demonstrated a majority of patients were classified as intermediate risk by FRS, but when coronary microvascular dysfunction was added, a total of 23% were correctly reclassified.46 Moreover, novel CVD risk factors are emerging in INOCA patients (inflammatory milieu, altered expression of local vasoactive substances, genetic loci).44, 47 Finally, changing trends in traditional CVD risk factors are not captured in scores developed decades ago. In contemporary populations, adverse lifestyle trends including increases in obesity and diabetes mellitus have emerged.36, 48 The impact of psychosocial factors on cardiovascular risk in modern populations and especially in women is not well defined, and mental stress–induced ischemia is not reflected in any of the available scores.49, 50

Limitations

The maximum follow‐up for nonfatal events in WISE was 8.16 years. Only the mortality data extended beyond 8.16 years with the National Death Index search, and therefore the nonfatal events were likely underestimated at a maximum follow‐up of 10 years. In addition, during the WISE recruitment period, many contemporary advances in risk factor management were lacking and highly relevant recent technical advances had not been developed, including the use of invasive fractional flow reserve, instant wave‐free ratio, advanced intravascular imaging, cardiac magnetic resonance quantitative perfusion assessment, and computed tomography angiography with computed tomography angiography–fractional flow reserve, plaque quantitation, and characterization. Finally, there were low event rates in some categories, making definitive conclusions more difficult (ie, risk >20%).

Conclusions

In summary, 6 available primary prevention risk assessment scores fail to adequately predict outcomes in INOCA patients. These results emphasize the need for new, specifically tailored, stepwise comprehensive risk‐assessment tools addressing INOCA, the presence of coronary microvascular dysfunction, and other novel risk factors in this increasingly prevalent population.

Sources of Funding

Research reported in this publication was supported by the National Heart, Lung, and Blood Institute (NHLBI) under grant numbers N01HV68161, N01HV68162, N01HV68163, N01HV68164, U01HL64829, U01HL64914, U01HL64924, K23HL105787, T32HL69751, R01HL090957, R01HL33610, R01HL56921, and UM1HL087366; the National Institute on Aging under grant number R03AG032631; the National Center for Research Resources under grant number M01RR000425; the National Center for Advancing Translational Sciences under grant numbers UL1TR000124, UL1TR000064, and UL1TR001427. This work was also supported by grants from the Gustavus and Louis Pfeiffer Research Foundation, Danville, NJ; The Ladies Hospital Aid Society of Western Pennsylvania, Pittsburgh, PA; The Society for Women's Health Research, Washington, DC; QMED, Inc., Laurence Harbor, NJ; The Women's Guild of Cedars‐Sinai, the Edythe L. Broad, the Constance Austin Women's Heart Research Fellowships, the Barbra Streisand Women's Cardiovascular Research and Education Program, the Linda Joy Pollin Women's Heart Health Program, the Erika J. Glazer Women's Heart Research Initiative, and The Adelson Family Foundation, Cedars‐Sinai Medical Center, Los Angeles, CA; the Gatorade Trust and the PCORnet‐One Florida Clinical Research Consortium CDRN‐1501‐26692, University of Florida, Gainesville, FL.

Disclosures

Dr Bittner serves on the Executive Steering Committee of the ODYSSEY OUTCOMES trial (Sanofi), as National Coordinator for STRENGTH (Astra Zeneca), DalGene (Dalcor), CLEAR (Esperion), as local site investigator for ORION IV (The Medicines Company), and was a local site investigator for ARTEMIS (Astra Zeneca) and COMPASS (Bayer Healthcare), both completed. All activities are contracted through the University of Alabama at Birmingham. She has served as a consultant for Sanofi. Dr Handberg receives research grants from Aastrom Biosciences, Amorcyte, Biocardia, Brigham and Women's Hospital, Capricor, Cytori Therapeutics, Department of Defense, Direct Flow Medical, Duke Clinical Research Institute, East Carolina University, Everyfit Inc., Medtronic, Merck & Co., Mesoblast, National Institutes of Health (NIH), NIH through University of Rochester, NIH through Brigham and Women's Health, NIH through University of Texas, Patient‐Centered Outcomes Research Institute (PCORI), and Sanofi Aventis; a research grant and educational grant from Gilead Sciences; unrestricted educational grants for the Vascular Biology Working Group from Amgen, AstraZeneca, Boehringer Ingelheim, Daiichi Sankyo, Ionis, and Relypsa; and consultant fees from Bristol‐Myers Squibb Company. Dr Bairey Merz has received honorarium from Abbott Diagnostics and serves as a Board Director for iRhythm. Dr Pepine receives support from the NIH NHLBI–HL087366 (University of Florida Regional Clinical Center for Cardiovascular Cell Therapy Research [UFRCC] for the Cardiovascular Cell Therapy Research Network), HL132448 (Brain‐Gut Microbiome‐Immune Axis in Hypertension), HL033610 (Angiotensin and Neuroimmune Activation in Hypertension), and HL146158 (WISE Heart Failure with Preserved Ejection Fraction [HFpEF]); the Gatorade Trust through funds distributed by the University of Florida, Department of Medicine; NIH National Center for Advancing Translational Sciences (NCATS)—University of Florida Clinical and Translational Science UL1TR001427; PCORnet‐OneFlorida Clinical Research Consortium CDRN‐1501‐26692; and US Department of Defense PR161603 (WARRIOR). The remaining authors have no disclosures to report.

(J Am Heart Assoc. 2020;9:e013234 DOI: 10.1161/JAHA.119.013234.)