This secondary analysis of the PROMISE randomized clinical trial describes a risk tool developed to use only pretest clinical data to identify patients with chest pain with normal coronary arteries and no clinical events during follow-up.
Key Points
Question
Is it possible to create a risk tool to identify intermediate-risk patients with stable chest pain unlikely to benefit from noninvasive testing?
Findings
In this secondary analysis of a randomized clinical trial, 1156 of 4632 patients (25.0%) with stable chest pain had normal coronary arteries (without atherosclerosis) and no long-term clinical events. These minimal-risk patients can be identified with good discrimination using pretest clinical characteristics alone.
Meaning
A clinical tool using readily available pretest variables discriminates such minimal-risk patients, for whom deferred testing may be considered.
Abstract
Importance
Guidelines recommend noninvasive testing for patients with stable chest pain, although many subsequently have normal test results and no adverse clinical events.
Objective
To describe a risk tool developed to use only pretest clinical data to identify patients with chest pain with normal coronary arteries and no clinical events during follow-up (minimal-risk cohort).
Design, Setting, and Participants
This secondary analysis of a randomized, pragmatic comparative effectiveness trial (Prospective Multicenter Imaging Study for Evaluation of Chest Pain [PROMISE]) includes stable, symptomatic outpatients without known coronary artery disease referred for noninvasive testing at 193 sites in North America.
Interventions
Patients were randomized to receive coronary computed tomography angiography (CCTA) vs functional testing.
Main Outcomes and Measures
A low-risk tool was developed and internally validated from July 27, 2010, to September 19, 2013, in 4632 patients receiving CCTA as their initial test, with a median follow-up of 25 months. Logistic regression analysis was used to evaluate pretest variables to determine factors associated with minimal risk using a two-thirds random sample for model derivation (n = 3104) and a one-third sample for testing and validation (n = 1528). The model was then applied to the CCTA and functional testing arms, and test results and event rates were ascertained.
Results
A total of 1156 of 4632 patients (25.0%) were in the minimal-risk cohort. The final minimal-risk model included 10 clinical variables that together were correlated with normal CCTA results and no clinical events (C statistic = 0.730 for the derivation subset; 95% CI, 0.710-0.750): younger age; female sex; racial or ethnic minority; no history of hypertension, diabetes, dyslipidemia, or family history of premature coronary artery disease; never smoking; symptoms unrelated to physical or mental stress; and higher high-density lipoprotein cholesterol level. Across the entire PROMISE cohort, this model was associated with the lowest rates of severely abnormal test results (1.5% for CCTA; 4.9% for functional) and cardiovascular death or myocardial infarction (0.2% for a median of 25 months) among patients at the highest probability (10th decile) of minimal risk.
Conclusions and Relevance
In contemporary practice, more than 25% of patients with stable chest pain referred for noninvasive testing will have normal coronary arteries and no long-term clinical events. A clinical tool using readily available pretest variables discriminates such minimal-risk patients, for whom deferred testing may be considered.
Trial Registration
clinicaltrials.gov Identifier: NCT01174550
Introduction
Current guidelines recommend noninvasive testing in symptomatic patients with intermediate pretest probability of obstructive coronary artery disease (CAD).1 However, studies of outpatients with a clinical syndrome of possible ischemia found that most functional test results are normal, and more than 95% of those patients will not experience an untoward clinical event during 2 years of follow-up,2,3,4,5 even among those thought to be at intermediate pretest likelihood of CAD by traditional risk assessment tools, such as the Diamond-Forrester risk score.6,7,8,9 These findings have led to substantial concerns about overtesting10 and development of appropriate use criteria to guide clinical decision making11 but little in the way of novel approaches to quantitative risk assessment that might better guide decision making.
Patients unlikely to have CAD, clinical events, or revascularization will derive minimal benefit and value from noninvasive testing. If such patients can be identified before testing, the need for testing and its associated costs might be reduced. Safely deferred testing even in a modest percentage of patients would be an attractive strategy in an era when health care systems are accountable for costs and outcomes.12
We hypothesized that the highly annotated baseline demographic and clinical pretest data available in the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) trial would allow creation of a risk model useful for identifying patients unlikely to benefit from noninvasive testing, defined as those with normal test results (including no coronary atherosclerosis) and no clinical events during the subsequent 2 years of follow-up. We then used the model to develop a clinical tool to aid health care professionals in identifying patients who may derive minimal or no value from noninvasive testing for whom deferring testing may be considered.
Methods
Study Population and Design
PROMISE was a pragmatic comparative effectiveness trial that enrolled 10 003 evaluable patients at 193 sites in North America representing community practices and academic medical centers. The PROMISE study design and primary results have been described in detail.7,13 The study enrolled stable symptomatic outpatients without known CAD referred for noninvasive testing for further evaluation from July 27, 2010, to September 19, 2013. Patients were randomized to an initial anatomical testing strategy with coronary computed tomography angiography (CCTA) or a functional testing strategy (exercise treadmill testing, stress echocardiography, or stress nuclear imaging) and then followed up for a median of 25 months for outcome events. The study protocol was approved by the local or central institutional review board at each coordinating center and at each of the 193 enrolling sites in North America (eAppendix 1 in the Supplement). Written informed consent was obtained from all patients.
Definition and Derivation of the Minimal-Risk Cohort
Of the 10 003 patients randomized in PROMISE, the 4633 patients who were randomized to and received CCTA as an initial noninvasive test served as the initial analytic group in this secondary analysis of PROMISE (Figure). Only patients receiving CCTA as their initial test were included in model development because the definition of normal in this cohort included those without any detectable atherosclerosis who are at the lowest risk of adverse events.14 Patients were categorized as being at minimal risk if they underwent CCTA that produced normal results and if all of the following conditions defined a priori were met: (1) coronary artery calcium score was 0 or was not obtained; (2) no evidence of atherosclerosis; (3) overall study quality was diagnostic (ie, sufficient data quality for interpretation); (4) left ventricular function was normal or not reported; (5) no wall motion abnormalities were present or not reported; and (6) no relevant cardiovascular incidental findings that could account for the patients’ symptoms (ie, aortic dissection or pulmonary embolism) were noted. All patients with normal CCTA results were included in the minimal-risk cohort in the absence of any of the following adjudicated clinical events during the median 25-month follow-up period: all-cause death, nonfatal myocardial infarction (MI), unstable angina hospitalization, or revascularization during the entire follow-up period. The final study cohort consisted of 1156 minimal-risk patients and 3476 other patients (n = 4632) (Figure).
Figure. Cohort Derivation, Model Derivation, and Validation Populations.
Of the 10 003 patients randomized in the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE), 4633 who received coronary computed tomography angiography (CCTA) as an initial noninvasive test were used to derive the model cohorts. Patients with no clinical events (all-cause death, myocardial infarction, unstable angina hospitalization, or revascularization) and a normal CCTA result were included in the minimal-risk cohort; all other patients were included in the other cohort. One patient with insufficient data to categorize them as minimal risk vs other was excluded. The final analysis population was then divided into derivation and validation subsets with a random 2:1 split. CAC indicates coronary artery calcium; ICA, invasive coronary angiography.
Statistical Analysis
Continuous baseline characteristics were summarized using means and SDs or medians with 25th and 75th percentiles. Categorical variables were summarized using frequencies and percentages. Group comparisons with respect to continuous baseline variables were performed using the Wilcoxon rank sum test; the Pearson χ2 test or Fisher exact test was used for comparisons that involved categorical variables. Statistical significance was set at α = .05. All analyses were performed with SAS statistical software, version 9.4 or JMP Pro, version 12.1 (SAS Institute Inc).
Complete data were available for most of the baseline variables included in the a priori–specified list of interest. However, for total and high-density lipoprotein cholesterol, which had more than 5% missingness, imputation was performed with a single imputation procedure, namely, the fully conditional specification method using PROC MI in SAS, version 9.4.
Using the final study cohort, we performed a random two-thirds to one-third data split to create a model development subset (n = 3104) and a model testing and validation subset (n = 1528) (Figure). In the model development subset, multivariable logistic regression analysis was used to identify baseline (pretest) variables that were useful in discriminating the minimal-risk patients from the other patients and creating a model for identifying which patients were likely to fall into the minimal-risk category. For continuous baseline variables, we examined the shape and strength of the association between individual variables and whether patients were in the minimal-risk group using a flexible model-fitting approach that involved restricted cubic spline functions (cubic polynomials).15 Categorical variables with more than 3 levels were collapsed to 3 clinically meaningful categories. After examining univariate associations, a multivariable logistic regression model for assessing the probability of minimal risk was fit using the backward variable selection method with all the a priori–specified covariates that were significantly different at the α = .05 level between the minimal-risk and other groups or were considered clinically relevant. The model derived in the model development data set, including the variables selected and their corresponding variable estimates, was then applied to the testing and validation data set to assess the model’s performance in an independent sample.
The measure of predictive discrimination used to characterize model performance in the model development and validation samples was the area under the receiver operating characteristic curve (AUC) or C statistic.16 Calibration of the model predictions was assessed by graphic comparison of the observed proportion of minimal-risk patients to the mean model prediction across deciles of the potential probabilities of minimal risk. In addition, the Hosmer-Lemeshow statistic was calculated to statistically assess model calibration. Finally, a sensitivity analysis using bootstrapping as an alternative model-building strategy was performed as a comparison to the initial model-building approach. The mean C statistic was computed and reported based on 1000 bootstrap samples.
With the use of the final model, the potential probability of minimal risk was computed for each person in the CCTA and functional testing cohorts. For each cohort, the distribution of potential probabilities was divided into deciles, with the first decile including patients with the lowest potential probabilities of minimal risk (ie, the patients least likely to be minimal risk) and the 10th decile including patients with the highest potential probabilities (ie, the patients most likely to be minimal risk). Additional details of the model application and the comparison to other risk scores are provided in eAppendix 2 in the Supplement.
After validation of the model, the variables selected in the model development phase were refit in a final multivariable logistic regression model using the entire CCTA cohort (n = 4632) to obtain more robust variable estimates for application to other patient cohorts. The resulting C statistic was reported, as well as odds ratios and 95% CIs (as estimated by the Mann-Whitney test), for each variable in the model.
Results
Study Population and Cohorts
Of the 4632 patients in the CCTA arm, 1156 patients (25.0%) had a completely normal CCTA result and no clinical events during the median 25-month follow-up period; these patients were categorized as the minimal-risk cohort (Figure). In contrast, 3476 patients (75.0%) had an abnormal CCTA result and/or experienced a clinical event; they were categorized as the other cohort. Among patients in the derivation subset (two-thirds of patients, n = 3104), there were many statistically significant differences in baseline clinical characteristics between the minimal-risk (n = 775; mean [SD] age, 57.5 [7.0] years; 484 females [62.5%]; 200 racial or ethnic minority [26.0%]) and other (n = 2329) patients (Table 1). A complete list of baseline clinical characteristics for the entire CCTA study population, including all variables considered for model building, is given in eTable 1 in the Supplement.
Table 1. Selected Baseline Clinical Characteristics in the Derivation Subseta.
| Characteristic | No. (%) | P Value | |
|---|---|---|---|
| Minimal Risk (n = 775) |
Other (n = 2329) |
||
| Age, mean (SD), yb | 57.5 (7.03) | 61.2 (8.25) | <.001 |
| Femaleb | 484 (62.5) | 1077 (46.2) | <.001 |
| Racial or ethnic minorityb | 200 (26.0) | 514 (22.2) | .03 |
| Physician estimate of likelihood of obstructive diseaseb | <.001 | ||
| Very low or low (≤30%) | 357 (46.1) | 816 (35.1) | |
| Intermediate, high, or very high (>30%) | 418 (53.9) | 1507 (64.9) | |
| Cardiac risk factor | |||
| Hypertensionb | 449 (57.9) | 1549 (66.5) | <.001 |
| Diabetesb | 114 (14.7) | 526 (22.6) | <.001 |
| Metabolic syndromeb | 231 (29.8) | 918 (39.4) | <.001 |
| Dyslipidemiab | 477 (61.5) | 1608 (69.0) | <.001 |
| Family history of premature CADb | 237 (30.7) | 797 (34.3) | .06 |
| Peripheral artery diseaseb | 5 (0.6) | 39 (1.7) | .04 |
| History of TIA | 5 (0.6) | 40 (1.7) | .03 |
| History of strokeb | 6 (0.8) | 52 (2.2) | .009 |
| History of cerebrovascular diseaseb | 14 (1.8) | 106 (4.6) | <.001 |
| History of CASb | 0 | 14 (0.6) | .03 |
| NYHA classb | .44 | ||
| Normal | 753 (97.5) | 2279 (98.1) | |
| II or III | 19 (2.5) | 43 (1.9) | |
| IV | 0 | 2 (0.1) | |
| Tobacco smokingb | <.001 | ||
| Never | 461 (59.6) | 1070 (45.9) | |
| Ever | 313 (40.4) | 1259 (54.1) | |
| History of depression | 155 (20.0) | 450 (19.3) | .68 |
| Participate in physical activity | 426 (55.0) | 1194 (51.4) | .08 |
| Risk factors per patient | <.001 | ||
| No. | 775 | 2329 | |
| Mean (SD) | 2.1 (0.99) | 2.5 (1.08) | |
| CAD equivalentb,c | 128 (16.5) | 611 (26.2) | <.001 |
| Framingham risk score | <.001 | ||
| No. | 774 | 2322 | |
| Mean (SD) | 14.8 (10.21) | 23.5 (15.45) | |
| Diamond-Forrester/CASS risk score, mean (SD) | 47.8 (20.38) | 55.2 (21.34) | <.001 |
| ASCVD pooled cohort risk score | <.001 | ||
| No. | 768 | 2298 | |
| Mean (SD) | 9.3 (8.39) | 15.9 (11.74) | |
| Chest pain characterization | .38 | ||
| Typical | 78 (10.1) | 276 (11.9) | |
| Atypical | 614 (79.2) | 1799 (77.2) | |
| Noncardiac | 83 (10.7) | 254 (10.9) | |
| Primary symptomc | <.001 | ||
| Chest pain | 608 (78.5) | 1669 (71.7) | |
| Dyspnea | 81 (10.5) | 358 (15.4) | |
| Other | 86 (11.1) | 300 (12.9) | |
| Primary symptom related to physical or mental stressb | <.001 | ||
| No | 397 (51.2) | 986 (42.4) | |
| Yes | 291 (37.5) | 1081 (46.5) | |
| Unknown | 87 (11.2) | 260 (11.2) | |
| Primary symptom relieved by rest or nitroglycerin within 10 minb | .04 | ||
| Always or usually | 230 (29.7) | 792 (34.0) | |
| Rarely or never | 173 (22.3) | 529 (22.7) | |
| Unknown | 372 (48.0) | 1006 (43.2) | |
Abbreviations: ASCVD, atherosclerotic cardiovascular disease; CAD, coronary artery disease; CAS, carotid artery stenosis; CASS, Coronary Artery Surgery Study; NYHA, New York Heart Association; TIA, transient ischemic attack.
Data are presented as number (percentage) of patients unless otherwise indicated.
Variables included in the multivariable analysis.
The CAD risk equivalent was defined as diabetes, peripheral vascular disease, or cerebrovascular disease.
Results of Multivariable Model
On the basis of the multivariable analysis in the model development subset, 10 pretest clinical variables independently assessed the probability of patients being categorized as minimal risk (Table 2). The C statistic was 0.730 (95% CI, 0.710-0.750). Using the variable estimates obtained in the model development subset and applying the model to the independent validation subset, which consisted of 1528 randomly selected patients undergoing CCTA, yielded a nearly identical C statistic of 0.713 (95% CI, 0.684-0.742); when the model was applied to the entire CCTA population, the C statistic was 0.725 (95% CI, 0.709-0.741). Sensitivity analyses using bootstrapping resulted in a similar C statistic of 0.723.
Table 2. Factors Associated With Minimal Risk in the Final Derivation Modela.
| Factor | Odds Ratio (95% CI)b | P Value | χ2 |
|---|---|---|---|
| Age (per 5-y decrease) | 1.55 (1.45-1.66) | <.001 | 173.9 |
| Female sex | 2.66 (2.17-3.25) | <.001 | 90.0 |
| Racial or ethnic minority | 1.07 (0.87-1.31) | .54 | 0.4 |
| No hypertension | 1.44 (1.20-1.73) | <.001 | 15.5 |
| No dyslipidemia | 1.48 (1.23-1.79) | <.001 | 17.1 |
| Never smokerc | 1.80 (1.51-2.15) | <.001 | 41.8 |
| No family history of CAD | 1.36 (1.13-1.65) | .001 | 10.3 |
| No diabetes | 1.50 (1.18-1.92) | .001 | 10.6 |
| Symptoms unrelated to physical or mental stress (reference group: Yes) | .006 | 10.2 | |
| No | 1.35 (1.12-1.63) | ||
| Unknown | 1.20 (0.89-1.62) | ||
| HDL-C (per 5-point increase) | 1.04 (1.01-1.07) | .02 | 5.3 |
Abbreviations: CAD, coronary artery disease; HDL-C, high-density lipoprotein cholesterol.
Model derivation C statistic = 0.730; model validation C statistic = 0.713.
Odds ratios greater than 1.00 indicate increased probability of minimal risk for every 5-unit increase or decrease in continuous variables and when comparing category shift in categorical variables.
Compared with ever smoking.
Model Calibration
Calibration of the model when applied to the independent validation data set was examined by comparing the observed proportion of minimal-risk patients with the mean potential probability of minimal risk by deciles of the potential probabilities (eFigure 1 in the Supplement). Excellent calibration is demonstrated graphically and by the Hosmer–Lemeshow calibration statistic (χ2 = 5.91, P = .66).
Test Results and Clinical Event Rates by Probability of Minimal Risk
With the variables selected, the minimal-risk model was refit in the full PROMISE CCTA cohort (n = 4632). As reported in Table 3, as the probability of minimal risk increased, the probability of a normal CCTA also increased, whereas the probability decreased of an abnormal or severely abnormal test result. As outlined in eTable 2 in the Supplement, a severely abnormal CCTA result was defined as having 1 of the following: 2-vessel disease (≥70%) or 50% or more left main artery stenosis or 70% or more proximal left anterior descending artery stenosis. Similar data were found when the model was applied to the functional testing cohort. However, compared with the CCTA cohort, at the highest probability of minimal risk, patients in the functional arm had greater rates of normal test results (84.0% vs 65.6%) and severely abnormal test results (4.9% vs 1.5%). In the entire study population (combined CCTA and functional cohorts), as the probability of minimal risk increased, the probability of cardiovascular death and MI decreased, with the lowest event rate among patients with the highest probability of minimal risk (0.2% during a median 25 months of follow-up; 10th decile or 0.1% per year).
Table 3. Test Results and Overall Event Rates by Probability of Minimal Riska.
| Decile | Mean Probability of No Riskb | No. of Patients in Decile | No. of Patients With CCTA Results | CCTA Test Results, No. (%) | CV Death and MI, No. (%)c | ||||
|---|---|---|---|---|---|---|---|---|---|
| Normal | Abnormal | High-Risk Abnormal | Significantly Abnormal | Mildly Abnormal | |||||
| 1 | 0.05 | 458 | 440 | 32 (7.3) | 408 (92.7) | 63 (14.3) | 52 (11.8) | 293 (66.6) | 17 (3.7) |
| 2 | 0.09 | 459 | 441 | 51 (11.6) | 390 (88.4) | 47 (10.7) | 42 (9.5) | 301 (68.3) | 6 (1.3) |
| 3 | 0.13 | 459 | 449 | 79 (17.6) | 370 (82.4) | 43 (9.6) | 29 (6.5) | 298 (66.4) | 8 (1.7) |
| 4 | 0.17 | 458 | 454 | 100 (22.0) | 354 (78.0) | 29 (6.4) | 34 (7.5) | 291 (64.1) | 9 (2.0) |
| 5 | 0.21 | 459 | 451 | 129 (28.6) | 322 (71.4) | 28 (6.2) | 19 (4.2) | 275 (61.0) | 5 (1.1) |
| 6 | 0.25 | 459 | 453 | 165 (36.4) | 288 (63.6) | 17 (3.8) | 24 (5.3) | 247 (54.5) | 5 (1.1) |
| 7 | 0.29 | 458 | 451 | 173 (38.4) | 278 (61.6) | 14 (3.1) | 27 (6.0) | 237 (52.5) | 5 (1.1) |
| 8 | 0.35 | 459 | 452 | 219 (48.5) | 233 (51.5) | 8 (1.8) | 18 (4.0) | 207 (45.8) | 4 (0.9) |
| 9 | 0.42 | 459 | 451 | 253 (56.1) | 198 (43.9) | 8 (1.8) | 11 (2.4) | 179 (39.7) | 3 (0.7) |
| 10 | 0.54 | 458 | 453 | 297 (65.6) | 156 (34.4) | 7 (1.5) | 10 (2.2) | 139 (30.7) | 1 (0.2) |
Abbreviations: CCTA, coronary computed tomography angiography; CV, cardiovascular; MI, myocardial infarction.
Patients without a test result were excluded for model application purposes.
Final model formula for calculating minimal risk: 1/(1 + exp [−1.783 + age (0.084) + sex (female) (−1.026) + ethnicity (yes) (−0.142) + tobacco (never) (−0.526) + diabetes (no) (−0.314) + dyslipidemia (no) (−0.412) + family history of coronary artery disease (no) (−0.309) + hypertension (no) (−0.408) + stress (no) (−0.309) + stress (unknown) (−0.195) + high-density lipoprotein cholesterol (−0.006)]). cMedian follow-up of 25 months.
Comparison With Other Risk Scores
eFigure 2 in the Supplement shows the AUCs for the PROMISE minimal-risk model compared with the Framingham and Diamond-Forrester/Coronary Artery Surgery Study (CASS) risk scores for the assessment of minimal risk when applied to the entire CCTA population. The PROMISE minimal-risk model (AUC = 0.725; 95% CI, 0.709-0.741) better characterized patients as minimal risk compared with the Framingham (AUC = 0.687; 95% CI, 0.670-0.704) and Diamond-Forrester/CASS risk scores (AUC = 0.610; 95% CI, 0.592-0.628).
PROMISE Minimal-Risk Tool and Patient Example
As a patient example, consider a 50-year-old white woman with atypical chest pain, a medical history of hypertension, and a high-density lipoprotein cholesterol level of 70 mg/dL (to convert to millimoles per liter, multiply by 0.0259) is in the 10th decile of minimal risk. On average, patients in the 10th decile from PROMISE had a 65.6% probability of having a normal CCTA result, a 1.5% chance of a severely abnormal CCTA result, and a 0.2% risk of cardiovascular death or nonfatal MI during a 25-month follow-up period. The individual patient whose data are presented is estimated to have a probability of 0.58 of being at minimal risk (normal CCTA result and no clinical events).
Discussion
Among PROMISE patients undergoing evaluation for stable chest pain with an intermediate pretest probability of CAD, 25.0% had a normal CCTA result, no clinical events, and no revascularization during a median 25 months of follow-up. A model that included only pretest demographic and clinical data identified this minimal-risk cohort with good discrimination (C statistic = 0.730) and improved existing risk scores. From that model we developed a pretest decision support tool to help clinicians identify patients with stable chest pain who are likely to derive minimal benefit and value from testing.
The ability to identify a subset of intermediate pretest probability patients with stable chest pain who might safely defer noninvasive testing is appealing given concerns about the low yield of testing in current practice and the associated costs. Studies2,3,4,5,7 suggest that the results of most functional stress studies on outpatients with a clinical syndrome of possible ischemia are normal, and nearly all such patients will not experience an untoward clinical event. To our knowledge, few prior analyses have attempted to identify and validate a set of descriptors that could be used to help clinicians identify a subset of symptomatic, intermediate-likelihood patients who are likely to receive little or no benefit from noninvasive testing. The concept of not testing such patients is commensurate with guideline recommendations to not test unless the pretest probability of obstructive disease is greater than 10%.1
The model used 10 pretest variables commonly available during the evaluation of a symptomatic patient with chest pain; we incorporated these variables into a simple risk assessment tool, allowing individual risk estimation. This tool provides useful information in terms of the probability of minimal risk but also the probabilities of an individual patient having any abnormal or a severely abnormal test result (CCTA and functional) or a clinical event. Although the clinical threshold to recommend noninvasive testing may vary, it may be reasonable to consider deferral of noninvasive testing for some patients. We acknowledge that approximately one-third of patients in the 10th decile (highest probability of minimal risk) still have some mild degree of coronary atherosclerosis (Table 3). However, few in this group have obstructive disease, and even fewer have a subsequent adverse event during follow-up. To put this information into context, patients in our study with the highest probability of minimal risk had a lower risk profile (0.2% risk of cardiovascular death and MI at a median 25 months of follow-up) compared with a disease-free, asymptomatic 50-year-old cohort from the Framingham Heart Study (1.4% annual risk of MI, coronary insufficiency, cardiovascular death, angina pectoris, atherothrombotic stroke, and intermittent claudication for males and 0.7% for females).17
Strengths and Limitations
The analysis has several advantages. Data from the patients in the CCTA arm of PROMISE were used to develop the risk tool model because CCTA can identify patients with normal coronary arteries with high accuracy,18,19 and having no atherosclerosis represents the lowest-risk patients.14 The pragmatic nature of PROMISE well reflects contemporary community practice. The large size of the PROMISE CCTA database allowed cohorts with large sample sizes, as well as the creation of distinct derivation and validation groups. Furthermore, application of the model derived and validated within the CCTA arm to the PROMISE functional testing arm supports its generalizability, with the lowest event rates among patients with the greatest probability of minimal risk (10th decile) in both arms. Therefore, clinicians who are considering functional testing may also contemplate using this risk tool because its risk outputs are not specific to the intended test result type. Finally, the model itself has good discrimination (C statistic = 0.730), consistent validation (including using sensitivity analyses with bootstrapping), and excellent calibration. Importantly, this model identifies the minimal-risk patient cohort better than the traditional Diamond-Forrester/CASS and Framingham risk scores that are currently in clinical use.
There are some advantages of this PROMISE clinical tool compared with other risk scores. The combined Diamond-Forrester/CASS risk score, currently recommended in the 2012 American College of Cardiology/American Heart Association guidelines,1 was not designed to identify patients unlikely to benefit from testing but rather to describe the likely findings because patients were already destined to receive noninvasive or invasive testing. The first Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicenter (CONFIRM) Registry risk score20 demonstrates the incremental prognostic information derived from identification of plaque burden and stenosis on CCTA, was derived in a mixed population of patients (asymptomatic and symptomatic), requires noninvasive testing by definition as part of the score, and thus does not aid in the decision to test or defer testing. A subsequent CONFIRM risk score developed using only medical history included only symptomatic patients but is limited by not reporting cardiovascular death (only MI and all-cause mortality are reported).21 Unlike the PROMISE risk tool, this score did not aim to identify patients without coronary plaque (only those without a coronary stenosis ≥50%), which CONFIRM itself has shown to be prognostically important. Furthermore, our risk tool is more generalizable to a full functional testing population because neither CONFIRM score was applied to patients undergoing exercise treadmill testing alone, stress echocardiography, or nuclear perfusion with pharmacologic stress. Thus, in addition to the entirely pretest nature of the variables in our model, we evaluated a symptomatic cohort and incorporated the probability of normal test results (unlike the CONFIRM and Diamond-Forrester/CASS risk scores) and freedom from downstream events (unlike the Diamond-Forrester risk score) as end points in our model.
A method to identify patients who may derive minimal benefit from testing has several potential implications for patients, clinicians, and clinical practice in general. For patients, this process can mean a reduction in testing from which they would not benefit, thereby saving time, anxiety, and cost, as well as a potential reduction in radiation exposure and false-positive test results that could lead to more invasive, unnecessary procedures. For clinicians, this tool has the potential to help optimize office-based decision making. From a practice and societal perspective, in an era in which practitioners are increasingly held accountable for costs and quality, identifying patients unlikely to benefit from potentially expensive testing and who may be managed conservatively has many potential economic and process-of-care advantages.
This study has some limitations. It is unclear how these results might apply to a higher-risk group with a greater prevalence of disease or incidence of events or those who did not consent to participate in a clinical trial. However, the results are generated from the largest contemporary, real-world evaluation of the role of noninvasive testing among patients with stable chest pain, which strengthens its generalizability. In addition, the risk model was derived and validated using the CCTA cohort and therefore does not address other valid reasons to test, including but not limited to obtaining knowledge of functional capacity or reproduction of symptoms and their association with exertion. Although the model was found to be robust when applied to a distinct validation subset with the PROMISE trial data set, further validation of the tool within other independent data samples will provide greater generalizability. Finally, our tool is not intended to act as concrete decision support to test or not test patients, which would require a randomized clinical trial for validation. Rather, it is an aid that quantifies minimal risk and potentially identifies a subset of patients who are unlikely to benefit from proceeding directly to noninvasive testing.
Conclusions
More than one-quarter of stable, symptomatic outpatients with suspected CAD who currently have an indication for noninvasive testing have no coronary atherosclerosis and no revascularization procedures or clinical cardiac events during a median 2-year period. These patients can be identified using pretest clinical characteristics alone, including risk factor profile, symptom characteristics, and lipid values. These variables can be used to inform a user-friendly, point-of-care decision support tool quantifying the likelihood of having a normal test result and no clinical events. If a high likelihood of a normal test result and low clinical event rate is identified, clinicians could consider, including through shared decision making, a strategy of deferred testing for such patients because the benefit and value of testing are likely to be low.
eAppendix 1. PROMISE Trial Organization
eAppendix 2. Statistical Methods
eTable 1. Complete Baseline Clinical Characteristics in the Full CCTA Analysis Population
eTable 2. Prospective Categorization of Noninvasive Imaging Test Results in the Anatomical and Functional Testing Arms of the Study
eFigure 1. Model Calibration in the CCTA Validation Cohort
eFigure 2. Comparison of Risk Scores for the Prediction of Minimal Risk
References
- 1.Fihn SD, Gardin JM, Abrams J, et al. ; American College of Cardiology Foundation . 2012 ACCF/AHA/ACP/AATS/PCNA/SCAI/STS guideline for the diagnosis and management of patients with stable ischemic heart disease: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines, and the American College of Physicians, American Association for Thoracic Surgery, Preventive Cardiovascular Nurses Association, Society for Cardiovascular Angiography and Interventions, and Society of Thoracic Surgeons. Circulation. 2012;126(25):3097-3137. [DOI] [PubMed] [Google Scholar]
- 2.Hachamovitch R, Berman DS, Kiat H, et al. Exercise myocardial perfusion SPECT in patients without known coronary artery disease: incremental prognostic value and use in risk stratification. Circulation. 1996;93(5):905-914. [DOI] [PubMed] [Google Scholar]
- 3.Bangalore S, Gopinath D, Yao S-S, Chaudhry FA. Risk stratification using stress echocardiography: incremental prognostic value over historic, clinical, and stress electrocardiographic variables across a wide spectrum of Bayesian pretest probabilities for coronary artery disease. J Am Soc Echocardiogr. 2007;20(3):244-252. [DOI] [PubMed] [Google Scholar]
- 4.Mudrick DW, Cowper PA, Shah BR, et al. Downstream procedures and outcomes after stress testing for chest pain without known coronary artery disease in the United States. Am Heart J. 2012;163(3):454-461. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rozanski A, Gransar H, Hayes SW, et al. Temporal trends in the frequency of inducible myocardial ischemia during cardiac stress testing: 1991 to 2009. J Am Coll Cardiol. 2013;61(10):1054-1065. [DOI] [PubMed] [Google Scholar]
- 6.Cheng VY, Berman DS, Rozanski A, et al. Performance of the traditional age, sex, and angina typicality-based approach for estimating pretest probability of angiographically significant coronary artery disease in patients undergoing coronary computed tomographic angiography: results from the multinational Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicenter Registry (CONFIRM). Circulation. 2011;124(22):2423-2432, 1-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Douglas PS, Hoffmann U, Patel MR, et al. ; PROMISE Investigators . Outcomes of anatomical versus functional testing for coronary artery disease. N Engl J Med. 2015;372(14):1291-1300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Douglas PS, Pontone G, Hlatky MA, et al. ; PLATFORM Investigators . Clinical outcomes of fractional flow reserve by computed tomographic angiography-guided diagnostic strategies vs. usual care in patients with suspected coronary artery disease: the prospective longitudinal trial of FFR(CT): outcome and resource impacts study. Eur Heart J. 2015;36(47):3359-3367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.SCOT-HEART investigators . CT Coronary Angiography in Patients With Suspected Angina Due to Coronary Heart Disease (SCOT-HEART): an open-label, parallel-group, multicenter trial. Lancet. 2015;385(9985):2383-2391. [DOI] [PubMed] [Google Scholar]
- 10.Ladapo JA, Blecker S, Douglas PS. Physician decision making and trends in the use of cardiac stress testing in the United States: an analysis of repeated cross-sectional data. Ann Intern Med. 2014;161(7):482-490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wolk MJ, Bailey SR, Doherty JU, et al. ; American College of Cardiology Foundation Appropriate Use Criteria Task Force . ACCF/AHA/ASE/ASNC/HFSA/HRS/SCAI/SCCT/SCMR/STS 2013 multimodality appropriate use criteria for the detection and risk assessment of stable ischemic heart disease: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2014;63(4):380-406. [DOI] [PubMed] [Google Scholar]
- 12.Fordyce CB, Newby DE, Douglas PS. Diagnostic strategies for the evaluation of chest pain: clinical implications from SCOT-HEART and PROMISE. J Am Coll Cardiol. 2016;67(7):843-852. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Douglas PS, Hoffmann U, Lee KL, et al. ; PROMISE investigators . PROspective Multicenter Imaging Study for Evaluation of chest pain: rationale and design of the PROMISE trial. Am Heart J. 2014;167(6):796-803.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Cho I, Chang HJ, Sung JM, et al. ; CONFIRM Investigators . Coronary computed tomographic angiography and risk of all-cause mortality and nonfatal myocardial infarction in subjects without chest pain syndrome from the CONFIRM Registry (Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicenter Registry). Circulation. 2012;126(3):304-313. [DOI] [PubMed] [Google Scholar]
- 15.Harrell FEJ. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. New York, NY: Springer; 2001. [Google Scholar]
- 16.Hosmer DW Jr, Lemeshow S. Applied Logistic Regression. New York, NY: Wiley; 2004. [Google Scholar]
- 17.Lloyd-Jones DM, Leip EP, Larson MG, et al. Prediction of lifetime risk for cardiovascular disease by risk factor burden at 50 years of age. Circulation. 2006;113(6):791-798. [DOI] [PubMed] [Google Scholar]
- 18.Budoff MJ, Dowe D, Jollis JG, et al. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol. 2008;52(21):1724-1732. [DOI] [PubMed] [Google Scholar]
- 19.Meijboom WB, Meijs MF, Schuijf JD, et al. Diagnostic accuracy of 64-slice computed tomography coronary angiography: a prospective, multicenter, multivendor study. J Am Coll Cardiol. 2008;52(25):2135-2144. [DOI] [PubMed] [Google Scholar]
- 20.Hadamitzky M, Achenbach S, Al-Mallah M, et al. ; CONFIRM Investigators . Optimized prognostic score for coronary computed tomographic angiography: results from the CONFIRM registry (Coronary CT Angiography EvaluatioN For Clinical Outcomes: An InteRnational Multicenter Registry). J Am Coll Cardiol. 2013;62(5):468-476. [DOI] [PubMed] [Google Scholar]
- 21.Min JK, Dunning A, Gransar H, et al. Medical history for prognostic risk assessment and diagnosis of stable patients with suspected coronary artery disease. Am J Med. 2015;128(8):871-878. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix 1. PROMISE Trial Organization
eAppendix 2. Statistical Methods
eTable 1. Complete Baseline Clinical Characteristics in the Full CCTA Analysis Population
eTable 2. Prospective Categorization of Noninvasive Imaging Test Results in the Anatomical and Functional Testing Arms of the Study
eFigure 1. Model Calibration in the CCTA Validation Cohort
eFigure 2. Comparison of Risk Scores for the Prediction of Minimal Risk