Abstract
Objective To develop a clinical risk prediction tool for estimating the cumulative six month risk of death and death or myocardial infarction to facilitate triage and management of patients with acute coronary syndrome.
Design Prospective multinational observational study in which we used multivariable regression to develop a final predictive model, with prospective and external validation.
Setting Ninety four hospitals in 14 countries in Europe, North and South America, Australia, and New Zealand.
Population 43 810 patients (21 688 in derivation set; 22 122 in validation set) presenting with acute coronary syndrome with or without ST segment elevation enrolled in the global registry of acute coronary events (GRACE) study between April 1999 and September 2005.
Main outcome measures Death and myocardial infarction.
Results 1989 patients died in hospital, 1466 died between discharge and six month follow-up, and 2793 sustained a new non-fatal myocardial infarction. Nine factors independently predicted death and the combined end point of death or myocardial infarction in the period from admission to six months after discharge: age, development (or history) of heart failure, peripheral vascular disease, systolic blood pressure, Killip class, initial serum creatinine concentration, elevated initial cardiac markers, cardiac arrest on admission, and ST segment deviation. The simplified model was robust, with prospectively validated C-statistics of 0.81 for predicting death and 0.73 for death or myocardial infarction from admission to six months after discharge. The external applicability of the model was validated in the dataset from GUSTO IIb (global use of strategies to open occluded coronary arteries).
Conclusions This risk prediction tool uses readily identifiable variables to provide robust prediction of the cumulative six month risk of death or myocardial infarction. It is a rapid and widely applicable method for assessing cardiovascular risk to complement clinical assessment and can guide patient triage and management across the spectrum of patients with acute coronary syndrome.
Introduction
Although patients with acute coronary syndrome share key pathophysiological mechanisms, they present with diverse clinical, electrocardiographic, and enzyme or marker characteristics and experience a wide range of serious cardiovascular outcomes.1 2 Estimated risk, based on clinical characteristics, is challenging and imprecise, yet risk assessment is needed to guide triage and key management decisions. Regulatory authorities such as the National Institute for Health and Clinical Excellence (NICE) and guideline groups recommend treatments according to specific clinical and risk groupings, and trials show that certain benefits may be predominantly or exclusively restricted to higher risk patients with coronary syndrome.2 3 4 Binary methods of stratifying risk (for example, normal or raised troponin concentration or abnormal or normal findings on electrocardiography) lack sufficient precision.5 6 7 8 9 10 11 To provide more accurate prognostic information, and to target treatment more appropriately, more precise yet user friendly risk stratification is required. To ensure general applicability, risk stratification methods should be derived from unrestricted populations that are representative of patients with acute coronary syndrome in the real world12 and should use widely available clinical variables.
The large multinational observational global registry of acute coronary events (GRACE) has been used to derive regression models to predict death in hospital13 and death after discharge14 in patients with acute coronary syndrome. However, a comprehensive risk model is required to predict the cumulative risk of death and death or myocardial infarction during the high risk first six months after initial presentation with acute coronary syndrome, the period when most complications occur.15 16 Because triage and management decisions are required within the first hours or days after initial presentation we derived a risk tool from characteristics of patients with acute coronary syndrome at initial presentation.
Methods
GRACE methods and design
Full details of the GRACE rationale and methods have been published elsewhere.17 18 The registry was designed to reflect an unbiased population of patients with acute coronary syndrome in 94 hospitals in 14 countries. All cases were assigned to one of the following categories: ST segment elevation myocardial infarction, non-ST elevation myocardial infarction, or unstable angina (see appendix on bmj.com for inclusion criteria and standard definitions). Trained coordinators collected data using standardised case report forms.
Statistical methods
We used two primary end points: all cause death or the composite measure of death or non-fatal myocardial infarction during admission to hospital or after discharge (presentation to six months).
We have summarised the distributions of continuous variables with medians and 25th and 75th centiles and reported the categorical variables as frequencies and percentages. Events that occurred after six months were censored. Table 1shows the variables included in the analysis from hospital admission to six month follow-up. We used a Cox regression model to compute crude hazard ratios and 95% confidence intervals to examine the individual relation between each predictor and death and death or myocardial infarction during follow-up (0 to 6 months).
Table 1.
Factors associated with death and death or myocardial infarction (MI) from hospital admission to six month follow-up (hazard ratios and 95% confidence intervals)
| Predictors | χ2 | Death model | χ2 | Death/MI model |
|---|---|---|---|---|
| Demographics | ||||
| Age (per 10 year increase) | 915.3 | 1.34 (1.31 to 1.36) | 345.4 | 1.13 (1.11 to 1.15) |
| Male | 73.9 | 0.7 (0.60 to 0.72) | 15.5 | 0.9 (0.80 to 0.93) |
| Weight (per 1 kg increase) | 133.2 | 0.98 (0.97 to 0.98) | 30.9 | 0.99 (0.990 to 0.995) |
| Height (per 1 cm increase) | 59.0 | 0.98 (0.97 to 0.98) | 21.1 | 0.99 (0.987 to 0.995) |
| Medical history | ||||
| Angina | 7.5 | 0.9 (0.80 to 0.96) | 17.2 | 0.9 (0.80 to 0.92) |
| Smoking | 65.2 | 0.7 (0.61 to 0.74) | 34.1 | 0.8 (0.75 to 0.87) |
| Stroke | 69.2 | 1.8 (1.56 to 2.10) | 36.5 | 1.4 (1.26 to 1.58) |
| Diabetes | 61.2 | 1.5 (1.36 to 1.67) | 29.4 | 1.2 (1.15 to 1.35) |
| Coronary artery disease | 13.4 | 0.8 (0.72 to 0.91) | 78.1 | 0.7 (0.63 to 0.74) |
| Myocardial infarction | 18.5 | 1.2 (1.13 to 1.37) | 1.0 | 1.0 (0.96 to 1.12) |
| Congestive heart failure | 373.6 | 3.0 (2.66 to 3.32) | 142.8 | 1.8 (1.65 to 2.00) |
| Peripheral vascular disease | 91.0 | 1.9 (1.64 to 2.12) | 33.7 | 1.4 (1.23 to 1.52) |
| Hypertension | 30.8 | 1.3 (1.20 to 1.47) | 4.0 | 1.1 (1.00 to 1.16) |
| Hyperlipidaemia | 109.6 | 0.6 (0.52 to 0.64) | 104.9 | 0.7 (0.63 to 0.73) |
| Atrial fibrillation | 152.8 | 2.3 (2.00 to 2.60) | 46.9 | 1.5 (1.33 to 1.66) |
| Renal dysfunction | 129.5 | 2.2 (1.90 to 2.50) | 25.8 | 1.3 (1.20 to 1.50) |
| PCI | 42.8 | 0.6 (0.49 to 0.68) | 67.4 | 0.6 (0.55 to 0.69) |
| CABG | 3.1 | 0.9 (0.75 to 1.02) | 22.6 | 0.8 (0.68 to 0.85) |
| Positive exercise tolerance test | 22.1 | 0.6 (0.54 to 0.77) | 37.3 | 0.7 (0.58 to 0.75) |
| Bleeding | 27.1 | 2.1 (1.60 to 2.77) | 4.3 | 1.3 (1.02 to 1.79) |
| Delay in admission | 0.1 | 1.0 (1.00 to 1.00) | 0.9 | 1.0 (1.00 to 1.00) |
| Presentation characteristics | ||||
| Pulse | 286.8 | 1.02 (1.01 to 1.02) | 177.4 | 1.01 (1.009 to 1.012) |
| Diastolic blood pressure | 261.2 | 0.98 (0.98 to 0.98) | 66.9 | 0.99 (0.989 to 0.993) |
| Systolic blood pressure | 278.2 | 0.99 (0.98 to 0.99) | 134.5 | 0.99 (0.991 to 0.994) |
| Killip class | 1318.5 | 2.6 (2.47 to 2.74) | 658.7 | 1.9 (1.80 to 1.98) |
| Cardiac arrest | 306.6 | 5.5 (4.52 to 6.62) | 230.6 | 3.8 (3.21 to 4.50) |
| Initial cardiac markers | 246.1 | 2.2 (1.97 to 2.40) | 375.7 | 2.1 (1.94 to 2.24) |
| Initial serum creatinine | 334.2 | 1.3 (1.25 to 1.32) | 125.1 | 1.2 (1.15 to 1.21) |
| Findings on electrocardiography | ||||
| ST elevation | 112.3 | 1.7 (1.52 to 1.84) | 276.6 | 1.8 (1.72 to 1.98) |
| ST depression | 95.8 | 1.6 (1.47 to 1.78) | 95.5 | 1.4 (1.34 to 1.54) |
| ST segment deviation | 215.1 | 2.2 (1.98 to 2.45) | 294.2 | 2.0 (1.84 to 2.14) |
| T wave inversion or pseudonormalisation | 31.8 | 0.7 (0.65 to 0.81) | 34.6 | 0.8 (0.72 to 0.85) |
| ST elevation anterior | 105.8 | 1.8 (1.56 to 1.97) | 138.3 | 1.7 (1.52 to 1.79) |
| ST elevation inferior | 13.0 | 1.2 (1.10 to 1.40) | 88.4 | 1.5 (1.37 to 1.62) |
| ST depression anterior | 58.0 | 1.6 (1.39 to 1.75) | 75.4 | 1.5 (1.35 to 1.60) |
| ST depression inferior | 16.3 | 1.4 (1.17 to 1.59) | 24.9 | 1.3 (1.19 to 1.50) |
| No of leads with ST elevation | 144.8 | 1.5 (1.37 to 1.56) | 284.4 | 1.5 (1.44 to 1.58) |
| No of leads with ST depression | 97.8 | 1.4 (1.28 to 1.44) | 86.0 | 1.2 (1.19 to 1.30) |
| Any significant Q wave | 63.6 | 1.5 (1.37 to 1.67) | 40.4 | 1.3 (1.19 to 1.39) |
| Left bundle branch block | 82.7 | 2.1 (1.79 to 2.47) | 32.2 | 1.5 (1.30 to 1.70) |
| Right bundle branch block | 56.4 | 1.9 (1.58 to 2.17) | 18.2 | 1.3 (1.17 to 1.54) |
| Other changes | 168.8 | 2.1 (1.89 to 2.33) | 84.0 | 1.5 (1.40 to 1.68) |
| Previous use of medical therapy | ||||
| Anti-arrhythmic drugs | 16.9 | 1.7 (1.30 to 2.11) | 0.1 | 1.0 (0.83 to 1.27) |
| Oral/topical nitrates | 17.2 | 1.3 (1.13 to 1.39) | 3.0 | 0.9 (0.85 to 1.01) |
| Aspirin | 4.1 | 0.9 (0.82 to 0.99) | 41.8 | 0.8 (0.73 to 0.84) |
| ACE inhibitors | 20.5 | 1.3 (1.15 to 1.41) | 0.01 | 1.0 (0.92 to 1.08) |
| Calcium channel blocker | 10.1 | 1.2 (1.07 to 1.35) | 1.0 | 1.0 (0.87 to 1.04) |
| Statins | 55.8 | 0.6 (0.52 to 0.68) | 89.8 | 0.6 (0.57 to 0.69) |
PCI=percutaneous coronary intervention; CABG=coronary artery bypass grafting; ACE=angiotensin converting enzyme.
We entered all demographic and clinical variables identified by the crude regression analysis into the stepwise multiple Cox regression (backward) analysis to produce final models for predicting death and death or myocardial infarction. Only those variables associated with an α≤0.05 were retained; all variables in the final model met the assumptions for proportional hazards. No imputation was performed in these final models. Imputation was tested but did not influence the identification of multivariable predictors or the discriminative power of the model for predicting death.13 The discriminative power of the final models was assessed by the mean of the area under the receiver operating characteristic (ROC) curve (C-statistic). The curve is a measure of the discriminating ability of the risk model and is a plot of sensitivity versus 1−specificity. Accuracy of calibration was evaluated by plotting the predicted versus the observed mortality according to population tenths of predicted risk. The model was tested prospectively in a separate dataset in GRACE (n=22 122) and also in an independent external dataset, the GUSTO IIb (global use of strategies to open occluded coronary arteries IIb) dataset,19 comprising the entire spectrum of patients with acute coronary syndrome (12 142 patients, 4131 with ST elevation myocardial infarction, 8011 with non-ST elevation myocardial infarction). The analysis was performed with SAS software package (version 8.2, SAS Institute, Cary, NC) and S-Plus (MathSort, Seattle, WA).
Results
Study population
The derivation population comprised 26 267 patients with suspected acute coronary syndrome enrolled between 1 April 1999 and 30 September 2002. We excluded patients found to have a non-cardiac or non-acute coronary cardiac diagnosis (fig 1). We also excluded patients transferred into a study hospital because they lacked some baseline information and their inclusion may also have led to bias because of morbidity associated with the indications for transfer. The study population therefore comprised 21 688 patients of whom 19 931 were alive at six month follow-up.
Fig 1 GRACE study profile (derivation set of patients)
A total of 1757 (9.1%) deaths occurred, 1046/21 573 in hospital (4.9% among patients with a diagnosis of acute coronary syndrome on admission) and 711/15 265 during the period after discharge (4.7%). We had no information on mortality (in hospital or after discharge) for 51 patients. In the derivation set, 3110 (15.8%) patients died (n=1757) or experienced a non-fatal myocardial infarction (n=1353) between presentation and six month follow-up.
Early risks were highest for patients with ST segment elevation myocardial infarction but by six months the risk of death was similar to those with non-ST segment elevation myocardial infarction (fig 2). Of those who survived to six months after discharge, 36.2% (258/711) presented with ST segment elevation myocardial infarction compared with 50.0% (880/1757) of those who died during admission or follow-up. Raised cardiac markers were detected in 35.0% (6883/19688) of those who survived compared with 53.2% (905/1701) of those who died.
Fig 2 Overall risk of death in hospital, from hospital admission to six months after discharge (patients separated into unstable angina, non-ST segment elevation myocardial infarction, and ST segment elevation myocardial infarction), and from hospital discharge to six months
Validation population
The validation set comprised 22 122 patients enrolled in this multinational registry between 1 October 2003 and 30 September 2005. A total of 1730 (9.0%) patients died between hospital admission and six month follow-up, 948 in hospital (4.3% among patients with an admission diagnosis of acute coronary syndrome) and 782 (5.4%) after discharge. No information on mortality was available for 38 patients. In total, 2720 patients died (n=1730) or experienced a non-fatal myocardial infarction (n=990) between presentation and six month follow-up.
Predictors of mortality
From admission to six month follow-up, Killip class20 and advanced age were the most powerful predictors of death in the univariable analysis (table 1). Table 1 also shows the other baseline characteristics and clinical parameters that predicted death or death or myocardial infarction.
After multivariable analysis, the highest hazard ratios for death were cardiac arrest on admission and increasing age. These two key prognostic factors were closely followed by raised cardiac markers or enzyme activity and ST segment deviation (table 2).
Table 2.
Final risk models predicting death and death or myocardial infarction from hospital admission to six month follow-up (hazard ratios and 95% confidence intervals)
| Predictors | χ2 | Death model | χ2 | Death/MI model |
|---|---|---|---|---|
| Age (per 10 year increase) | 505.7 | 1.8 (1.68 to 1.84) | 176.3 | 1.25 (1.21 to 1.29) |
| Medical history: | ||||
| Congestive heart failure | 34.2 | 1.5 (1.32 to 1.73) | 22.1 | 1.3 (1.17 to 1.45) |
| Hypertension | 8.8 | 1.2 (1.05 to 1.33) | — | |
| Peripheral vascular disease | 21.8 | 1.4 (1.21 to 1.62) | 10.5 | 1.2 (1.08 to 1.36) |
| PCI | 8.3 | 0.8 (0.64 to 0.93) | — | |
| Presentation characteristics: | ||||
| Pulse (per 30 beats/min increase) | 44.3 | 1.2 (1.16 to 1.31) | — | |
| Systolic blood pressure (per 20 mm Hg decrease) | 152.0 | 1.2 (1.22 to 1.30) | 52.9 | 1.1 (1.07 to 1.13) |
| Killip class20 (per level increase) | 142.8 | 1.5 (1.41 to 1.62) | 126.2 | 1.4 (1.30 to 1.46) |
| Initial serum creatinine (per 88 µmol/l* increase) | 135.3 | 1.2 (1.19 to 1.29) | 41.1 | 1.1 (1.08 to 1.16) |
| Initial cardiac markers or enzymes | 63.0 | 1.6 (1.42 to 1.78) | 184.3 | 1.7 (1.60 to 1.87) |
| Cardiac arrest | 58.5 | 2.6 (2.00 to 3.32) | 55.4 | 2.2 (1.76 to 2.63) |
| Findings on electrocardiography: | ||||
| ST segment deviation | 46.8 | 1.6 (1.41 to 1.88) | — | |
| Left bundle block branch | 10.0 | 1.3 (1.10 to 1.60) | — | |
| No of leads with ST segment elevation or depression | 20.1 | 1.2 (1.10 to 1.33) | 158.4 | 1.4 (1.34 to 1.49) |
| ST depression, anterior | — | 36.2 | 1.3 (1.22 to 1.47) | |
| ST depression, inferior | — | 10.8 | 1.2 (1.09 to 1.40) | |
| Other changes | — | 7.2 | 1.1 (1.04 to 1.27) | |
| Hosmer and Lemeshow goodness of fit test | 0.30 | 0.42 | ||
| C-statistic | 0.82 | 0.70 | ||
PCI=percutaneous coronary intervention.
*Equivalent to 1 mg/dl.
Risk models predicting death and death or myocardial infarction
The risk model comprises 14 predictors of death and 12 predictors of death or myocardial infarction. The predictive accuracy of the model was good, with C-statistics of 0.82 for death in hospital and 0.70 for death or myocardial infarction in hospital (table 3). Nine factors independently predicted death and the combined end point in the period from admission to six months after discharge: age, congestive heart failure, peripheral vascular disease, systolic blood pressure, Killip class, initial serum creatinine concentration, positive initial cardiac markers, cardiac arrest on admission, and number of leads with ST deviation. The highest hazard ratio for adverse outcome was for cardiac arrest (tables 1 and 2).
Table 3.
C-statistics for validation of the full model and the simplified model (as used for the nomogram) for all GRACE patients and for acute coronary syndrome subgroups
| All patients | STEMI | Unstable angina/ NSTEMI | |
|---|---|---|---|
| All GRACE patients | |||
| Death: | |||
| Full model | 0.82 | 0.82 | 0.81 |
| Simplified model | 0.81 | 0.82 | 0.79 |
| Death or myocardial infarction: | |||
| Full model | 0.70 | 0.66 | 0.71 |
| Simplified model | 0.70 | 0.66 | 0.70 |
| Transferred patients | |||
| Death: | |||
| Full model | 0.83 | — | — |
| Simplified model | 0.83 | — | — |
| Death or myocardial infarction: | |||
| Full model | 0.71 | — | — |
| Simplified model | 0.70 | — | — |
| Model validation* | |||
| Death: | |||
| Full model | 0.82 | 0.83 | 0.81 |
| Simplified model | 0.81 | 0.82 | 0.81 |
| Death or myocardial infarction: | |||
| Full model | 0.73 | 0.73 | 0.73 |
| Simplified model | 0.73 | 0.73 | 0.73 |
STEMI=ST segment elevation myocardial infarction; NSTEMI=non-ST segment elevation myocardial infarction.
*On subsequent patients with acute coronary syndrome (22 122 enrolled between 1 October 2003 and 30 September 2005).
Prospective and external validation of the GRACE risk score
When we tested the risk model in the prospective validation set, it had excellent predictive accuracy for death (C-statistic=0.81, simplified model) and death or myocardial infarction (C-statistic=0.73).The predictive accuracy was maintained across the acute coronary syndrome subgroups (table 3).
We validated the model externally using the GUSTO IIb dataset of 12 142 patients with acute coronary syndrome. There was excellent discrimination despite the fact that one of the key parameters was not recorded in GUSTO IIb (cardiac arrest). The C-statistic for the death model in all patients was 0.82 (C-statistics=0.80 for ST segment elevation myocardial infarction and 0.76 for non-ST segment elevation myocardial infarction).
Development of a simplified nomogram for clinical application
We reduced the overall models to include the most important variables that contained most (>90%) of the predictive information. This nomogram retained excellent discriminant characteristics based on eight variables and was used for the calculation of risk (fig 3).
Fig 3 GRACE risk calculator for death or myocardial infarction from admission to hospital to six months after discharge with the simplified model (www.outcomes.org/grace)
Discussion
The GRACE risk prediction tool (simplified nomogram) includes variables that are readily available to clinicians even in smaller community hospitals. It provides a novel and widely applicable method of assessing the cumulative six month risk of death and death or myocardial infarction across the spectrum of patients admitted to hospital with acute coronary syndrome. Accurate longer term assessment of risk is important because most cardiac ischaemic events occur within the first few weeks after initial presentation with acute coronary syndrome.15 16 Our findings, based on 48 389 patients, support the validity of the GRACE models for mortality in hospital and after discharge,14 which were derived from data from about 11 000 and 15 000 patients, respectively.
The need for risk prediction in patients with acute coronary syndrome
In clinical practice, initial stratification of patients aims to identify those suitable for reperfusion therapy (on the basis of a clinical syndrome and ST segment elevation or other electrocardiographic markers of acute infarction). Binary approaches are commonly applied among others with acute coronary syndrome, but separating patients based on one or two characteristics may substantially overestimate or underestimate the risk of death or myocardial infarction. There is therefore a need for one predictive instrument that performs well in all patients with acute coronary syndrome.
Robust evidence and practice guidelines (including NICE) suggest that interventional and pharmacological therapies predominantly benefit patients at higher risk.2 3 21 Despite the availability of such guidelines, identification of patients at high risk of cardiac ischaemic events remains challenging.22 23 In addition, the triage of patients into high intensity care units (cardiac care units) is based predominantly on the criteria for reperfusion therapy rather than risk in the patient. For example, a 55 year old woman (blood pressure 142/80 mm Hg; heart rate 88 per minute) who presents with ST elevation and raised troponin concentration but without complications of a myocardial infarction (normal creatinine concentration, no heart failure) has a probability of death of only 3% in the next six months. However, a 55 year old woman with non-ST segment elevation myocardial infarction (blood pressure 118/68; heart rate 92 per min) with mild heart failure and raised creatinine concentration has a six month risk of death of 16%. Without formal risk stratification, the second patient would probably be managed in a low intensity ward area and the management on discharge may not reflect the risk in the patient.
Resolving intermediate risk
Despite similarities in key pathophysiological mechanisms, the characteristics on presentation of patients with acute coronary syndrome depend on the extent of the ischaemic territory (influenced by acute thrombotic risk) and previous risk features (such as older age, heart failure, and renal insufficiency). Whereas patients with high risk features, including cardiogenic shock and heart failure, are relatively straightforward to identify, most patients lie in the intermediate range and risk is less obvious (table 4). This intermediate range encompasses up to 10-fold differences in the risk of death. Binary approaches, including those that require separation of patients into high or low risk, are not accurate enough for most patients in the middle range.2 3 We propose that an appropriate instrument for risk prediction needs to be applicable across the spectrum of acute coronary syndrome, should be derived from a representative and broadly based population, and needs to use variables that are readily available to most clinicians shortly after the patient arrives at hospital.
Table 4.
Resolving intermediate risk (examples). Which patient has higher risk of death or death or myocardial infarction? Is most of risk in hospital or later?
| Variable | Example 1 | Example 2 | Example 3 |
|---|---|---|---|
| Sex | Female | Male | Female |
| Age (years) | 49 | 60 | 62 |
| Heart rate (bpm) | 109 | 94 | 90 |
| Systolic BP (mm Hg) | 100 | 110 | 114 |
| Creatinine (µmol/l ) | 104 | 71 | 106 |
| Killip class | II | I | II |
| Electrocardiographic results | T wave inversion | Non-specific T wave changes | T wave t inversion |
| Troponin T (µg/l) | <0.1 | 1.5 | 2.0 |
| Death in hospital (death at six months) | 1% (2%) | 2% (7%) | 5% (12%) |
| Death or MI at six months | 12% | 22% | 31% |
BP=blood pressure; bpm=beats per minute; MI=myocardial infarction.
How does the present model differ from previous methods of risk stratification?
Several other multivariable prognostic models have been developed,5 6 7 8 9 10 24 25 26 27 28 most of which were derived from clinical trial databases or specific subgroups of patients with acute coronary syndrome. Patients with complications and comorbidity tend to be excluded from such trials, thus limiting applicability in clinical practice. Models developed from large claims databases are potentially subject to bias.8 11 In contrast, the GRACE registry spans the spectrum of acute coronary syndrome and is based on an unselected contemporary population.
A C-statistic of less than 0.70 has been suggested to be of limited clinical value.12 The TIMI (thrombolysis in myocardial infarction) model performs well in patients who are eligible for reperfusion therapy but is less effective in more general patients, including those who are ineligible for reperfusion (C-statistic=0.65).24 An independent study suggests that the unselected GRACE mortality model is superior to either the TIMI or the PURSUIT (platelet glycoprotein IIb/IIIa in unstable angina: receptor suppression with eptifibatide) models.29 We have shown that the cumulative (0 to six month) GRACE risk model performs well across the spectrum of acute coronary syndrome and has prospective and external validity. External validation with the GUSTO IIb dataset confirms the discriminant characteristics of the model when applied to patients with ST segment elevation myocardial infarction and those with non-ST segment elevation myocardial infarction. Although we excluded transferred patients from the derivation of this model (because such patients may lack data for several baseline characteristics), testing the model in the transfer dataset confirmed its applicability to such patients (C-statistic=0.83 for predicting death and 0.70 for predicting myocardial infarction, simplified model).
Simplified risk calculation for clinical application
The simplified model includes most the predictive information: >92% of the total model χ2 for death and >90% for death or myocardial infarction (fig 3). The GRACE risk calculator (fig 3) (available at www.outcomes.org/grace) can be used to derive a prognostic score and to estimate the risk of clinically important end points—death or the combined risk of death or myocardial infarction—in individual patients. For ease of use, this nomogram can be installed into a handheld device or personal computer (data entry takes about 30 seconds) and is also available as a score card.14
Limitations
GRACE is designed to enrol an unselected and generalisable population of patients, though some participating centres are required to obtain informed consent from patients before enrolment. Therefore some patients who died early or who experienced major clinical complications immediately on arrival in hospital may be under-represented. The model may not be appropriate for stratifying low risk patients with non-specific chest pain without acute coronary syndrome, but such patients do not require the same therapeutic and management decisions as those with acute coronary syndrome.
What is already known on this topic
Specific treatments are indicated in higher or lower risk patients with acute coronary syndrome
Conventional clinical assessment and binary methods for predicting risk based on results of electrocardiography and markers of injury are not sufficiently accurate
Previous risk models were based on subgroups of patients with acute coronary syndrome and were derived from large clinical trials or healthcare claims databases
What this study adds
The GRACE risk tool can be used to predict the cumulative risk of death and death or myocardial infarction in the period from admission to hospital to six months after discharge
The tool is simple to apply, robust, externally validated, and applicable to patients across the complete spectrum of acute coronary syndrome
Supplementary Material
We thank the physicians and nurses who participated in GRACE. The risk calculator is available together with further information about the project and the complete list of participants from www.outcomes.org/grace. We thank Sophie Rushton-Smith for editorial services.
Contributors: KAAF, RJG, KAE, FVdeW, ÁA, SGG, FAA, and CBG were responsible for study concept and design. KAAF, KAE, FVdeW, ÁA, SGG, and CBG acquired the data. KAAF drafted the manuscript and is guarantor. All authors critically revised the manuscript for important intellectual content and approved the final version. OHD and KSP carried out statistical analyses.
Funding: The GRACE Registry is supported by an unrestricted educational grant from Sanofi-Aventis to the Center for Outcomes Research, University of Massachusetts Medical School. Sophie Rushton-Smith was funded by Sanofi-Aventis.
Competing interests: KAAF has received grant funding from the British Heart Foundation and his department is supported by the British Heart Foundation, Medical Research Council, Wellcome Trust, Sanofi-Aventis, Bristol-Myers Squibb, and MSD. KAE has received grants from Biosite, Bristol-Myers Squibb, Cardiac Sciences, Blue Cross Blue Shield of Michigan, Hewlett Foundation, Mardigian Fund, Sanofi-Aventis, Varbedian Fund, National Heart, Lung and Blood NIH, and Pfizer. FVdeW has received research grants from Boehringer Ingelheim, Sanofi-Aventis, Proctor and Gamble, Servier, Novartis, MSD, and Schering Plough. ÁA has received funding from Sanofi-Aventis, Population Health Research Institute, and Boehringer Ingelheim. SGG has received funding from AstraZeneca, Sanofi-Aventis, Boehringer Ingelheim, Bristol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Hoffmann-LaRoche Pharmaceuticals, Merck, Novartis, Pfizer, Sanofi-Synthelabo, Schering Corp, and Millennium Pharmaceuticals. MDF, FAA, CBG, and BK have all received funding from Sanofi-Aventis.
Ethical approval: Approval was obtained from local institutional review boards.
Full details of inclusion criteria and standard definitions can be found on bmj.com.
References
- 1.Van de Werf F, Ardissino D, Betriu A, Cokkinos DV, Falk E, Fox KA, et al. Management of acute myocardial infarction in patients presenting with ST-segment elevation. The task force on the management of acute myocardial infarction of the European Society of Cardiology. Eur Heart J 2003;24:28-66. [DOI] [PubMed] [Google Scholar]
- 2.Braunwald E, Antman EM, Beasley JW, Califf RM, Cheitlin MD, Hochman JS, et al. ACC/AHA guidelines for the management of patients with unstable angina and non-ST-segment elevation myocardial infarction. J Am Coll Cardiol 2000;36:970-1062. [DOI] [PubMed] [Google Scholar]
- 3.Bertrand ME, Simoons ML, Fox KA, Wallentin LC, Hamm CW, McFadden E, et al. Management of acute coronary syndrome: acute coronary syndrome without persistent ST segment elevation; recommendations of the task force of the European Society of Cardiology. Eur Heart J 2000;21:1406-32. [DOI] [PubMed] [Google Scholar]
- 4.Fox KA, Poole-Wilson P, Clayton TC, Henderson RA, Shaw TR, Wheatley DJ, et al. 5-year outcome of an interventional strategy in non-ST-elevation acute coronary syndrome: the British Heart Foundation RITA 3 randomised trial. Lancet 2005;366:914-20. [DOI] [PubMed] [Google Scholar]
- 5.Lee KL, Woodlief LH, Topol EJ, Weaver WD, Betriu A, Col J, et al. Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction. Results from an international trial of 41,021 patients. GUSTO-I investigators. Circulation 1995;91:1659-68. [DOI] [PubMed] [Google Scholar]
- 6.Morrow DA, Antman EM, Charlesworth A, Cairns R, Murphy SA, de Lemos JA, et al. TIMI risk score for ST-elevation myocardial infarction: a convenient, bedside, clinical score for risk assessment at presentation: an intravenous nPA for treatment of infarcting myocardium early II trial substudy. Circulation 2000;102:2031-7. [DOI] [PubMed] [Google Scholar]
- 7.Morrow DA, Antman EM, Giugliano RP, Cairns R, Charlesworth A, Murphy SA, et al. A simple risk index for rapid initial triage of patients with ST-elevation myocardial infarction: an InTIME II substudy. Lancet 2001;358:1571-5. [DOI] [PubMed] [Google Scholar]
- 8.Krumholz HM, Chen J, Wang Y, Radford MJ, Chen YT, Marciniak TA. Comparing AMI mortality among hospitals in patients 65 years of age and older: evaluating methods of risk adjustment. Circulation 1999;99:2986-92. [DOI] [PubMed] [Google Scholar]
- 9.Boersma E, Pieper KS, Steyerberg EW, Wilcox RG, Chang WC, Lee KL, et al. Predictors of outcome in patients with acute coronary syndrome without persistent ST-segment elevation. Results from an international trial of 9461 patients. The PURSUIT investigators. Circulation 2000;101:2557-67. [DOI] [PubMed] [Google Scholar]
- 10.Lindahl B, Toss H, Siegbahn A, Venge P, Wallentin L. Markers of myocardial damage and inflammation in relation to long-term mortality in unstable coronary artery disease. FRISC Study Group. Fragmin during instability in coronary artery disease. N Engl J Med 2000;343:1139-47. [DOI] [PubMed] [Google Scholar]
- 11.Jollis JG. Measuring the effectiveness of medical care delivery. J Am Coll Cardiol 2001;37:998-1000. [DOI] [PubMed] [Google Scholar]
- 12.Ohman EM, Granger CB, Harrington RA, Lee KL. Risk stratification and therapeutic decision making in acute coronary syndrome. JAMA 2000;284:876-8. [DOI] [PubMed] [Google Scholar]
- 13.Granger CB, Goldberg RJ, Dabbous OM, Pieper KS, Eagle KA, Goodman SG, et al. Predictors of hospital mortality in the global registry of acute coronary events. Arch Int Med 2003;163:2345-53. [DOI] [PubMed] [Google Scholar]
- 14.Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, et al. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA 2004;291:2727-33. [DOI] [PubMed] [Google Scholar]
- 15.Van Domburg RT, van Miltenburg-van Zijl AJ, Veerhoek RJ, Simoons ML. Unstable angina: good long-term outcome after a complicated early course. J Am Coll Cardiol 1998;31:1534-9. [DOI] [PubMed] [Google Scholar]
- 16.Cohen M, Antman EM, Murphy SA, Radley D. Mode and timing of treatment failure (recurrent ischemic events) after hospital admission for non-ST segment elevation acute coronary syndrome. Am Heart J 2002;143:63-9. [DOI] [PubMed] [Google Scholar]
- 17.GRACE Investigators. Rationale and design of the GRACE (global registry of acute coronary events) project: a multinational registry of patients hospitalized with acute coronary syndrome. Am Heart J 2001;141:190-9. [DOI] [PubMed] [Google Scholar]
- 18.Eagle KA, Goodman SG, Avezum A, Budaj A, Sullivan CM, Lopez-Sendon J. Practice variation and missed opportunities for reperfusion in ST-segment-elevation myocardial infarction: findings from the global registry of acute coronary events (GRACE). Lancet 2002;359:373-7. [DOI] [PubMed] [Google Scholar]
- 19.Randomized trial of intravenous heparin versus recombinant hirudin for acute coronary syndrome. The global use of strategies to open occluded coronary arteries (GUSTO) IIa Investigators. Circulation 1994;90:1631-7. [DOI] [PubMed] [Google Scholar]
- 20.Killip T 3rd, Kimball JT. Treatment of myocardial infarction in a coronary care unit. A two year experience with 250 patients. Am J Cardiol 1967;20:457-64. [DOI] [PubMed] [Google Scholar]
- 21.Guideline for the management of patients with acute coronary syndrome without persistent ECG ST segment elevation. British Cardiac Society Guidelines and Medical Practice Committee and Royal College of Physicians Clinical Effectiveness and Evaluation Unit. Heart 2001;85:133-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Grover SA, Lowensteyn I, Esrey KL, Steinert Y, Joseph L, Abrahamowicz M. Do doctors accurately assess coronary risk in their patients? Preliminary results of the coronary health assessment study. BMJ 1995;310:975-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.McManus RJ, Mant J, Meulendijks CF, Salter RA, Pattison HM, Roalfe AK, et al. Comparison of estimates and calculations of risk of coronary heart disease by doctors and nurses using different calculation tools in general practice: cross sectional study. BMJ 2002;324:459-64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Antman EM, Cohen M, Bernink PJ, McCabe CH, Horacek T, Papuchis G, et al. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA 2000;284:835-42. [DOI] [PubMed] [Google Scholar]
- 25.Savonitto S, Ardissino D, Granger CB, Morando G, Prando MD, Mafrici A, et al. Prognostic value of the admission electrocardiogram in acute coronary syndrome. JAMA 1999;281:707-13. [DOI] [PubMed] [Google Scholar]
- 26.Tu JV, Austin PC, Walld R, Roos L, Agras J, McDonald KM. Development and validation of the Ontario acute myocardial infarction mortality prediction rules. J Am Coll Cardiol 2001;37:992-7. [DOI] [PubMed] [Google Scholar]
- 27.Jacobs DR Jr, Kroenke C, Crow R, Deshpande M, Gu DF, Gatewood L, et al. PREDICT: a simple risk score for clinical severity and long-term prognosis after hospitalization for acute myocardial infarction or unstable angina: the Minnesota heart survey. Circulation 1999;100:599-607. [DOI] [PubMed] [Google Scholar]
- 28.Newby LK, Bhapkar MV, White HD, Topol EJ, Dougherty FC, Harrington RA, et al. Predictors of 90-day outcome in patients stabilized after acute coronary syndrome. Eur Heart J 2003;24:172-81. [DOI] [PubMed] [Google Scholar]
- 29.De Araujo Goncalves P, Ferreira J, Aguiar C, Seabra-Gomes R. TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005;26:865-72. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.