Adjustment of the GRACE score by growth differentiation factor 15 enables a more accurate appreciation of risk in non-ST-elevation acute coronary syndrome

Abstract

Aims

The aim of the study was to evaluate whether knowledge of the circulating concentration of growth differentiation factor 15 (GDF-15) adds predictive information to the Global Registry of Acute Coronary Events (GRACE) score, a validated scoring system for risk assessment in non-ST-elevation acute coronary syndrome (NSTE-ACS). We also evaluated whether GDF-15 adds predictive information to a model containing the GRACE score and N-terminal pro-B-type natriuretic peptide (NT-proBNP), a prognostic biomarker already in clinical use.

Methods and results

The GRACE score, GDF-15, and NT-proBNP levels were determined on admission in 1122 contemporary patients with NSTE-ACS. Six-month all-cause mortality or non-fatal myocardial infarction (MI) was the primary endpoint of the study. To obtain GDF-15- and NT-proBNP-adjusted 6-month estimated probabilities of death or non-fatal MI, statistical algorithms were developed in a derivation cohort (n = 754; n = 66 reached the primary endpoint) and applied to a validation cohort (n = 368; n = 33). Adjustment of the GRACE risk estimate by GDF-15 increased the area under the receiver-operating characteristic curve (AUC) from 0.79 to 0.85 (P < 0.001) in the validation cohort. Discrimination improvement was confirmed by an integrated discrimination improvement (IDI) of 0.055 (P = 0.005). A net 31% of the patients without events were reclassified into lower risk, and a net 27% of the patients with events were reclassified into higher risk, resulting in a total continuous net reclassification improvement [NRI(>0)] of 0.58 (P = 0.002). Addition of NT-proBNP to the GRACE score led to a similar improvement in discrimination and reclassification. Addition of GDF-15 to a model containing GRACE and NT-proBNP led to a further improvement in model performance [increase in AUC from 0.84 for GRACE plus NT-proBNP to 0.86 for GRACE plus NT-proBNP plus GDF-15, P = 0.010; IDI = 0.024, P = 0.063; NRI(>0) = 0.42, P = 0.022].

Conclusion

We show that a single measurement of GDF-15 on admission markedly enhances the predictive value of the GRACE score and provides moderate incremental information to a model including the GRACE score and NT-proBNP. Our study is the first to provide simple algorithms that can be used by the practicing clinician to more precisely estimate risk in individual patients based on the GRACE score and a single biomarker measurement on admission. The rigorous statistical approach taken in the present study may serve as a blueprint for future studies exploring the added value of biomarkers beyond clinical risk scores.

Introduction

Patients with non-ST-elevation acute coronary syndrome (NSTE-ACS) are heterogeneous in terms of clinical presentation and immediate- and long-term risk of death or non-fatal ischaemic events. The current guidelines emphasize the importance of early risk stratification to select the site of care and match the intensity of therapy with an individual patient's risk.^1–3 Risk stratification is important to identify patients at high risk, in whom an invasive strategy with its adjunctive medical therapy may reduce that risk. It is equally important to identify patients at low risk in whom potentially hazardous and costly treatments provide little benefit and may cause harm.^1–6 Risk stratification is a multivariable task that needs to account for clinical characteristics and electrocardiographic and biochemical variables. The current guidelines therefore recommend a standardized approach that uses validated scoring systems, such as the Global Registry of Acute Coronary Events (GRACE) score,^7–9 to calculate a patient's risk and guide triage and management decisions.^1–3 Because risk scores reflect only some disease dimensions related to outcome in NSTE-ACS, biomarkers addressing separate aspects of NSTE-ACS pathophysiology may provide additional information. Indeed, B-type natriuretic peptide and N-terminal pro-B-type natriuretic peptide (NT-proBNP) have been shown to supplement risk assessment in NSTE-ACS in several studies.^10–15

Growth differentiation factor 15 (GDF-15) is a member of the transforming growth factor-β cytokine superfamily that is weakly expressed under healthy conditions, but produced in response to oxidative stress, inflammation, and tissue injury.¹⁶ Its prominent anti-apoptotic, anti-hypertrophic, and anti-inflammatory actions in cardiovascular disease models suggest that GDF-15 may play a counter-regulatory role in the context of cardiovascular injury.^17–19 In patients, GDF-15 has been detected in the infarcted myocardium and in atherosclerotic plaques.^18,20 Retrospective studies found that high levels of GDF-15 are associated with increased risks of death or myocardial infarction (MI) in NSTE-ACS.^21–23 The incremental value of GDF-15 in conveying prognostic information in NSTE-ACS above that provided by an established risk assessment tool such as the GRACE score has never been assessed, however.²⁴

Many strategies assessing the potential usefulness of new biomarkers focus on significance of the candidate marker added to a model containing an existing risk score as a sole predictor. Since this addition is performed and assessed using the same sample, the results are likely over-optimistic. Here, we adopt the preferable development–validation approach and focus on the predicted risk algorithm obtained using the derivation sample and applied to the validation sample. This reduces over-optimism, offers a realistic assessment, and gives practitioners a simple algorithm allowing adjustment of prior risk depending on the level of the new biomarker.

Accordingly, the purpose of the present study was to develop an algorithm to adjust GRACE-only-based risk using GDF-15 levels, to examine prospectively whether this adjustment adds predictive information to the GRACE score in unselected contemporary patients with NSTE-ACS, and to explore whether the information provided by GDF-15 is additive to that provided by NT-proBNP, a biomarker recommended for risk assessment in NSTE-ACS by the current guidelines.^1,3

Methods

Study population and follow-up

Between July 2006 and March 2010, we recruited 1122 consecutive patients with a final diagnosis of NSTE-ACS who were admitted to our departments in Heidelberg or Hannover. Refusal to provide written informed consent was the only exclusion criterion. Non-ST-elevation acute coronary syndrome was diagnosed according to the criteria of the Joint ESC/AACF/AHA/WHF Task Force.²⁵ All treatment and management decisions were left to the discretion of the attending cardiologist. An invasive strategy with coronary angiography and revascularization is the preferred practice in both departments. Deaths and non-fatal MIs were recorded during the time interval from admission to 6 months. Follow-up was accomplished by telephone contact or questionnaire at 6 months (±2 weeks) after discharge. When a patient reported another hospital admission for cardiovascular reasons during this time interval, hospital discharge letters were obtained and checked for a diagnosis of non-fatal MI. We obtained follow-up information from 98% of our patients. Only patients with completed follow-up were included in the analyses (n = 754 from Heidelberg and n = 368 from Hannover). The study was approved by the institutional committees on human research at both institutions. All patients provided written informed consent.

Calculation of the GRACE score

The GRACE risk prediction tool has been described elsewhere.⁷ The GRACE score is derived from eight variables that are readily available at hospital admission (age, heart rate, systolic blood pressure, serum creatinine concentration, Killip class, cardiac arrest, presence of ST-segment deviation, and elevated cardiac enzymes/markers). On admission, values for these variables were entered into the GRACE risk calculator (available at http://www.outcomes-umassmed.org/grace) to obtain estimates of the cumulative risks of all-cause mortality and the combined endpoint of all-cause mortality or non-fatal MI in the period from admission to hospital to 6 months. The combined endpoint was the pre-specified primary endpoint of the study. Only patients with complete GRACE score variables were included in the study. Because the GRACE risk prediction tool is not designed to estimate the risk of non-fatal MI separately (http://www.outcomes-umassmed.org/grace), our models which adjusted for the GRACE score did not analyse non-fatal MI as a separate endpoint.

Laboratory parameters and biomarker testing

Serum samples were obtained by venipuncture on admission and stored at −70°C. Serum creatinine and cardiac troponin T (cTnT) concentrations were measured at the local study sites. Cardiac troponin T was measured with the 4th generation Roche Diagnostics assay on an Elecsys 2010 platform. Cardiac troponin T values ≥0.03 μg/L, a concentration that can be measured with less than 10% total imprecision,²⁶ were regarded as an elevated marker of myocardial necrosis for diagnostic and prognostic purposes. N-terminal pro-B-type natriuretic peptide was determined with a sandwich immunoassay on an Elecsys 2010 instrument (Roche Diagnostics). Growth differentiation factor 15 was measured by an immunoradiometric assay with a limit of detection of 20 ng/L and a linear range from 200 to 50 000 ng/L.²⁷ The intra-assay imprecision of the assay ranges from 2.8 to 10.6% for samples containing 248–22 480 ng/L GDF-15, and the inter-assay imprecision ranges from 4.0 to 12.2% for samples containing 232–39 370 ng/L GDF-15.²⁷

Statistical methods

Continuous variables are reported as medians with inter-quartile range (IQR). Intergroup comparisons were performed with the Mann–Whitney U-test. Categorical variables are expressed as frequencies and percentages and compared with the χ² test. To limit the influence of extreme observations, both NT-proBNP and GDF-15 were natural logarithmically transformed to obtain ln NT-proBNP and ln GDF-15. Pearson's r was calculated to assess the correlations between NT-proBNP, GDF-15, and GRACE score points. Hazard ratios (HRs) for the associations of biomarkers with outcomes were obtained using the Cox model.

To obtain NT-proBNP- and GDF-15-adjusted 6-month predicted probabilities of all-cause mortality and all-cause mortality or non-fatal MI, we proceeded as follows. First, we fitted logistic regression to centred GRACE, NT-proBNP, and GDF-15 scores in the derivation cohort (Heidelberg patients). Regression coefficients obtained in this manner were then applied to the centred GRACE, NT-proBNP, and GDF-15 scores in the validation cohort (Hannover patients) to calculate the predicted risks. Furthermore, these were calibrated to the incidence rate in the validation cohort by simple multiplication by a ratio of incidence rate over the mean risk before calibration adjustment. To reduce over-optimism in the assessment of NT-proBNP or GDF-15 over GRACE, GRACE-only 6-month risks were also adjusted based on the derivation cohort and calibration procedures, repeating the above steps without including NT-proBNP or GDF-15 in the model. Since our main objective was to investigate improvement in discrimination and reclassification, we recalibrated the score to the validation set to achieve the most objective comparison. Unlike the area under the receiver-operating characteristic curve (AUC) which remains unaltered by such recalibration, the integrated discrimination improvement (IDI) and net reclassification improvement (NRI) can be affected. Therefore, we recalibrated to be conservative and achieve the fairest comparisons on IDI and NRI. In a secondary analysis, we also ran our models without recalibration in the validation set. The above procedures produced three sets of predicted 6-month risks: one based on the GRACE score-only and two others based on the GRACE score adjusted with either NT-proBNP or GDF-15. The performance of the biomarker-adjusted risk prediction tools was compared with the GRACE score-only prediction tool in the validation set using several measures of improvement in discrimination: increase in the AUC, as well as IDI, and continuous NRI [NRI(>0)].²⁸ Briefly, the AUC assesses the probability that given two randomly selected subjects, one who experiences an event and one who does not, the one with event has a higher predicted risk. The IDI is equal to the increase in discrimination slope defined as the mean difference in predicted risks between those with and without events. The continuous NRI is a non-parametric analogue of the IDI and equals twice the difference in probabilities of upward reclassification for events minus for non-events.²⁸ To get a sense of clinical usefulness, categorical NRIs were applied with risk categories of <6% (low risk), 6–14% (intermediate risk), and >14% (high risk) chosen in accord with the observed incidence rate of about 9%, or with risk categories of <12% (low risk), 12–21% (intermediate risk), and >21% (high risk) reflecting the tertile boundaries of GRACE-predicted risk in the validation cohort. The categorical NRI defines upward and downward reclassification only if predicted risks move from one category to another. Calibration was assessed using the Hosmer–Lemeshow χ² statistic.²⁹ Furthermore, we explored the improvement in performance achieved with GDF-15 added to a model already containing the GRACE score and NT-proBNP. The same development–validation approach described above was used, but this time, we focused on comparing models based on GRACE plus NT-proBNP vs. GRACE plus NT-proBNP plus GDF-15.

The development–validation approach adopted here is the most optimal statistical assessment as it reduces the problem of over-optimism inherent in studies that use the same sample for development and validation. However, for completeness, in the Supplementary material online, we present the results obtained when using the combined samples from Heidelberg and Hannover for both development and validation. All analyses have been performed using Statview 5.0.1 and SAS 9.2 (SAS Institute Inc., Cary, NC, USA).

Results

Patient populations, GRACE scores, N-terminal pro-B-type natriuretic peptide, and growth differentiation factor 15 concentrations

Baseline characteristics, GRACE variables, and biomarker concentrations are shown in Table 1. The derivation cohort included 754 patients (71% men) with a median age of 70 years (IQR, 60–77 years). The validation cohort consisted of 368 patients (76% men) with a median age of 69 years (IQR, 59–76 years). In the derivation cohort, the GRACE score, NT-proBNP, and GDF-15 were moderately correlated (GRACE vs. NT-proBNP, Pearson's r = 0.62; GRACE vs. GDF-15, r = 0.52; NT-proBNP vs. GDF-15, r = 0.50; each P < 0.001). Similar correlations were observed in the validation cohort (data not shown).

Table 1.

Patient populations

	Derivation cohort (Heidelberg)	Validation cohort (Hannover)	P-value
Number of patients	754	368
Male gender	532 (71)	278 (76)	0.09
Unstable angina	275 (36)	174 (47)	<0.001
NSTEMI	479 (64)	194 (53)	<0.001
Patient management
Coronary angiography	754 (100)	325 (88)	<0.001
PCI	454 (60)	212 (58)	0.44
CABG	49 (7)	43 (12)	0.004
GRACE variables
Age (years)	70 (60–77)	69 (59–76)	0.24
Heart rate (min−1)	72 (63–83)	73 (62–86)	0.62
Systolic blood pressure (mmHg)	144 (127–158)	144 (128–162)	0.22
Creatinine (μmol/L)	0.91 (0.75–1.12)	0.94 (0.83–1.18)	<0.001
Killip Class I	684 (91)	345 (94)	0.11
Class II	51 (7)	15 (4)	0.10
Class III/IV	19 (2)	8 (2)	0.88
Cardiac arrest at admission	0 (0)	1 (0.3)	0.71
ST-segment deviation	87 (12)	30 (8)	0.10
cTnT ≥0.03 μg/L	464 (62)	153 (42)	<0.001
GRACE score	128 (98–157)	112 (85–145)	<0.001
NT-proBNP (ng/L)	624 (171–2203)	419 (152–1554)	0.043
GDF-15 (ng/L)	2068 (1390–3478)	1725 (1205–2797)	<0.001

Data are shown as number (percentage) or median (inter-quartile range). NSTEMI, non-ST-elevation myocardial infarction; PCI, percutaneous coronary intervention (during the hospital course); CABG, coronary artery bypass graft surgery (within 30 days after admission); cTnT, cardiac troponin T.

Six months after admission, 66 patients from the derivation cohort (8.8%) and 33 patients from the validation cohort (9.0%) had reached the combined, primary endpoint of death (47 and 25 patients, respectively) or non-fatal MI (19 and 8 patients, respectively). Patients who reached the combined endpoint presented with higher GRACE scores and higher NT-proBNP and GDF-15 concentrations compared with patients who did not (Table 2). As illustrated in the spline plots in Figure 1, increasing values of the GRACE score, NT-proBNP, and GDF-15 were associated with increasing risks of the combined endpoint in the derivation cohort. Both biomarkers were associated with the individual endpoints of death [HR per 1 SD increment in the natural log scale for NT-proBNP, 3.2; 95% confidence interval (CI), 2.3–4.5; P < 0.001; HR for GDF-15, 2.4, 95% CI, 1.9–3.0; P < 0.001] and non-fatal MI (HR for NT-proBNP, 2.1; 95% CI, 1.3–3.3; P = 0.002; HR for GDF-15, 1.8, 95% CI, 1.2–2.6; P = 0.005). Similar associations were observed in the validation cohort (data not shown).

Table 2.

GRACE scores and biomarker concentrations in the derivation and validation cohorts

	GRACE score	NT-proBNP (ng/L)	GDF-15 (ng/L)
Derivation cohort
No event (n = 688)	124 (96–152)	509 (147–1878)	1986 (1353–3229)
Event (n = 66)	162 (145–184)	3793 (1360–10 152)	4789 (2122–8940)
P-value	<0.001	<0.001	<0.001
Validation cohort
No event (n = 335)	110 (83–139)	354 (138–1287)	1619 (1179–2485)
Event (n = 33)	159 (132–193)	3081 (1336–14 294)	4677 (2761–7371)
P-value	<0.001	<0.001	<0.001

The derivation and validation cohorts were divided into subgroups of patients who did or did not reach the primary endpoint of 6-month death or non-fatal MI. Data are shown as median with inter-quartile range.

Figure 1.

Spline plots illustrating the associations between the GRACE score, N-terminal pro-B-type natriuretic peptide, and growth differentiation factor 15 and the primary endpoint of death or non-fatal myocardial infarction at 6 months in the derivation cohort. (A) GRACE score, (B) N-terminal pro-B-type natriuretic peptide, and (C) growth differentiation factor 15. The knots represent the median and 5th and 95th percentiles of the GRACE score, N-terminal pro-B-type natriuretic peptide, or growth differentiation factor 15, respectively. The upper and lower 95% confidence limits are shown. Ln-transformed odds ratios (OR) in relation to risk at median GRACE (A) or median biomarker levels (B and C) are shown on the Y-axis (each set as 0).

Performance of single biomarker-adjusted vs. GRACE score-only prediction of the combined endpoint in the validation cohort: discrimination, net reclassification, and calibration

Applying the algorithms for GRACE score adjustment by NT-proBNP or GDF-15 developed in the derivation cohort to the validation cohort, the AUC increased from 0.79 (95% CI, 0.71–0.88) for GRACE-only-estimated risks to 0.84 (95% CI, 0.76–0.92; P = 0.015) for NT-proBNP-adjusted GRACE-estimated risks and to 0.85 (95% CI, 0.77–0.93; P < 0.001) for GDF-15-adjusted GRACE-estimated risks. Improvements in discrimination were confirmed by the IDI (0.044; 95% CI, 0.007–0.081; P = 0.019 for NT-proBNP and 0.055; 95% CI, 0.017–0.093; P = 0.005 for GDF–15), suggesting further average separation of events from non-events by these amounts.

Both biomarkers led to a significant net reclassification of patients in the appropriate directions. The continuous, category-free NRI(>0) for NT-proBNP was 0.74 (95% CI, 0.42–1.06; P < 0.001), with events contributing 0.45 and non-events 0.29 (Figure 2); the continuous NRI(>0) for GDF-15 was 0.58 (95% CI, 0.24–0.92; P = 0.001), with events contributing 0.27 and non-events 0.31 (Figure 3).

Figure 2.

Individual patient risk levels as predicted by the GRACE score and the N-terminal pro-B-type natriuretic peptide-adjusted GRACE score. Risk refers to 6-month risk of death or non-fatal myocardial infarction. Data are from the validation cohort using the algorithm for GRACE score adjustment by N-terminal pro-B-type natriuretic peptide developed in the derivation cohort. (A) Patients with events and (B) patients without events [a magnification from (B) is shown on the right as indicated by the broken line square]. See text for details.

Figure 3.

Individual patient risk levels as predicted by the GRACE score and the growth differentiation factor 15-adjusted GRACE score. Risk refers to 6-month risk of death or non-fatal myocardial infarction. Data are from the validation cohort using the algorithm for GRACE score adjustment by growth differentiation factor 15 developed in the derivation cohort. (A) Patients with events and (B) patients without events (a magnification from (B) is shown on the right as indicated by the broken line square). See text for details.

Potential for clinical benefit achieved when NT-proBNP or GDF-15 were added to the GRACE score was assessed using the category-based NRI. Using 6 and 14% as arbitrary thresholds to define patients at low, intermediate, and high risk, NT-proBNP achieved an NRI of 0.12 (95% CI, –0.04 to 0.29; P = 0.14), whereas GDF-15 achieved an NRI of 0.23 (95% CI, 0.08–0.38; P = 0.003); of 335 patients without events, 54 were correctly downgraded and 17 were wrongly upgraded by at least one category by GDF-15 (NRI for non-events = 0.11), whereas of 33 patients with an event, 5 were correctly upgraded, and 1 was wrongly downgraded (NRI for events = 0.12) (Table 3).

Table 3.

Reclassification across pre-defined risk thresholds in the validation cohort using the algorithm for GRACE score adjustment by growth differentiation factor 15 developed in the derivation cohort

	GRACE score adjusted by GDF-15
	<6%	6–14%	>14%	All
Patients without events
GRACE score
<6%	194 (58)	11 (3)	1 (0.3)	206 (61)
6–14%	25 (7)	33 (10)	5 (1)	63 (19)
>14%	4 (1)	25 (7)	37 (11)	66 (20)
All	223 (67)	69 (21)	43 (13)	335 (100)
				NRI = 0.11
Patients with events
GRACE score
<6%	5 (15)	2 (6)	0	7 (21)
6–14%	1 (3)	2 (6)	3 (9)	6 (18)
>14%	0	0	20 (61)	20 (61)
All	6 (18)	4 (12)	23 (70)	33 (100)
				NRI = 0.12

The number (percentage) of patients in each risk category is shown. Patients were divided into subgroups that did or did not reach the primary endpoint of 6-month death or non-fatal MI. NRI, net reclassification improvement. Total category-based NRI was 0.23 (95% CI: 0.08–0.38; P= 0.003).

Using the tertile boundaries of GRACE-predicted risk in the validation cohort (12 and 21%) to categorize risk, adjustment of GRACE-predicted risks by NT-proBNP resulted in an NRI of 0.18 (95% CI, 0.01–0.36; P = 0.051) and adjustment by GDF-15 in an NRI of 0.23 (95% CI, 0.09–0.36; P = 0.003).

All three prediction tools achieved good calibration as evidenced by the Hosmer–Lemeshow χ² values of 0.04 for GRACE-only (P = 0.98), 0.24 for NT-proBNP-adjusted (P = 0.89), and 0.44 for GDF-15-adjusted risk estimates (P = 0.80) (see Supplementary material online, Figure S1).

Incremental value of adding growth differentiation factor 15 to a model containing GRACE and N-terminal pro-B-type natriuretic peptide

To explore whether GDF-15 adds prognostic information on the combined endpoint of death or non-fatal MI to a model containing the GRACE score and NT-proBNP, we developed an algorithm for GRACE score adjustment by NT-proBNP and GDF-15 in the derivation cohort, applied it to the validation cohort, and compared it with the algorithm using only NT-proBNP for GRACE score adjustment. Addition of GDF-15 improved model discrimination with an increase in the AUC from 0.84 (95% CI, 0.76–0.92) for GRACE plus NT-proBNP to 0.86 (95% CI, 0.78–0.94) for GRACE plus NT-proBNP plus GDF-15 (difference in the AUCs, 0.020; 95% CI, 0.005–0.034; P = 0.010). A trend in the same direction was observed for the IDI (0.024; 95% CI, –0.001 to 0.049; P = 0.063). Moreover, addition of GDF-15 to the model containing GRACE and NT-proBNP led to a significant net reclassification of patients' risk using either the category-free or category-based NRIs (Table 4).

Table 4.

Net reclassification improvement by growth differentiation factor 15 on top of the GRACE score and N-terminal pro-B-type natriuretic peptide in the validation cohort using the algorithm for risk adjustment developed in the derivation cohort

	NRI (95% CI)	Patients with events	Patients without events	P-value
NRI(>0)	0.42 (0.07–0.77)	0.21	0.21	0.022
NRI (6/14%)	0.20 (0.04–0.35)	0.15	0.04	0.017
NRI (12/21%)	0.21 (0.07–0.34)	0.18	0.03	0.006

Net reclassification improvement (NRI) was assessed using the category-free NRI(>0) and two category-based NRIs using 6 and 14% or 12 and 21% as cut-offs to define patient subgroups at low, intermediate, or high risk. Reclassification of patients that did or did not reach the primary endpoint of 6-month death or non-fatal MI is shown. CI, confidence interval.

Secondary analyses

The results remained almost identical when the models were re-run without recalibration in the validation set.

Results remained consistent when we developed and validated our algorithms on all 1122 patients, i.e. the Heidelberg and Hannover cohorts combined (see Supplementary material online, Tables S1–S3). Algorithms, calibrated to the combined Heidelberg–Hannover population, allowing adjustment of the GRACE score risk based upon levels of NT-proBNP and/or GDF-15 are provided at the end of the Supplementary material online.

Applying the same statistical metrics on the secondary endpoint of death, we found that both NT-proBNP and GDF-15 enabled a similar reclassification and improvement in discrimination when added to the GRACE score. The effect of adding GDF-15 to the model with GRACE and NT-proBNP was not significant in the validation cohort but tended to improve discrimination and led to a modest further net reclassification when examined using the combined cohorts (see Supplementary material online, Tables S4–S9).

Discussion

Using a development–validation design, the present study shows that a single measurement of GDF-15 on admission enhances the predictive value of the GRACE score in contemporary, unselected patients with NSTE-ACS. The prognostic information provided by GDF-15 is additive to that provided by NT-proBNP, a biomarker already in clinical use for risk assessment in NSTE-ACS.

Management decisions in NSTE-ACS should be based on a rapid and accurate assessment of risk.^1–3 Physicians relying on a ‘subjective’ assessment of risk may fail to consider important prognostic factors, and physicians' underestimation of risk may result in high-risk patients paradoxically receiving less intensive therapies.^30–33 Validated scoring systems, such as the GRACE score, provide incremental prognostic information beyond subjective risk assessment in NSTE-ACS; however, the ability of scoring systems to discriminate outcome groups leaves room for improvement (AUCs between ∼0.7 and 0.8 have been reported).^7,8,34 Part of this limitation is related to the stochastic nature of cardiovascular events and the difficulty to predict outcome based on an assessment of risk at a single point in time. In addition, disease dimensions, such as inflammation or myocardial strain that are related to outcome in NSTE-ACS,³⁵ are not fully captured by the variables included in scoring systems. Biomarkers such as NT-proBNP and GDF-15 reflecting such additional mechanisms might therefore enhance risk assessment beyond scoring systems. Indeed, GDF-15 and NT-proBNP levels were moderately correlated with the GRACE score in our study, indicating that these markers are related to disease pathways not fully represented by the GRACE variables.

Notably, GDF-15 levels in NSTE-ACS do not show the rise and fall pattern observed with necrosis biomarkers, NT-proBNP, or inflammatory markers such as C-reactive protein, but remain in a narrow range from admission up to several months after the event, indicating that GDF-15 levels are not related to the acute injury.^21,36 Growth differentiation factor 15 levels are independently related to cardiovascular risk factors (diabetes, smoking, low HDL cholesterol) and biochemical risk markers (high-sensitivity C-reactive protein, NT-proBNP) in elderly individuals and patients with coronary artery disease.^21–23,37 Moreover, growth differentiation factor 15 was found to be associated with endothelial dysfunction, plaque burden, left ventricular hypertrophy, and systolic dysfunction in elderly individuals.³⁷ Thus, these associations with underlying cardiovascular disease burden can be expected to explain some of the prognostic value of GDF-15.

Increasing levels of GDF-15 were associated with an increased risk of death or non-fatal MI in our study. More importantly, GDF-15 added discriminatory information to the GRACE score as evidenced by a considerable increase in the AUC from 0.79 to 0.85 for the combined primary endpoint of death or non-fatal MI in the validation cohort. An interaction between GDF-15 levels and benefit from an invasive strategy was observed in a retrospective analysis of the Fast Revascularisation during InStability in Coronary artery disease (FRISC) 2 trial.²² As a consequence, the association between GDF-15 and death or non-fatal MI was stronger in the non-invasive group compared with the invasive group in FRISC-2.²² If anything, the invasive strategy used in most of our patients may therefore have mitigated the prognostic value of GDF-15 in the present study. Because most of our patients were treated with an invasive treatment strategy, which is known to reduce the risk of death/MI in NSTE-ACS,^4–6 our risk estimates might underestimate the absolute risks of the patients who had been managed non-invasively.

Adjustment of GRACE-predicted risks by GDF-15 led to a substantial proportion of patients appropriately being reclassified into higher or lower risks. Using a recently developed category-free, continuous NRI(>0),²⁸ we found that a net 31% of the patients without events were reclassified into lower risk and that a net 27% of patients with events were reclassified into higher risk. The category-free, continuous NRI(>0) thus reached an impressive 0.58. As a reference point, in the multivariable normal case, an NRI(>0) of this magnitude would correspond to an effect size of about 0.8, which Cohen classified as strong.³⁸ Because risk categories have not been established for the GRACE score, the category-free, continuous NRI may be a more objective measure of improvement in risk prediction compared with an NRI that is based on arbitrarily chosen categories (which we still present here to gain insight into potential clinical usefulness).²⁸

Using the same development–validation approach and statistical metrics, we found that NT-proBNP, compared with GDF-15, enabled a similar reclassification and improvement in discrimination when added to the GRACE score. Addition of GDF-15 to a model containing GRACE and NT-proBNP led to a further moderate improvement in reclassification and discrimination, indicating that GDF-15 provides information that is additive not only to the GRACE score but also to NT-proBNP, a marker already in clinical use.

Our study has a number of strengths and limitations that merit consideration. The rigorous development–validation design used in the present study is a strength as it avoids over-optimism in assessing the incremental predictive information provided by GDF-15.

Three studies have previously explored whether biomarkers can add prognostic information to the GRACE score.^39–41 None of these studies used the inclusion and outcome criteria, for which the score was developed. This is not trivial, because application of the score outside of its intended use will leave more room for biomarkers to improve on the score. Eggers et al.,⁴¹ for example, investigated the incremental value of four biomarkers, including NT-proBNP and GDF-15, in a heterogeneous cohort of 453 patients with chest pain and looked at long-term mortality (only 224 of their patients had an ACS, median follow-up was 5.8 years). Ours is the very first study that created optimal conditions for the GRACE score to predict outcome events by utilizing the patient population (NSTE-ACS), the follow-up interval (admission to 6 months), and the primary endpoint (all-cause mortality or non-fatal MI) for which the score was developed. Thus, our study is the very first to unequivocally demonstrate that biomarkers can add meaningful information to the GRACE score.

Since we recalibrated the model in our validation cohort, calibration (agreement between predicted and observed risks) may be overstated. Because the rates of death or non-fatal MI were similar in our derivation and validation cohorts (8.8 vs. 9.0%), the degree of such overestimation was small and, indeed, our results remained almost identical when the models were run without recalibration in the validation set. In any case, before applying our model, clinicians should recalibrate the model to the incidence rate of death or non-fatal MI in their own population (if the event rates are likely to differ between our population and the one in which the score is to be applied).

As a potential limitation, the number of patients and outcome events in the validation cohort was relatively small. Our findings, however, were robust and reproducible when analysing the combined patient population. Moreover, as we included only patients with complete GRACE score variables, very sick patients with missing variables could not be included in our analyses. Finally, although we carefully ascertained all non-fatal MIs during 6-month follow-up, we cannot exclude that we missed a few events.

In conclusion, GDF-15 and NT-proBNP, when measured individually, enable a more accurate appreciation of risk in NSTE-ACS on top of the GRACE score. In combination, both markers enable a further moderate improvement in risk stratification. Whether application of the GRACE score, alone or in combination with biomarkers such as GDF-15 and NT-proBNP, can improve patients' outcome in clinical practice needs further prospective evaluation. This would require defining risk thresholds linked to specific management decisions in NSTE-ACS.

Supplementary material

Supplementary material is available at European Heart Journal online.

Funding

This work was supported by the German Ministry of Education and Research (BMBF, BioChancePlus to K.C.W.).

Conflict of interest: T.K. and K.C.W. hold a patent and have a contract with Roche Diagnostics to develop a GDF-15 assay for cardiovascular applications.