Comparison of contemporary preoperative risk models at predicting acute kidney injury after isolated coronary artery bypass grafting: a retrospective cohort study

Abstract

Objectives

Acute kidney injury (AKI) after coronary artery bypass grafting (CABG) is associated with short-term and long-term adverse outcomes. The European System for Cardiac Operative Risk Evaluation (EuroSCORE), EuroSCORE II, the Society of Thoracic Surgeons (STS) score and Age, Creatinine and Ejection Fraction (ACEF) score, have been widely used for predicting the operative risk of cardiac surgery. The aim of this study is to investigate the discriminant ability among current available models in predicting postoperative AKI.

Methods

From January 2010 to December 2012, 353 patients who underwent isolated CABG were enrolled. The clinical characteristics, outcomes and scores of prognostic models were collected. The primary outcome was postoperative AKI, defined based on the Kidney Disease Improving Global Outcome (KDIGO) Clinical Practice Guideline for AKI, in 2012.

Results

102 patients (28.9%) developed postoperative AKI. For AKI prediction, EuroSCORE II, STS score and ACEF score were all good tools for stage-3 AKI. The ACEF score was shown to have satisfied discriminant ability to predict postoperative AKI with area under a receiver operating characteristic curve: 0.781±0.027, (95% CI 0.729 to 0.834, p value <0.001). Multivariate logistic analysis identified that lower ejection fraction and higher serum creatinine were independent risk factors for AKI.

Conclusions

The simple and extremely user-friendly ACEF score can accurately identify the risk of postoperative AKI and has shown satisfactory discriminant ability when compared with other systems. The ACEF score might be the easiest tool for predicting postoperative AKI.

Strengths and limitations of this study.

• This is the first assessment comparing the prognostic utility of contemporary risk scoring systems for predicting post-operative acute kidney injury (AKI) in isolated coronary artery bypass grafting (CABG).

• We found that preoperative risk models in cardiac surgery, including European System for Cardiac Operative Risk Evaluation (EuroSCORE), EuroSCORE II, Society of Thoracic Surgeons score and Age, Creatinine and Ejection Fraction (ACEF score), can be used in predicting postoperative AKI in isolated CABG patients. The prediction ability was positively correlated with AKI severity.

• The simple and extremely user-friendly ACEF score can accurately identify the risk of postoperative AKI and has shown satisfactory discriminatory ability when compared with other systems; the ACEF score may be the easiest tool to provide guidance of preventive and early therapeutic strategies for AKI to improve the clinical outcomes of patients.

• Our study is limited by its post hoc analysis nature and all of its inherent limitations. Also, this series has the heterogeneity of off pump/on pump surgery, and urgent and elective surgery.

• Since the aetiology of AKI is often multifactorial, intraoperative factors and post-surgical care that were not involved in the scoring system may cause inaccurate prediction of AKI occurrence.

Introduction

Acute kidney injury (AKI) affects 12–48% of patients undergoing coronary artery bypass grafting (CABG).¹ This statistical variant is dependent on different classifications and study populations.^2–6 Postoperative AKI not only contributes to increased in-hospital mortality and decreased long-term survival, but also results in high medical expenditure, chronic kidney disease and dialysis dependence.^7–9 Since AKI has been known as a well-established predictor of all-cause mortality in CABG, an accurate and validated prediction model for AKI after cardiac surgery would be invaluable for clinical practices.

Numerous prognostic risk models for cardiac surgery have been introduced into the current practices. Among them, the European System for Cardiac Operative Risk Evaluation (EuroSCORE), published in 1999,¹⁰ the revised EuroSCORE II, published in 2012,¹¹ and the Society of Thoracic Surgeons (STS) score, published in 2008, have been widely used.¹² The patient's age, creatinine and ejection fraction (ACEF) score was first described in 2009, in a publication for quick bedside evaluations.¹³ Although all these scoring models have been widely used for assessing the risk of post-surgical death, the various preoperative demographic and clinical variables have shown only that the STS database has successfully implicated the end point of renal failure.¹⁴

However, the benefits of utilising the risk model may be very limited due to the low dialysis rate in 1–5% cases.¹⁵ More effort is required to develop and validate different prediction scores to identify all levels of severity of AKI, since stage 1 AKI is very common and contributes to short-term and long-term adverse outcomes with almost threefold increase in the long-term risk of end stage renal disease (ESRD), as demonstrated by a nationwide study.¹⁶ Nevertheless, no previous study has ever applied the contemporary preoperative risk modes to predict occurrence of AKI and its severity after surgery. The aim of this investigation is to compare the utility of scoring systems for predicting AKI after isolated CABG.

Patients and methods

Study design and patient population

This post hoc analysis of a prospective collected database was approved by the institution's research bureau (IRB) of Chang Gung Memorial Hospital, where the need for individual consent was waived. Medical records, from January 2010 to December 2012, of 440 consecutive patients who had received isolated CABG in a single tertiary referral hospital were reviewed. We excluded five patients who had had prior cardiac surgery, seven patients who had received extracorporeal membrane oxygenation (ECMO) immediately after surgery and six patients who had died on the day of surgery. In order to appraise renal outcomes, 57 patients who had undergone dialysis and 12 patients who had experienced AKI before their operation, were also excluded. The final cohort comprised a total of 353 patients.

Data collection and definitions

Clinical characteristics, demographic data and EuroSCORE I were extracted from the database. The STS score and EuroSCORE II were re-calculated with an online calculator. The ACEF score was calculated by age/ejection fraction (EF) +1 (if creatinine >2.0 mg/dL). The primary outcome was AKI and 3-month mortality. Base on the KDIGO Clinical Practice Guideline for Acute Kidney Injury, AKI was defined as any of the following: increase in serum creatinine (SCr) ≥0.3 mg/dL within 48 hours or increase in SCr ≥1.5 times baseline in 7 days or urine volume <0.5 mL/kg/hour for 6 hours. Urine output and SCr level were routinely measured and the data collected in our medical records. Finally, patients were categorised into three different severities, using the KDIGO guideline^{17 18} (table 1).

Table 1.

Staging and definition of AKI according to the KDIGO guidelines in 2012

Definition	Stage	Serum creatinine	Urine output
Increase in SCr by ≥0.3 mg/dL (≥26.5 μmol/L) within 48 hours or Increase in SCr to ≥1.5 times baseline, which is known or presumed to have occurred within the prior 7 days or Urine volume <0.5 mL/kg/hour for 6 hours	1	1.5–1.9 times baseline or ≥0.3 mg/dL (≥26.5 μmol/L) increase	<0.5 mL/kg/hour for 6–12 hours
	2	2.0–2.9 times baseline	<0.5 mL/kg/hour for ≥12 hour
	3	3.0 times baseline or Increase in serum creatinine to ≥4.0 mg/dL (≥353.6 μmol/L) or Initiation of renal replacement therapy or, In patients <18 years, decrease in eGFR to <35 mL/min per 1.73 m2	<0.3 mL/kg/hour for ≥24 hours or Anuria for ≥12 hour

AKI, acute kidney injury; eGFR, estimated glomerular filtration rate; SCr, serum creatinine.

Statistical analysis

Continuous variables were summarised as mean and SE unless otherwise stated. The primary end point was the comparison between AKI and non-AKI groups. The Kolmogorov-Smirnov test was used to determine the normal distribution for each variable, and the Student's t-test was used to compare the means of continuous variables and normally distributed data; otherwise, the Mann-Whitney U test was used. Categorical data were tested using the χ² test or Fisher's exact test. Risk factors for AKI were assessed using univariate analysis, and variables that were statistically significant in the univariate analysis were included in multivariate logistic regression analysis based on the backward elimination of data. Discrimination was assessed using area under a receiver operating characteristic curve (AUROC) analysis, and the discriminatory ability between the ACEF and other scoring systems was compared using a non-parametric approach. The AUROC analysis calculated cut-off values, sensitivity, specificity and overall correctness. Finally, cut-off points were calculated using the best Youden index, defined as sensitivity+specificity–1, where sensitivity and specificity were calculated as proportions. Youden's index has minimum and maximum values of −1 and +1, respectively, with a value of +1 representing the optimal value for an algorithm. A p value of <0.05 was considered to indicate statistical significance.

Results

Characteristics of the study population: non-AKI versus AKI groups

The study cohort consisted of 353 adult patients with a mean age of 64±1.0 years (288 (81.6%) males and 65 females), of whom 102 (28.9%) developed postoperative AKI (46 (45.1%) stage I, 14 (13.7%) stage II and 42 (41.2%) stage III). All clinical characteristics are listed in table 2. Compared to the patients without AKI, those with AKI were older, had a higher frequency of diabetes mellitus (DM), higher level of SCr, lower level of albumin and poorer preoperative heart condition including lower EF, higher frequency of the use of preoperative intra-aortic balloon pumping (IABP), more recent myocardial infarction (MI) and more vessels with coronary artery disease. The mean EuroSCORE I, EuroSCORE II and STS scores for the risk of mortality were 12.6±0.9%, 5.4±0.5% and 5.7±0.6%, respectively. All preoperative scores were significantly different between the AKI and non-AKI groups. Twenty (5.7%) patients died within 30 days, with a significantly higher rate in the AKI group than in the non-AKI group (18.6% vs 0.4%; p<0.001). Furthermore, the patients in the AKI group also had longer durations of ventilator use and intensive care unit stay.

Table 2.

Demographic data and clinical characteristics of patients with/without AKI during their hospital course (expression as mean±SE)

	All Patients (n=353)	Non-AKI (n=251)	AKI (n=102)	p Value
Preoperative demographic data
Age (years)	64±1	63±1	67±1	0.001
Gender, male (%)	288 (81.6)	202 (80.5)	86 (84.3)	0.399
Diabetes mellitus (%)	176 (49.9)	112 (44.6)	64 (62.7)	0.002
Hypertension (%)	275 (77.9)	186 (74.1)	89 (87.3)	0.007
ALT (units/L)	36±3	32±2	47±12	0.220
Serum creatinine (mg/dL)	1.1±0.0	1.0±0.0	1.5±0.1	<0.001
Albumin (g/L)	3.9±0.0	4.0±0.0	3.7±0.1	<0.001
Haemoglobin (g/dL)	12.9±0.1	13.1±0.	12.3±0.2	0.006
hs-CRP (mg/L)	20.0±2.4	15.3±2.2	31.5±5.7	<0.001
Preoperative heart condition
CAD vessels	2.8±0.1	2.8±0.1	2.9±0.1	0.049
Recent MI, (%)	157 (44.5)	96 (38.2)	61 (59.8)	<0.001
CHF Fc III/IV (%)	51 (14.4)	22 (8.8)	29 (28.4)	<0.001
Ejection fraction (%)	54±1	57±2	45±6	<0.001
Inotropic agent (%)	38 (10.8)	21 (8.4)	17 (16.7)	0.023
Mechanical ventilation, n (%)	29 (8.2)	11 (4.4)	18 (17.6)	<0.001
IABP, (%)	37 (10.5)	14 (5.6)	23 (22.5)	<0.001
Preoperative scores
STS-risk of mortality	5.7±0.6	3.4±0.4	11.2±1.8	<0.001
STS-renal failure	6.9±0.7	3.7±0.4	14.9±1.7	<0.001
EuroSCORE I	12.6±0.9	8.7±0.8	22.2±2.4	<0.001
EuroSCORE II	5.4±0.5	3.5±0.3	10.3±1.3	<0.001
ACEF	1.4±0.1	1.2±0.1	1.9±0.1	<0.001
Surgical detail
Urgent operation (%)	87 (24.6)	48 (19.1)	39 (38.2)	<0.001
On pump CABG (%)	243 (68.8)	158 (62.9)	85 (83.3)	<0.001
Aortic clamp time (minutes)	91.3±3.4	87.0±3.2	96.4±6.6	0.184
Cardiopulmonary bypass time (minutes)	117.3±2.9	109.6±2.8	131.4±6.2	0.002
Bypass graft number	3.0±0.1	3.0±0	3.1±0	0.167
Patient outcome
ICU stay (days)	5.3±0.8	2.9±0.2	7.9±1.1	<0.001
Ventilator duration (hours)	38.3±7.5	22.8±2.6	58.4±11.8	0.004
AKI stage 1/2/3	–	–	46/14/42	–
Renal replacement therapy n (%)	28 (7.9)	–	28 (27.4)	–
30-day mortality	20 (5.7)	1 (0.4)	19 (18.6)	<0.001

ACEF, age, creatinine and ejection fraction; AKI, acute kidney injury; ALT, alanine transaminase; CAD, coronary artery disease; CHF Fc, congestive heart failure functional class; hs-CRP, high-sensitivity C reactive protein; IABP, intra-aortic balloon pump; ICU, intensive care unit; MI, myocardial infarction; NS, not significant; STS, Society of Thoracic Surgeons.

Scoring systems and AKI prediction

We tested the four mortality scores for their discriminatory ability in predicting all AKI and stage 3 AKI (table 3). The ACEF score had the best AUROC (0.781±0.027) to predict postoperative AKI among the five scoring systems, followed by STS-renal failure (0.765±0.029), EuroSCORE II (0.738±0.030), STS-risk of mortality (0.714±0.031) and EuroSCORE I (0.697±0.032). The ACEF score outperformed the STS-risk of mortality (p=0.016) and EuroSCORE I (p=0.003), but it was comparable with STS-renal failure and EuroSCORE II (see online supplementary table S1). In subgroup analysis, all the scoring systems, including EuroSCORE, had significant discriminatory ability for AKI stage 3. The AUROC was highest for STS-renal failure (0.892±0.025), followed by EuroSCORE II (0.851±0.030), STS-risk of mortality (0.843±0.032) and ACEF (0.838±0.034), and lowest for EuroSCORE I (0.776±0.038). The performance between the ACEF score and the other four scoring systems was comparable (see online supplementary table S1).

Table 3.

Comparison of calibration and discrimination of the scoring system in predicting AKIs of differing severity

	All AKI			AKI stage 3
Scoring system	AUROC±SE	95% CI	p Value	AUROC±SE	95% CI	p Value
STS-risk of mortality	0.714±0.031	0.653 to 0.775	<0.001	0.843±0.032	0.781 to 0.906	<0.001
STS-renal failure	0.765±0.029	0.709 to 0.822	<0.001	0.892±0.025	0.842 to 0.942	<0.001
EuroSCORE I	0.697±0.032	0.635 to 0.759	<0.001	0.776±0.038	0.701 to 0.851	<0.001
EuroSCORE II	0.738±0.030	0.680 to 0.797	<0.001	0.851±0.030	0.793 to 0.910	<0.001
ACEF	0.781±0.027	0.729 to 0.834	<0.001	0.838±0.034	0.772 to 0.904	<0.001

ACEF, age, creatinine and ejection fraction; AKI, acute kidney injury; AUROC, areas under the receiver operating characteristic curve; STS, Society of Thoracic Surgeons.

In order to assess cut-off points to predict AKI, we determined the sensitivity and specificity (table 4). The optimal cut-off point for the ACEF score to predict AKI was 1.2, with a sensitivity of 82% and specificity of 67%. With regard to predicting stage 3 AKI, a cut-off ACEF value of 1.5 had an even higher sensitivity of 83%, and specificity of 72%.

Table 4.

Prediction of AKI in different scoring systems in isolated coronary artery bypass surgery

Predictive factors	Cut-off point	Youden index	Sensitivity (%)	Specificity (%)
All AKI
STS-risk of mortality	2.3	0.35	61	75
STS-renal failure	4.9	0.42	60	82
EuroSCORE I	5.9	0.35	72	63
EuroSCORE II	3.0	0.39	70	68
ACEF	1.2	0.46	82	67
AKI stage 3
STS-risk of mortality	3.9	0.56	76	80
STS-renal failure	5.9	0.65	83	81
EuroSCORE I	9.1	0.41	74	67
EuroSCORE II	3.2	0.52	88	64
ACEF	1.5	0.55	83	72

ACEF, age, creatinine and ejection fraction; AKI, acute kidney injury; STS, Society of Thoracic Surgeons.

To determine the discriminatory ability of the STS-renal failure score, we plotted calibration curves by quintile of STS-renal failure score for all AKIs (figure 1). This score was found to underestimate postoperative AKI in all quintiles, with increasing discrepancy in the higher quintiles.

Figure 1.

Quintile calibration plots of STS-renal failure for expected AKI and observed AKI. AKI, acute kidney injury; STS, Society of Thoracic Surgeons.

Logistic regression analysis for AKI according to preoperative variable

A logistic regression model was used for the patients with AKI according to the preoperative factors. After incorporating the significant independent variables in the univariate analysis into the multiple logistic regression analysis with backward selection, only lower ejection fraction and higher SCr level were independently associated with the risk of postoperative AKI (table 5).

Table 5.

Logistic regression analysis of presurgical factors for all AKI

Parameter	β-Coefficient	SE	OR (95% CI)	p Value
Univariate logistic regression
Age	0.036	0.011	1.037 (1.015 to 1.060)	0.001
Diabetes mellitus	0.737	0.241	2.090 (1.303 to 3.352)	0.002
Hypertension	−0.872	0.330	0.418 (0.219 to 0.798)	0.008
Inotropic agent	0.784	0.350	2.190 (1.103 to 4.350)	0.025
Mechanical ventilation	−1.542	0.403	0.214 (0.097 to 0.471)	<0.001
Serum creatinine	2.114	0.329	8.284 (4.346 to 15.792)	<0.001
Albumin	−1.349	0.354	0.260 (0.130 to 0.520)	<0.001
Haemoglobin	−0.146	0.054	0.864 (0.777 to 0.961)	0.007
hs-CRP	0.016	0.005	1.016 (1.005 to 1.027)	0.004
IABP	−1.595	0.363	0.203 (0.100 to 0.413)	0.001
Ejection fraction	−0.048	0.008	0.953 (0.938 to 0.968)	<0.001
Recent MI	0.876	0.240	2.402 (1.500 to 3.846)	<0.001
CAD vessels	0.511	0.296	1.667 (0.993 to 2.980)	0.084
Multivariate logistic regression
Age	0.034	0.019	1.035 (0.997 to 1.074)	0.072
Serum creatinine	3.005	0.667	20.283 (5.459 to 70.659)	<0.001
Ejection fraction	−0.042	0.014	0.959 (0.933 to 0.986)	0.003
CHF Fc III/IV	0.934	0.537	2.545 (0.889 to 7.287)	0.082
constant	−4.421	1.385	–	–

ACEF, age, creatinine and ejection fraction; AKI, acute kidney injury; CAD, coronary artery disease; CHF Fc, congestive heart failure functional class; hs-CRP, high-sensitivity C reactive protein; MI, myocardial infarction; STS, Society of Thoracic Surgeons.

Short-term outcomes

Figure 2 illustrates the cumulative survival rate of all 353 isolated CABG patients stratified by the severity of AKI. As expected, the patients with stage 2 or 3 AKI had a significantly lower survival rate (log rank p<0.001). Compared to the patients without AKI, although there was no significant difference in survival, those with stage 1 AKI had a higher number of admissions for recurrent angina and hyperkalaemia during follow-up.

Figure 2.

Short-term survival according to non-AKI and AKI groups. AKI, acute kidney injury.

Discussion

The development of AKI is associated with unfavourable outcomes and a high mortality rate in patients undergoing isolated CABG. The mechanisms of AKI after cardiac surgery are multifactorial, and can include ischaemic-reperfusion, cytokine release, haemolysis, oxidase stress and exposure to nephrotoxins. All of these factors can result in endothelial dysfunction, inflammatory responses and tubule cell damage. AKI after cardiac surgery can be classified as a form of cardiorenal syndrome type I, a bidirectional condition that reflects an abrupt degradation of renal function, secondary to acute cardiac disease or procedures.¹⁹ Since renal dysfunction is a well-established predictor of all-cause mortality in patients undergoing CABG, an accurate and validated prediction model for AKI after cardiac surgery would be invaluable to facilitate clinical decision-making, patient counselling and preoperative medical optimisation.²⁰ Currently, the prediction models for severe AKI requiring dialysis are the most robust and externally validated. Among them, the STS score for renal failure has been shown to have a high discriminatory ability in most tested populations.²¹ However, due to the low dialysis rate (1–2%) and late occurrence, the benefits of using these risk models may be limited, and further research is required to develop and validate different prediction scores to identify all severities of AKI, since stage 1 AKI is very common, and contributes to many short-term and long-term adverse outcomes.

Several validated scoring models, including EuroSCORE I and II, and STS score, have been developed and are widely used to assess the risk of mortality in patients undergoing cardiac surgery. These risk scores are based on collected data and use online tools to calculate scores, which can then predict the risk of mortality from cardiac surgery based on patient demographics and clinical variables. Recently, these scoring systems have been expanded to incorporate clinical characteristics specific for cardiac surgery, and have been applied to patients undergoing percutaneous coronary interventions due to the similar study population.^22–24 Thus, these contemporary preoperative mortality risk scoring systems have been used to predict major complications such as postoperative AKI. Furthermore, our recent study demonstrated that the STS score is also an effective tool to predict AKI in patients with coronary artery disease without CABG.²⁵ In another study, Ando et al ²⁶demonstrated that the ACEF score can be used as a predictor of AKI in patients undergoing primary percutaneous coronary interventions. The major risk factors for postoperative mortality after cardiac surgery are well known, and postoperative major complications share the same risk factors that may contribute to the development of AKI, such as decreased left ventricle ejection fraction and increased SCr level, as demonstrated in this study. We also demonstrated that the current mortality risk scoring systems can be used to predict post-CABG AKI.

The ACEF score is based on the ‘law of parsimony’ to exclude some of the confounders that could bias the other current scoring systems, where the incorporation of too many variables has resulted in inaccuracies and poor overfitting.¹³ Further, Andò et al²⁷ demonstrated that the predictive ability of the ACEF score can, often, be refined by the use of additional prognostic indicators that are specific to the population studied. A recent study reported that the ACEF score is a similar or more accurate model to stratify the risk of mortality in patients undergoing CABG and in patients undergoing percutaneous coronary interventions.²⁸ This simple score combines three important clinical variables: age, creatinine (renal insufficiency) and left ventricular ejection fraction. These three preoperative clinical variables are well known independent risk factors for postoperative AKI in patients undergoing cardiac surgery. Therefore, the ACEF scoring system may be a useful and applicable risk model to predict postoperative AKI. Furthermore, because it uses clinical variables that can be obtained easily and quickly, it is more suitable for non-elective surgery. A recent study demonstrated that the performance of the ACEF score in predicting in-hospital mortality in elective and non-elective cardiac surgery is comparable.²⁹ We also demonstrated its superior application in predicating all severities of AKI in elective and non-elective surgery. Currently, novel biomarkers allow for the early detection of postoperative AKI, however, the cost of screening every case may be a concern.^{30 31} Hence, combining risk assessment tools and post-surgical biomarkers may improve the cost efficacy in daily practice.

Study limitations

There are several limitations to this study. First, the nature of post hoc analysis and all of its inherent limitations. Second, this series has a relatively small number of cases, with heterogeneity for off pump/on pump surgery, and urgent and elective surgery. Finally, this study was conducted at a single tertiary care medical centre in Taiwan, and thus the results may not be directly extrapolated to other patient populations. Finally, since the aetiology of AKI is often multifactorial, scoring systems not including intraoperative factors and postsurgical care may cause inaccuracies in the prediction of the risk of AKI.

Conclusions

To the best of our knowledge, this is the first study to compare the prognostic use of contemporary risk scoring systems to predict postoperative AKI in patients undergoing isolated CABG. The principle findings of the current study are: first, preoperative risk models, including the EuroSCORE II, STS and ACEF scoring systems, can be used to predict postoperative AKI in patients undergoing isolated CABG. The prediction ability was positively correlated with the severity of AKI. Second, the multivariate analysis identified a low ejection fraction and high SCr level to be independently associated with postoperative AKI. Third, the simple ACEF score can accurately identify the risk of postoperative AKI with satisfactory discriminatory ability compared with the other systems. In conclusion, the ACEF score may be the best and easiest tool to guide preventive and early therapeutic strategies for AKI to improve patient clinical outcomes. Future studies should be focused on combining preoperative risk models and postoperative AKI biomarkers to allow for an accurate diagnosis for future clinical management.