Utility of Biomarkers to Improve Prediction of Readmission or Mortality after Cardiac Surgery

Abstract

Background.

Hospital readmission within 30 days is associated with higher risks of complications, death, and increased costs. Accurate statistical models to stratify the risk of 30-day readmission or death after cardiac surgery could help clinical teams focus care on those patients at highest risk. We hypothesized biomarkers could improve prediction for readmission or mortality.

Methods.

Levels of ST2, Galectin-3, NTproBNP, Cystatin C, IL-6, and IL-10 were measured in samples from 1046 patients discharged after isolated CABG surgery from 8 medical centers with external validation in 1194 patients from 5 medical centers. Thirty-day readmission or mortality were ascertained using Medicare, state all-payer claims, and the National Death Index. We tested and externally validated the clinical models and the biomarker panels using AUROC statistics.

Results.

There were 112 patients (10.7%) that were readmitted or died within 30 days after CABG surgery. The STS augmented clinical model resulted in an AUROC of 0.66 (95%CI: 0.61–0.71). The biomarker panel with the STS augmented clinical model resulted in an AUROC of 0.74 (bootstrapped 95%CI: 0.69–0.79; p-value <0.0001). External validation of the model showed limited improvement with the addition of a biomarker panel with an AUROC of 0.51 (95%CI: 0.45–0.56)

Conclusions.

Although biomarkers significantly improved prediction of 30-day readmission or mortality in our derivation cohort, the external validation of the biomarker panel was poor. Biomarkers, much like other efforts to improve prediction of readmission, perform poorly suggesting there are many other factors yet to be explored to improve prediction of readmission.

The Affordable Care Act of 2010 identified hospital 30-day readmissions and mortality rates as key metrics for the development of incentives for better hospital care and reduced costs [1]. On average, each readmission costs hospitals $9,923, with total unplanned readmissions accounting for 17% of all Medicare reimbursement to hospitals and an annual estimated cost of $17 billion dollars in 2015 [2]. Hospitals are developing protocols to improve preparation and confirm readiness for discharge and establish routine follow-up protocols for continuity of care with primary care physicians [3]. Prediction models are helpful to identify patients at highest risk of readmission to aid clinical care teams in discharge and follow-up care.

Common readmission risk models using clinical factors from registries and claims data are not well calibrated or discriminating [4-10]. The addition of biomarkers that serve as measures of biological processes linked to causes of readmission and death are hypothesized to provide considerable value through personalization of care (i.e. targeted allocation of resources for risk mitigation) [11,12]. Novel biomarkers which include ST2, galectin-3 (gal-3), cystatin C (cys-C) and conventional biomarkers such as B-type natriuretic peptide (BNP), interleukin-6 (IL-6) may help in predicting 30-day readmission and death [13]. ST2 levels correlate with the process of ventricular remodeling, and clinically with heart failure (HF), infection and inflammation [14-18]. ST2 was reported to be strongly correlated with death, HF, and major cardiovascular events (MACE) [14]. Gal-3, which is secreted by activated macrophages can lead to progressive inflammation, cardiac remodeling and fibrosis, and HF [19]. These proposed novel and clinically available biomarkers are likely strong predictors of readmission and mortality and have the ability to improve risk models [14,20]. Specifically we hypothesized that the addition of biomarkers can improve current risk models and better identify patients at high risk of readmission or death within 30-days after discharge from cardiac surgery.

PATIENTS AND METHODS

Derivation Cohort: Northern New England Biomarker Study

The Northern New England (NNE) Biomarker Study is an initiative designed to assess the role of biomarkers in cardiac surgery [12,20-23]. This study uses a harmonized dataset from eight medical centers in Vermont, New Hampshire and Maine in the NNE Cardiovascular Study Group. Patients undergoing coronary artery bypass grafting (CABG) surgery and/or valve surgery at any of participating eight hospitals were prospectively enrolled into the NNE Biomarker Study from 2004 to 2007 (n = 1,690). Patients were excluded from the study if they did not have pre-operative (n=−113) biomarkers and post-operative biomarkers (n=−347) (Figure 1). The Committee for the Protection of Human Subjects at Dartmouth College approved this study.

Figure 1.

Northern New England (NNE) Cohort Study Flow Diagram. Flowchart depicting the retrospective cohort study and final selection of study participants.

External Validation Cohort

The Translation Research Investigating Biomarker Endpoints for Acute Kidney Injury (TRIBE-AKI) study is a prospective cohort of 1194 adults with high risk of AKI who underwent cardiac surgery (CABG or valve surgery). Participants were prospectively enrolled at six academic centers in North America from 2007 through 2010 [24]. Full study details were previously described [24-26] and included in the Supplement.

Primary Outcome: 30-day Readmission or Mortality

Our analysis focused on the combined endpoint of readmission or mortality 30-days after discharge from the hospital following surgery. To achieve complete ascertainment of readmissions up to 30-days following discharge from cardiac surgery, the study cohort was linked to Medicare in-patient claims and state all-payer claims. Mortality was ascertained by linking to the National Death Index as described previously [11].

Biomarkers

We evaluated the associations of six different biomarkers with both 30-day readmission and 30-day combined readmission-or-death: ST2, Galectin-3, NTproBNP, Cystatin C, IL-6, and IL-10. All biomarkers were analyzed using MSD multiplex assays. See Supplemental Appendix for additional details.

Clinical Model Construction

The Society of Thoracic Surgeons (STS) pre-operative 30-day readmission model accounts for 21 different pre-operative factors (Supplemental Table 6) [27]. We initially evaluated a pre-operative biomarker panel against the STS 30-day readmission model using the pre-operative cohort (N=1393; Figure 1). We subsequently constructed an intra-operative readmission risk model based on the STS 30-day readmission model from Shahian et al (2014) [27], with some notable exceptions. We augmented the STS model using the NNE pre-and post-operative cohort (N=1046; Figure 1 and Supplemental Table 1); we refer to this new model as the STS augmented clinical model herein. Our model included 32 covariates (Supplemental Table 1).

Biomarker Panels

Our primary panel for this analysis included tertile forms of the pre-operative, post-operative, pre-to-post-operative difference, and pre-to-post-operative log difference biomarker measurements. As a secondary analysis, we evaluated a biomarker panel derived from fractional polynomials [28-30]. Last, we evaluated a pre-operative biomarker panel consisting of all tertiles for all pre-operative biomarkers compared to the STS 30-day readmission model [27].

Statistical Analysis

Patient and procedural characteristics were compared using Pearson’s chi-square tests; continuous variables were compared with two-sample t-test or Wilcoxon rank-sum tests. Univariate and multivariable logistic regression was used to examine the relationship between biomarker tertiles and readmission or mortality. We conducted simple logistic regressions in continuous form, natural log transformation, quantile categorical variables and clinically significant cut-points to further assess the associations between the biomarkers and endpoints. We conducted additional logistic regressions incorporating in-hospital deaths with the composite outcome of 30-day readmission or mortality. We also evaluated first-order pairwise interactions (Supplemental Tables 7-9). We employed multiple imputation as a primary approach in dealing with missing data. We repeated our analyses using the replacement approach used by Shahian and colleagues[27] for the pre-operative only model, where missing values of indicator variables were replaced with zero, missing categorical variables were replaced with the most common category and missing values for continuous variables were replaced with conditional medians. In order to assess the predictive ability of the biomarker panels, we used area-under-the-receiver-operating-curve (AUROC) statistics, Hosmer-Lemeshow goodness-of-fit tests [31,32]. We calculated cross-validated c-statistics to prevent model overfitting by using Harrell’s optimism correction method [33]. All analyses were performed using Stata 13.1 (College Station, TX) and SAS v9.4. Coefficients from the final derivation models were used by the external cohort for validation. Lastly, we calculated the relative contribution of biomarkers. A χ-pie chart was developed for the full model with biomarkers; full details are included in the Supplement.

RESULTS

Out of the 1,046 patients included in the study, 10.7% (n=112) were readmitted and/or dead within 30-days from discharge. Patients who experienced 30-day readmission or mortality were significantly older and were more likely to have a history of atrial fibrillation, myocardial infarction, congestive HF, renal failure, chronic obstructive pulmonary disease (COPD), any vascular disease, and cerebrovascular disease, compared to patients without an event (Table 1). Patient characteristics for TRIBE-AKI external validation cohort are reported in Table 1.

Table 1.

Northern New England and Translational Research Investigating Biomarker Endpoints Patient Characteristics

	Readmission or Death Within 30 days
	Northern New England			TRIBE
Characteristics	Yes (n = 112)	No (n = 934)	p Value	Yes (n = 164)	No (n = 1,030)	p Value
Age, years	67.24 ± 9.54	65.23 ± 9.88	0.042	73.65 ± 8.93	73.28 ± 8.35	0.596
Female	30 (26.79)	198 (21.20)	0.176	46 (28.04)	310 (30.09)	0.594
White race				153 (93.3)	1,002 (97.3)	0.008
Body mass index, kg/m2	29.41 ± 5.88	29.67 ± 5.37	0.638
Body surface area, m2	2.01 ± 0.25	2.05 ± 0.24	0.177
Atrial fibrillation	a	48 (5.14)	0.469
Congestive heart failure	16 (14.29)	90 (9.64)	0.123	41 (25)	203 (19.7)	0.119
Cerebrovascular disease	a	a
Last preoperative serum creatinine	1.14 ± 0.51	1.12 ± 0.61	0.821	1.06 ± 0.28	1.07 ± 0.34	0.649
Any diabetes mellitus	40 (35.71)	359 (38.44)	0.575	74 (45.1)	370 (35.9)	0.024
Insulin-requiring diabetes	a	a
Ejection fraction < 40%	17 (16.19)	100 (11.06)	0.120	19 (11.6)	95 (9.2)	0.339
Hypertension	89 (79.46)	758 (81.24)	0.650	128 (78)	828 (80.4)	0.486
Preoperative intraaortic balloon pump	a	33 (3.53)	0.639
Prior myocardial infarction				39 (23.8)	264 (25.6)	0.745
None	56 (50)	513 (54.93)	0.560
<24 hours preoperative	a	14 (1.5)
>24 hours and <7 days preoperative	24 (21.43)	193 (20.66)
>7 days and <365 days preoperative	16 (14.29)	89 (9.53)
>365 days preoperative	a	125 (13.38)
Peripheral vascular disease	38 (33.93)	244 (26.12)	0.079
Chronic obstructive pulmonary disease	15 (13.39)	109 (11.67)	0.594
Left main ≥ 50% stenosis	35 (31.25)	293 (31.37)	0.979	19 (11.6)	95 (9.2)	0.339
Prior CABG	a	20 (2.17)	0.382	a	a
CABG or valve				109 (66.5)	798 (77.5)	0.008
Prior percutaneous coronary intervention	27 (24.11)	174 (18.63)	0.164
Priority
Emergent/emergent salvage	a	16 (1.71)	0.442	a	a	0.326
Urgent	73 (65.18)	650 (69.59)		29 (17.7)	142 (13.8)
Elective	38 (33.93)	268 (28.69)		135 (82.3)	888 (86.2)
On dialysis versus no dialysis	a	a		<6 (<3.7)	12 (1.3)
Off-pump CABG	a	41 (4.4)	0.640	19 (11.6)	101 (9.8)	0.685
Cross-clamp time, minutes	63.68 ± 19.67	66.8 ± 23	0.192	82.53 ± 44.61	75.68 ± 41.09	0.057

^aIndicates suppressed or countersuppressed value owing to Centers for Medicare and Medicaid Services policy and Institute for Clinical Evaluative Sciences small cell policy.

Values are mean ± SD or n (%). Blank cells indicate variable was not collected in either Northern New England or Translational Research Investigating Biomarker Endpoints (TRIBE) acute kidney injury cohort.

CABG = coronary artery bypass graft surgery.

Biomarkers

We evaluated six biomarkers in terms of their ability to improve the risk prediction for 30-day readmission or mortality following CABG surgery. These biomarkers are summarized in Supplemental Table 2 and were included in a biomarker panel to be tested against the STS augmented clinical model shown in Table 2. Summary statistics for biomarkers are available in Supplemental Table 2.

Table 2.

Biomarker Panel with STS Augmented Clinical Model: Biomarker Tertiles and 30-day Combined Readmission or Death

Biomarker	Preop	Postop	Delta1	Delta Log2
	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)
Cystatin C3
1	Reference
2	0.64 (0.33 – 1.22)	1.63 (0.81 – 3.27)	0.34 (0.05 – 2.37)	4.48 (0.64 – 31.14)
3	1.19 (0.52 – 2.72)	1.44 (0.52 – 3.96)	0.11 (0.01 – 0.90)	9.79 (1.22 – 78.31)
IL-103
1	Reference
2	1.15 (0.63 – 2.08)	1.07 (0.50 – 2.29)	0.54 (0.21 – 1.40)	1.56 (0.65 – 3.71)
3	1.58 (0.73 – 3.39)	0.83 (0.28 – 2.44)	0.94 (0.24 – 3.59)	1.94 (0.58 – 6.52)
IL-63
1	Reference
2	0.96 (0.51 – 1.79)	1.03 (0.24 – 4.41)	1.17 (0.27 – 5.11)	0.68 (0.34 – 1.33)
3	0.70 (0.30 – 1.62)	0.46 (0.07 – 2.82)	2.81 (0.44 – 18.01)	0.61 (0.24 – 1.58)
Galectin 33
1	Reference
2	0.63 (0.30 – 1.33)	1.07 (0.51 – 2.24)	0.67 (0.23 – 1.94)	2.04 (0.73 – 5.72)
3	1.14 (0.47 – 2.77)	0.93 (0.35 – 2.47)	0.57 (0.17 – 1.88)	1.68 (0.49 – 5.77)
NT Pro BNP3
1	Reference
2	1.88 (0.95 – 3.72)	0.85 (0.38 – 1.90)	1.05 (0.49 – 2.24)	1.33 (0.70 – 2.54)
3	1.09 (0.38 – 3.17)	0.81 (0.25 – 2.59)	1.10 (0.39 – 3.12)	1.34 (0.55 – 3.29)
sST23
1	Reference
2	1.17 (0.65 – 2.13)	1.47 (0.36 – 5.98)	1.17 (0.26 – 5.21)	1.22 (0.57 – 2.62)
3	1.11 (0.53 – 2.33)	2.52 (0.38 – 16.77)	0.47 (0.06 – 3.86)	0.95 (0.32 – 2.85)

¹Difference between pre- and post-operative biomarker measurements

²Log transformation of difference between pre- and post-operative biomarker measurements

³Biomarkers adjusted for covariates in STS Augmented Clinical Model (Supplemental Table 1)

Biomarker Panel Performance

The STS 30-day readmission model had an AUROC of 0.66 (95%CI: 0.62-0.70) and the STS augmented clinical model had an AUROC of 0.66 (95%CI: 0.61-0.71). The biomarker panel enhancement of the STS augmented clinical model (Supplemental Table 1) for 30-day readmission or mortality significantly increased the AUROC to 0.74 (95%CI: 0.69-0.79; p value <0.001; Table 3; Figure 2). When incorporating in-hospital deaths, the AUROC increased to 0.75 (95%CI 0.70—0.79; p-value <0.001). The fractional polynomial biomarker panel enhancement of the STS augmented clinical model also significantly increased the predictive ability with AUROC: 0.72 (95%CI: 0.67–0.77; p value: <0.001). Therefore, two biomarker panels using different panel development strategies demonstrated that novel biomarker panels significantly improve the predictive ability of models using only established clinical risk factors. The results were comparable when we repeated our models using both multiple imputation and the replacement approach [27]. We repeated our modeling focusing only on 30-day readmission and restricting to pre-operative prediction alone (Table 3) and found similar statistically significant improvements in predicting readmission.

Table 3.

Clinical Model and Biomarker Panel Performance

	Derivation Cohort: NNE										Validation Cohort: TRIBE
	30-day Readmission and/or Mortality					30-day Readmission					30-day Readmission or Mortality			30-day Readmission
	AIC	AUCROC	95% CI (BS1)	P- value2	Harrell’s optimisim corrected AUC	AIC	AUCROC	95% CI (BS1)	P- value2	Harrell’s optimisim corrected AUC	AUCROC	95% CI (BS1)	P- value2	AUCROC	95% CI (BS1)	P- value2
Pre-operative
STS 30-day Readmission Model	944.54	0.66	0.62—0.70	REF	0.58	915.13	0.66	0.62—0.71	REF	0.58	0.46	(0.41—0.51)	REF	0.47	(0.41—0.52)	REF
Pre-operative Biomarker Panel	955.83	0.68	0.64—0.72	0.108	0.66	928.84	0.68	0.64—0.72	0.02	0.64	0.44	(0.38—0.49)	0.04	0.45	(0.40—0.50)	0.09
Pre-operative Biomarker Fractional Polynomial Panel	944.32	0.68	0.64—0.73	0.065	0.65	915.58	0.69	0.65—0.73	0.05	0.66	0.50	(0.48–0.52)	0.50	0.50	(0.45–0.56)	0.47
Post-operative
STS Augmented Clinical Model	686.48	0.66	0.61—0.71	REF	0.56	669.06	0.66	0.61—0.72	REF	0.56	0.48	(0.42—0.53)	REF	0.47	(0.42—0.53)	REF
All Biomarker Panel	732.32	0.74	0.69—0.79	<0.001	0.63	719.25	0.74	0.68—0.79	0.001	0.63	0.48	(0.42—0.54)	0.58	0.48	(0.42—0.54)	0.89
All Biomarker Fractional Polynomial Panel	687.25	0.72	0.67—0.77	0.010	0.66	669.55	0.73	0.68—0.78	0.002	0.67	0.50	(0.44–0.56)	0.76	0.50	(0.44–0.56)	0.72

¹BS = Bootstrap method

²ROCCOMP = Comparison of ROC curve.

Figure 2.

AUROC of All Biomarker Panel and STS Augmented Clinical Model. ROC curves for STS augmented Clinical Model (Blue) with and without pre- and post-operative biomarker panel (Red). The c-statistic for each model with 95% confidence intervals is listed (Table 3, p<0.0001).

Relative Contribution of Biomarkers

We quantified the relative contribution of each biomarker to the overall discrimination of 30-day readmission or mortality using a χ-pie chart (Figure 3). We observed that the STS augmented clinical model accounted for 32% of the predictability of the model, while the biomarkers contributed 68%. Figure 3 demonstrates the added predictive value that biomarkers contribute towards predicting 30-day readmission or mortality in cardiac surgery patients prior to discharge.

Figure 3.

Relative Contribution of Biomarker Predictors for 30-day Combined Readmission/Mortality. The relative contributions of biomarker predictors for 30-day Readmission/Mortality are plotted. The size represents the percent of contribution to the prediction model.

External Validation

We externally validated our results using coefficients from the STS augmented clinical model (Supplemental Table 1; Supplemental Table 4). The AUROC value for the STS augmented clinical was 0.48 (95%CI: 0.42–0.53; Table 3) and the AUROC value for the biomarker panel was 0.51 (95%CI: 0.45–0.56; Table 3). For isolated CABG AUROC was 0.50 (95%CI: 0.42–0.58) and 0.52 (95%CI:0.44–0.60). We plotted quintiles of predicted values from the NNE and TRIBE-AKI cohorts (Figure 4).

Figure 4.

Quintiles of Predicted Risk. Derivation (Blue) and external validation (Red) cohorts plotting the incidence of 30-day Readmission/Mortality by quintiles of predicted risk. *NNE Quintiles 1 and 2 are n<11 and required suppression.

COMMENT

We report that biomarker panels have poor performance in their ability to improve risk prediction for 30-day readmission or mortality following discharge after CABG surgery. In our derivation cohort, the addition of novel and clinically available biomarkers resulted in significant improvement in the performance of predictive models (AUROC: 0.74; 95%CI: 0.69–0.79; p <0.001). However, external validation of the STS augmented clinical model with the biomarker panel in an international cardiac surgery cohort showed poor performance.

There is limited evidence evaluating biomarker panels for readmission or mortality following cardiac surgery. Bayes-Genis and colleagues used a ST2 and NTproBNP panel to improve prediction of long-term mortality in heart failure patients. There was moderate improvement in AUROC, improving from 0.76 (95%CI: 0.73–0.79) when using only clinical variables to 0.79 (95%CI: 0.76–0.81, p <0.001) upon adding biomarkers [16]. In hospitalized ST-elevation myocardial infarction patients, Dhillon and colleagues demonstrated that the combination of ST2 and NTproBNP can significantly improve the discrimination of 30-day mortality with an AUROC of 0.90 (p=0.01) [34].

Brown and colleagues evaluated the use of a pre-operative biomarker panel including Cardiac Troponin T, NTproBNP, high sensitivity C-Reactive Protein, and blood glucose to improve the prediction of in-hospital mortality, demonstrating a moderate improvement in discrimination of mortality from an AUROC of 0.83 (95%CI: 0.74–0.92) to 0.87 (95%CI: 0.80–0.94) with biomarkers (p=0.09) [20]. In this analysis, we significantly build upon the literature by evaluating the clinical utility of a novel biomarker panel to improve prediction of 30-day readmission or mortality with external validation.

Strengths and Limitations

There are notable strengths of our research. STS functioned as a sub-contract on the grant with senior STS members providing oversight in the development and validation of our model development. We include both a multi-center derivation cardiac surgery cohort (NNE) and a multi-center international external validation cohort (TRIBE-AKI). We repeated our analyses to minimize over-fitting of the models by performing ridge regressions (Supplement Table 5). We also used fractional polynomials as an alternative approach to model development and to reduce over-fitting; we observed comparable significant improvements in the models with biomarkers (Table 3).

There are several limitations to consider. The composite endpoint of readmission and mortality was dominated by readmission. While readmission is often an intermediate variable on the causal pathway to 30-day mortality, our result should be interpreted with the understanding that our models aid in predicting primarily readmission that did not result in 30-day mortality. We are using established cardiac surgery biorepositories from historical patients (from 2004 – 2009). The generalizability of our models may be limited by our patients that were readmitted in an era prior to the Affordable Care Act penalties. Second, both cohorts either collected some but not all STS surgical data fields; the NNE harmonized their registry fields to the STS definitions and TRIBE-AKI only prospectively collected 50% of the STS model variables. External validation with the TRIBE-AKI cohort was disappointing; we believe this to be related to several contextual factors. First, TRIBE-AKI has a large population of patients from Ontario, which operating under a different payment model might handle readmission differently from US hospitals. Second, the NNE cohort was developed for isolated CABG while TRIBE-AKI is an older, higher risk cohort designed for AKI investigations with fewer isolated CABG procedures and includes a mixture of procedures including CABG and CABG/valve (Table 1). Approximately 50% of all CABG patients were eligible. Third, the NNE cohort comparably validates the STS readmission model performance with an AUROC of 0.66 while the TRIBE cohort reports an AUROC of 0.46 (Table 3). While there is poor performance in the validation cohort, the derivation cohort is comparable to the STS national readmission model performance. The poor validation performance might also possibly be due to the complexity of the TRIBE-AKI patient. An improvement to external validation would be to identify a biorepository with a large sample size of isolated CABG patients in addition to exploring other post-discharge risk factors including care transitions, proximity to hospital, and medication compliance.

Future research

Our novel research demonstrates biomarkers should be explored to improve discrimination of risks of readmission or mortality. There are other novel biomarkers to evaluate for readmission or mortality. To do so we should bring together collaborations of biorepositories in cardiac surgery and other conditions to evaluate new biomarker candidates for predicting readmission and mortality with external validation to determine if other biomarkers can improve prediction. We recommend surgical teams consider including biomarkers available at their sites along with clinical variables in a local model to identify the risk of readmission or death for each patient prior to discharge. Improved readmission risk models can provide case managers, therapists, and primary care providers a transitional care plan that is tailored to each individual patient to mitigate major adverse events and readmission.

Conclusions

In summary, the addition of biomarkers improved prediction of 30-day readmission or mortality in our derivation cohort, however, the external validation of the biomarker panel was poor. Biomarkers perform poorly when externally validated to predict readmission much like other efforts to improve prediction of readmission. Our findings suggest there are many other potential biomarkers and factors yet to be explored to improve prediction of readmission.