Development and validation of a prognostic nomogram for myocardial infarction patients in intensive care units: a retrospective cohort study

Abstract

Objectives

We aimed to develop and validate a prognostic nomogram and evaluate the discrimination of the nomogram model in order to improve the prediction of 30-day survival of critically ill myocardial infarction (MI) patients.

Design

A retrospective cohort study.

Setting

Data were collected from the Medical Information Mart for Intensive Care (MIMIC)-III database, consisting of critically ill participants between 2001 and 2012 in the USA.

Participants

A total of 2031 adult critically ill patients with MI were enrolled from the MIMIC-III database.

Primary and secondary outcome

Thirty-day survival.

Results

Independent prognostic factors, including age, heart rate, white blood cell count, blood urea nitrogen and bicarbonate, were identified by Cox regression model and used in the nomogram. Good agreement between the prediction and observation was indicated by the calibration curve for 30-day survival. The nomogram exhibited reasonably accurate discrimination (area under the receiver operating characteristic curve, 0.765, 95% CI, 0.716 to 0.814) and calibration (C-index, 0.758, 95% CI, 0.712 to 0.804) in the validation cohort. Decision curve analysis demonstrated that the nomogram was clinically beneficial. Additionally, participants could be classified into two risk groups by the nomogram, and the 30-day survival probability was significantly different between them (p<0.001).

Conclusion

This five-factor nomogram can achieve a reasonable degree of accuracy to predict 30-day survival in critically ill MI patients and might be helpful for risk stratification and decision-making for MI patients.

Strengths and limitations of this study.

• This is the first study to develop and validate a prognostic nomogram for myocardial infarction (MI) patients in the intensive care unit.

• The area under the receiver operating characteristic curve, calibration curves, decision curve analysis and survival curves were enrolled to evaluate the performance of this novel nomogram model in both the primary cohort and validation cohort.

• Multiple imputation was used to handle the covariates with less than 20% missing to minimise the bias resulting from missing values.

• ST elevation, oliguria and ventricular arrhythmias were not accessible in this study, and this might lead to reduced effectiveness of this nomogram.

• We could not compare the performance of nomogram model with existing model, such the Thrombolysis in Myocardial Infarction score and the Global Registry of Acute Coronary Events score.

Introduction

Myocardial infarction (MI) is a major health problem that causes high mortality and high economic costs worldwide.¹ A substantial proportion of MI patients are admitted to the intensive care unit (ICU).² However, not all MI patients benefit from ICU care.³ It is necessary to perform risk stratification for MI to help clinicians make more efficient decisions, which provides benefits for more MI patients.

Nomograms are popular prognostic tools with the ability to predict clinical events by integrating potential risk factors.⁴ Nomograms have been widely used for tumour prognosis, supporting the movement towards personalised oncology medicine.⁵ Recently, a nomogram was effectively used to predict both short-term and long-term survival for asymptomatic adults undergoing screening for cardiac risk factors.⁶ Thus, we hypothesised that a nomogram may also be feasible for the risk stratification of critically ill MI patients.

This study aimed to identify prognostic factors for the 30-day mortality of critically ill MI patients and establish a prognostic nomogram based on a multivariable Cox regression model in a primary cohort. Furthermore, the performance and clinical benefits of the nomogram were assessed in a validation cohort to validate the accuracy and utility of the prognostic nomogram model. The nomogram could be easily applied in clinical practice to identify high-risk patients and guide decision-making.

Methods

Data source

The data were retrieved from the Medical Information Mart for Intensive Care (MIMIC-III) data set. The MIMIC-III integrates the comprehensive clinical data of 53 423 stays of adult patients in the ICU between 2001 and 2012. An average of 4597 charted observations and 380 laboratory measurements are available for individual hospital admissions. The overall information is saved as a relational database, consisting of patient demographics, laboratory tests, discharge summaries, electrocardiographs, imaging examinations, diagnostic information such as the International Classification of Diseases−9 code, and in-hospital and out-of-hospital mortality. The use of MIMIC-Ⅲ database was under the approval from the review boards of the Massachusetts Institute of Technology and Beth Israel Deaconess Medical Center.⁷ All the patients in the database have been deidentified for privacy, and the need for informed consent was waived. This study was conducted in accordance with the recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis statement.⁸

Study cohort

Patients admitted to the ICU who were diagnosed with MI were eligible for inclusion. After screening of the MIMIC-III database, a total of 2031 patients with MI were included for analysis. The cohort was randomly divided into the primary cohort and the validation cohort in a ratio of 7:3; the primary cohort was used to establish the nomogram, and the validation cohort was used for validation.

Data extraction

Structured query language was used for data extraction. For patients with multiple ICU admissions, only the data of the patient’s first ICU admission were used in this study. All data regarding baseline characteristics were collected as the first value in the initial 24 hours following admission. The variables for the following analysis included (1) basic demographics, including age, sex, weight, coronary care unit stay and private insurance; (2) vital signs, including heart rate, mean arterial pressure (MAP), temperature and central venous pressure (CVP); and (3) laboratory tests, including tests of white blood cell count (WBC), haemoglobin, platelets, serum creatinine, creatine kinase, type B natriuretic peptide (BNP), blood urea nitrogen (BUN) level, bicarbonate level, pH, partial pressure of oxygen (pO₂), partial pressure of carbon dioxide (pCO₂), chloride, sodium, potassium, troponin and lactate.

In this study, we regarded 30-day survival as the outcome measure, which was also extracted from the MIMIC-III database.

Management of missing data

Variables with missing data are common in the MIMIC-III database. More than 20% of the data regarding CVP, pCO₂, pO₂, pH, BNP, troponin and lactate were missing, and these parameters were not qualified for establishment of the nomogram (online supplemental figure 1). A flag indicating whether these data were obtained is shown in the characteristics table. For variables that had missing data for less than 20% of patients, missing values were filled with predictors using multiple imputation to minimise the bias resulting from missing values.⁹

Statistical analysis

Continuous variables are expressed as the mean±SD or median (IQR), as appropriate. Categorical data are expressed as numbers (percentages). Continuous variables were compared using Student’s t-test or the rank-sum test, as appropriate. Categorical variables were compared by the χ² test.

In this study, the objective was to develop a fast-to-use prognostic model for 30-day mortality in critically ill MI patients. And Cox proportional hazards model was the most frequently used regression model for survival analysis and thus was enrolled in this study. Univariate Cox regression was used to screen for variables that were significantly associated with 30-day survival in the primary cohort. The proportional hazards assumption was checked based on the scaled Schoenfeld residuals using survival package in R tool. Potential prognostic factors that were significant in the univariate Cox regression model were entered into the multivariable Cox proportional hazard model, in which the HR, which was used to approximate risk of event, was also calculated. To avoid too many variables entering into the final model and influencing the practicality of model, a strict cut-off value of 0.05 was chosen. The backward stepwise process based on the Akaike information criterion was used to control the overfitting of the model.

A nomogram based on the results of previous multivariable analyses was constructed. The calibration, discrimination and clinical usefulness of the nomogram were calculated to evaluate its performance.¹⁰ The area under the receiver operating characteristic curve (AUC) and Harrell’s concordance index (C-index) were used to assess the predictive capacity of the prediction model. CIs were obtained by creating 1000 bootstrap samples from the corresponding cohort and replicating the estimation process. The calibration curve was used to analyse the agreement between the nomogram and actual observation. Decision curve analysis was performed to assess the clinical usefulness of the prognostic nomogram by quantifying the standardised net benefits at different threshold probabilities. Survival curves were used to compare the survival probability between the low-risk group and the high-risk group defined by the nomogram.

A two-tailed p value<0.05 was considered statistically significant in our study. SPSS software (V.23.0, IBM, New York, USA) and R software (V.3.6.3, R Foundation for Statistical Computing, Vienna, Austria) were used for statistical analysis.

Patient and public involvement

Patients and/or the public were not directly involved in this study.

Results

Baseline characteristics of the primary cohort and validation cohort

The primary cohort and validation cohort consisted of 1422 and 609 MI patients, respectively. All baseline characteristics of the primary cohort and validation cohort are shown in table 1. There were no significant differences in the baseline characteristics between the primary cohort and validation cohort (all p>0.05).

Table 1.

Comparison of basic demographics, vital signs, laboratory tests and 30-day mortality between the primary cohort and the validation cohort

Variables	Primary cohort (n=1422)	Validation cohort (n=609)	P value
Basic demographics
Age, years	67.6±14.2	68.5±14.1	0.195
Male, n(%)	902 (63.4)	397 (65.2)	0.450
Weight, kg	80.8±19.9	80.7±19.0	0.914
CCU stay, n(%)	931 (65.5)	390 (64.0)	0.535
Private insurance, n(%)	525 (36.92)	200 (32.84)	0.079
Vital signs
Heart rate, bpm	84.8±17.9	84.7±17.1	0.910
MAP, mm Hg	85.5±18.1	85.3±17.4	0.830
Temperature, ℃	36.3±0.9	36.3±0.9	0.619
CVP (tested)	525 (36.9)	228 (37.4)	0.825
Laboratory tests
WBC, ×109/L	12.6±5.6	12.7±5.5	0.813
Haemoglobin, g/dL	11.7±2.1	11.6±2.1	0.284
Platelet, ×109/L	227.8±94.0	231.0±95.8	0.474
Creatinine, mg/dL	0.9 (0.8 to 1.3)	1.0 (0.8 to 1.4)	0.295
Creatine kinase, U/L	338.0 (67.0 to 988.6)	378.0 (69. to 992.1)	0.510
BNP (tested)	14 (1.0)	5 (0.8)	0.726
BUN, mg/dL	18.0 (13.8 to 27.0)	19.7 (14.0 to 28.0)	0.165
Bicarbonate, mg/dL	22.9±4.1	23.1±3.9	0.421
pH (tested)	729 (51.3)	326 (53.5)	0.349
pO2 (tested)	719 (50.6)	319 (52.4)	0.453
pCO2 (tested)	719 (50.6)	319 (52.4)	0.453
Chloride, mg/dL	104.8±4.8	104.5±5.2	0.202
Sodium, mg/dL	137.8±3.8	137.7±3.9	0.672
Potassium, mg/dL	4.2±0.6	4.2±0.7	0.609
Troponin (tested)	757 (53.2)	339 (55.7)	0.314
Lactate (tested)	447 (31.4)	212 (34.8)	0.136
30-day mortality, n(%)	208 (14.6)	95 (15.6)	0.573

For each variable, the mean±SD, median (IQR) or frequency (per cent) was reported as appropriate. For variables that had missing data for more than 20% of the patients in the current cohort, flags indicating whether these data were obtained were used as covariates. Continuous variables were compared using either Student’s t-test or the rank-sum test as appropriate. The χ² test was used to compare the differences between categorical variables.

BNP, brain natriuretic peptide; bpm, beats per minute; BUN, blood urea nitrogen; CCU, cardiac care unit; CVP, central venous pressure; MAP, mean arterial pressure; WBC, white blood cell count.

Prognostic factors in the primary cohort

Basic demographics, vital signs, and laboratory tests in the primary cohort were further examined by the univariate Cox regression model for the prediction of 30-day mortality (online supplemental table 1). Variables, including age, male sex, weight, private insurance, heart rate, MAP, haemoglobin WBC, BUN level, bicarbonate level, creatinine level and potassium level, were potential predictors of 30-day mortality in the univariate analysis (p<0.05). All these candidate factors were entered into the multivariable Cox proportional hazard model, and five prognostic factors, namely, age, heart rate, WBC, BUN level and bicarbonate level, were included in the final prediction model (each p<0.05) (table 2).

Table 2.

Univariate and multivariable analyses for the relationship between the candidate risk factors and 30-day mortality in the primary cohort

Variables	Univariate model			Multivariable model
	HR	95% CI	P value	HR	95% CI	P value
Age	1.042	1.031 to 1.053	<0.001	1.033	1.022 to 1.045	<0.001
Male	0.549	0.418 to 0.721	<0.001	0.763	0.574 to 1.015	0.063
Weight	0.989	0.981 to 0.996	0.002
Private insurance	0.353	0.249 to 0.502	<0.001
Heart rate	1.022	1.015 to 1.029	<0.001	1.016	1.008 to 1.023	<0.001
MAP	0.985	0.977 to 0.993	<0.001
Haemoglobin	0.882	0.828 to 0.940	<0.001
WBC	1.064	1.049 to 1.079	<0.001	1.029	1.014 to 1.044	<0.001
BUN	1.025	1.021 to 1.030	<0.001	1.014	1.008 to 1.020	<0.001
Bicarbonate	0.842	0.819 to 0.866	<0.001	0.904	0.875 to 0.933	<0.001
Creatinine	1.257	1.181 to 1.338	<0.001
Potassium	1.394	1.193 to 1.630	<0.001	1.169	0.975 to 1.403	0.092

HRs were estimated by Cox proportional hazards regression. All statistical tests were two-sided. The selection of the final prediction model was performed with a backward stepwise selection process.

BUN, blood urea nitrogen; MAP, mean arterial pressure; WBC, white blood cell count.

A prognostic nomogram for 30-day survival

A prognostic nomogram for 30-day survival was established with the five prognostic factors obtained from the multivariable Cox proportional hazard model (figure 1). The nomogram was generated by assigning a weighted score to each of the independent prognostic parameters. The scales of age, heart rate, WBC, BUN level and bicarbonate level ranged from 10 to 100, 30 to 150, 0 to 75, 0 to 160, and 40 to 5, respectively. The highest total score was 300 points, and the scale of the 30-day survival probability ranged from 0.95 to 0.1. A higher score calculated from the sum of the assigned points for each prognostic factor in the nomogram corresponded to a lower probability of survival in 30 days.

Figure 1.

Nomogram to calculate risk score and predict 30-day survival probability in myocardial infarction patients. Scores were assigned for age, heart rate, WBC, BUN level and bicarbonate level by drawing a line upward from the corresponding values to the ‘score’ line. The sum of all these scores, plotted on the ‘Total score’ line, corresponds to predictions of 30-day survival probability in myocardial infarction patients. bpm, beats per minute; BUN, blood urea nitrogen; WBC, white blood cell count.

For instance, one MI patient with an age of 70 years old (57 points), a heart rate of 110 bpm (33 points), a WBC of 11 ×10⁹/L (10 points), a BUN level of 80 mg/dL (33 points) and a bicarbonate level of 10 mg/dL (85 points) had a total score of 218 points, which corresponded to an approximately 30% 30-day survival probability (online supplemental figure 2).

Another MI patient who had an age of 50 years old (39 points), a heart rate of 70 bpm (17 points), a WBC of 11 ×10⁹/L (10 points), a BUN level of 60 mg/dL (25 points) and a bicarbonate level of 18 mg/dL (63 points) had a total score of 154 points. Then this MI patient was predicted to suffer 90% 30-day survival probability (online supplemental figure 3).

Performance evaluation of the prognostic nomogram

The AUC indicated that the predictive capacity of the prediction model was 0.803 (95% CI, 0.771 to 0.835) in the primary cohort and 0.765 (95% CI, 0.716 to 0.814) in the validation cohort (figure 2A, B). The C-index was 0.787 (95% CI, 0.757 to 0.817) for the primary cohort and 0.758 (95% CI, 0.712 to 0.804) for the validation cohort. The calibration plot demonstrated adequate fit of the nomogram for predicting 30-day survival, which was consistent with the Kaplan-Meier estimate in both the primary cohort and validation cohort (figure 2C, D). The decision curve analysis showed the net benefits obtained from the application of our nomogram with threshold probabilities of 0.648 and 0.499 in the primary cohort and validation cohort, respectively (figure 2E, F). Participants could be classified into low-risk and high-risk groups by the nomogram. Survival curves revealed a significantly lower survival probability in the high-risk group than in the low-risk group in both the primary cohort and validation cohort (p<0.001), which indicated the substantial discriminatory power of the nomogram to distinguish low-risk and high-risk MI patients in the ICU (figure 3).

Figure 2.

Performance evaluation of the nomogram in the primary and validation cohorts. Receiver operating characteristic curve analysis in the primary (A) and validation (B) cohorts. Calibration curve analysis in the primary (C) and validation (D) cohorts. The horizontal axis represents the nomogram-predicted probability of 30-day survival, and the vertical axis represents the actual observed 30-day mortality. decision curve analysis for the primary (E) and validation (F) cohorts, implicating the net benefit with respect to the use of the nomogram. AUC, area under the curve.

Figure 3.

Survival curves classified by high-risk and low-risk group. Survival curves for two groups classified by prognostic total score calculated from the nomogram in the primary (A) and validation (B) cohort. For each survival curve, 95% CI and number at risk for each group were also presented.

Discussion

This study extracted clinical data and survival information of 2031 MI patients from the MIMIC-III database. Five risk factors for 30-day mortality of MI, including age, heart rate, WBC, BUN level and bicarbonate level, were identified by univariate and multivariable Cox regression models and used to establish a prognostic nomogram. To the best of our knowledge, this is the first study to develop and validate a prognostic nomogram for MI patients in the ICU. This novel nomogram showed satisfactory performance in both the primary cohort and validation cohort as assessed by the AUC, calibration curves, decision curve analysis and survival curves. Thus, this nomogram could be efficiently and effectively employed in clinical practice.

MI has been a global health problem with a high incidence and a high mortality, and it has led to economic and health burdens in patients.¹ The Thrombolysis in Myocardial Infarction (TIMI) score and the Global Registry of Acute Coronary Events (GRACE) score are two common tools to predict short-term and long-term outcomes for acute MI patients.^11–13 Both the TIMI risk score and the GRACE score require more than five factors to calculate the probability of mortality. In addition, the Soroka Acute Myocardial Infarction risk score, which was used to predict 1-year and 5-year mortality of acute MI in Israel, consists of 10 risk factors.¹⁴ Comparing with other existing models of which the AUC ranged from 0.66 to 0.90, the nomogram model showed an acceptable AUC of 0.80 (online supplemental table 2). Our nomogram uses five factors that can be collected at first-day admission, can be easily applied, and performs well in predicting short-term mortality of patients with MI. We hope that this short nomogram will be used for the quick identification of high-risk MI patients in the ICU.

Nomograms are of great utility in predicting an individual’s probability of a clinical event using individual variables, and they have become a common prognostic tool in oncology.⁴ A nomogram was developed for the 5-year to 15-year survival of asymptomatic adults undergoing coronary artery calcium scoring.⁶ For the mortality of MI, our study is the first to provide a simple-to-use prognostic nomogram with five factors that are easily accessible on the first-day admission, and this nomogram might improve timely individualised risk stratification and the prevention of fatal outcomes. The satisfactory performance of this model was reflected by its moderate predictive ability, indicated by an AUC greater than 0.75 in both the primary cohort and validation cohort. Additionally, the calibration analysis performed in two cohorts revealed that the predicted 30-day mortality was similar to the actual 30-day mortality. Furthermore, decision curve analysis indicated that the net clinical benefits were positive in MI patients, with a probability of up to 50% in both cohorts. A difference in threshold probability between the primary and validation cohort was observed in our study. This difference may be due to the potential heterogeneity between these two cohorts, such as the level of variables or mortality rate, which had not shown significant differences in statistical analyses. Overall, both two decision curves indicated a net benefit with respect to the use of nomogram model. The survival curves also revealed the good discriminative capacity to identify high-risk and low-risk patients in both the primary cohort and validation cohort.

It should be noted that only five prognostic factors were used in the final nomogram model. Age has been widely recognised as one of the most powerful risk factors in cardiovascular diseases, such as vascular senescence, cardiac remodelling and atrial fibrillation.^15–17 Decreased expression of antioxidative factors and increased expression of oxidative molecular mediators occur in elderly patients, leading to aggravating ischaemic injury.^{15 18} Heart rate is also an important prognostic factor for cardiovascular mortality. A higher resting heart rate was reported to be positively related to a higher risk of MI and all-cause mortality.^{19 20} These results were consistent with our study, in which heart rate was positively associated with mortality of MI.

Among lab tests, WBC has also been shown to be a potential risk factor and to be associated with myocardial perfusion and the severity of coronary artery disease.^{21 22} A recent cohort study of triple-vessel coronary artery disease revealed the independent prognostic value of both total and differential WBCs for predicting long-term mortality.²³ BUN level has also been demonstrated to be independently associated with mortality in patients with MI, even in patients with normal to mildly reduced glomerular filtration rates.^{24 25} Bicarbonate is a central biomarker that reflects acid-base equilibrium and is affected by electrolyte disturbance. In this study, bicarbonate level was negatively related to 30-day mortality, which was consistent with another cohort study of cardiogenic shock patients hospitalised in the ICU.²⁶ In short, these five factors included in the nomogram were all credible prognostic factors for cardiovascular mortality, and these factors could be used in clinical work.

Several limitations should be pointed out. First, a few previously reported independent predictors for major cardiovascular events, including BNP and troponin, were not included to minimise the bias from excessive missing values.^{27 28} Hence, the prognostic value of these factors for MI could not be estimated. Second, ST elevation, oliguria and ventricular arrhythmias, were not accessible in this study, and this might lead to reduced effectiveness of this nomogram. GRACE score and TIMI score could not be obtained, and thus the comparison between nomogram model and these two score models could not be made. Third, the model still required more samples to validate its viability. Although we performed random allocation to establish a validation cohort with 30% of the total sample size for the verification of the superiority of our model, a large external cohort would further enhance the credibility and effectiveness of our model in future studies.

In conclusion, our study developed a prognostic nomogram with five factors, including age, heart rate, WBC, BUN level and bicarbonate level, for the prediction of 30-day survival in critically ill MI patients in the ICU. This nomogram performed well and might be helpful in risk stratification and decision-making for MI patients undergoing clinical treatment.