Predicting open wound mortality in the ICU using machine learning

Abstract

Background:

Open wounds have a significant impact on the health of patients causing pain, loss of function, and death. Labeled as a comorbid condition, open wounds represent a “silent epidemic” that affect a large portion of the US population. Due to their burden of care, open wound patients face an increased risk of ICU stay and mortality. There is a dearth of studies that investigate mortality among wound patients in the ICU. We sought to develop a model that predicts the risk of mortality among wound patients in the ICU.

Methods:

Random forest and binomial logistic regression models were developed to predict the risk of mortality among open wound patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database. MIMIC-III includes de-identified data for patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012. Six variables were used to develop the model (wound location, gender, age, admission type, minimum platelet count and hyperphosphatemia). The Charlson Comorbidity Index (CCI) and Elixhauser Comorbidity Index were used to assess model strength.

Results:

A total of 3,937 patients were included with a mean age of 76.57. Of those, 3,372 (85%) survived and 565 (15%) died during their ICU stay. The random forest model achieved an area under the curve (AUC) of 0.924. The CCI and Elixhauser models resulted in AUC of 0.528 and 0.565, respectively.

Conclusions:

Machine learning models may allow clinicians to provide better care and management to open wound patients in the ICU.

Introduction

Open wounds in the skin have a significant impact on patients’ health (1). Labeled as a comorbid condition, open wounds represent a “silent epidemic” that affect a large portion of the United States population. Although most acute wounds heal without issue, unhealed wounds cost about $25 billion a year in management (2). Chronic wounds are very rarely seen in healthy patients; their care represents a cross section of many medical disciplines, diabetes, trauma, hypertension, vascular insufficiency, and rheumatologic diseases. While diabetes and obesity are well-known mortality risk factors, these diagnoses and similar ones tend to minimize the true role chronic wounds play in a patient’s morbidity (3). A patient in the ICU with a chronic wound of diabetic etiology and another postsurgical wound from a pressure ulcer are not considered a sum, rather as individual complications of diabetes and postop immobility. Many of these problems result in increased ICU stays and mortality (4).

Clinical and risk prediction models can help clinicians make better treatment decisions (5). In the literature, two well-established algorithms predict the mortality risk in the general population: the Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (6). Both models use ICD-9-CM diagnosis codes to stratify patients. The two algorithms rely on symptom severity and have been validated in multiple studies. They have also been applied to ICU patients yet they have not been validated to patients with chronic wounds. Machine learning algorithms have been applied to the ICU settings but never specifically to the chronic wound patient population in the ICU (7,8). Established ICU models such as the Acute Physiology and Chronic Health Evaluation (APACHE IV) and Sequential Organ Failure Assessment (SOFA) require many variables and about 37.3 minutes on average to enter the necessary input data to get a result (9). In the ICU setting, algorithms with the least number of variables are of high clinical utility (10).

In this context, patients with open wounds may be facing an increased risk of ICU stay and mortality. There is a dearth of studies that investigate mortality among patients with open wounds in the ICU. In this study, we sought to develop a model that predicts the risk of mortality among those patients in the ICU. We present the following article in accordance with the Materials Design Analysis Reporting (MDAR) reporting checklist (available at http://dx.doi.org/10.21037/jeccm-20-154).

Methods

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Patient data from the Medical Information Mart for Intensive Care III (MIMIC-III) database were used for this study (11). MIMIC-III is a publicly available database that includes deidentified electronic health record (EHR) data for patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012 and signed informed consent. This study did not require institutional review board approval as it used publicly available data. Patients with open wound diagnoses were selected using the ICD-9-CM codes 870.0 to 906.0.

Statistical analysis

The Julia programming language (v0.5) was used for data analyses at our institution’s Unified Research data Sharing and Analysis (URSA) Stronghold. Data processing was facilitated with MySQL and Julia packages including DataFrames.jl (to create data tables) and PredictMD.jl (for prediction modeling).

For each patient selected, CCI and Elixhauser scores were calculated. CCI scores range from 0 to 34 and incorporated comorbidities in 17 categories with more points assigned to severe comorbidities as reported in its guidelines [2 (hemiplegia, moderate or severe renal disease, diabetes with end stage-organ damage, tumor without metastasis, leukemia, lymphoma); 3 (moderate or severe liver disease); 6 (metastatic solid tumor, AIDS)]. Elixhauser scores range from 0 to 36 and were scored by assigning one point per comorbidity for a total of 36 categories (12).

Mortality during ICU stay was our primary outcome of interest. A new variable was created with value of either “0” (indicating that the patient did not die during ICU stay) or “1” (indicating that the patient died during the ICU stay). Model creation and fitting was conducted using the PredictMD.jl package. The random forest model developed to predict the mortality used six variables: wound location, gender, age, admission type, minimum platelet count, and hyperphosphatemia. The predictors were selected after careful review of the literature (13–16). Wound location was defined by ICD-9-CM codes designating the trunk, the extremities or the head and neck. Admission type included “elective, emergency, or urgent.” Minimum platelet count and hyperphosphatemia (>4.5 mg/dL) were calculated for each patient using lab data. Patients with a mortality occurring within 24 hours of admission were excluded. The data were randomly partitioned into a training set (70%) and testing set (30%). The training set was used to develop and train a random forest model for predicting mortality, with the predictive features as inputs and the mortality outcome as output. The testing set was then used to compute the sensitivity and specificity of the random forest model at different operating thresholds. A receiver operating characteristic (ROC) curve was generated with a resulting area under the curve (AUC) value. The area under the ROC curve was calculated using trapezoidal integration. These analyses were performed in the Julia programming language using the PredictMD toolkit.

Results

A total of 3,937 patients out of 61,532 ICU admissions were included in the study; 38.4% were female and 61.6% were male. Admission type was urgent (1.9%), elective (6.0%) or emergent (92.1%). The mean age was 76.57. Of those, 3,372 (85%) survived and 565 (15%) died during their ICU stay. Different types of insurance were represented in the study population. Most were on Medicare (56.7%), or private insurance (30.8%). Table 1 summarizes the patient demographics and statistics.

Table 1.

Summary statistics on the cohort of patients who died and those who survived

Category	Subgroup	Alive	Dead	Total
Average age		73.78	86.88
Gender	Female	1,298 (85%)	228 (15%)	1,526
	Male	2,074 (86%)	337 (14%)	2,411
Insurance	Government	102 (94%)	7 (6%)	109
	Medicaid	292 (90%)	32 (10%)	324
	Private	1,098 (90%)	116 (10%)	1,214
	Self-pay	77 (93%)	6 (7%)	83
	Medicare	1,803 (82%)	404 (18%)	2,207
Admission type	Elective	214 (91%)	22 (9%)	236
	Emergency	3,098 (85%)	528 (15%)	3,626
	Urgent	59 (79%)	16 (21%)	75

The data were randomly split into a training set of 2,756 patients and a testing set of 1,181 patients. The training set was used to develop and train a random forest machine learning model for predicting mortality. The testing set was used to evaluate the performance of the random forest model and generate a ROC curve (Figure 1). The model achieved an AUC of 0.924. In comparison, the CCI and Elixhauser models resulted in AUC scores of 0.528 and 0.565 (Figure 1). The random forest model was found to have a sensitivity of 98% and a specificity of 85% with a recall of 94%, 79% precision and 0.73 Cohen’s Kappa statistic. Figure 2 shows the classifier score histogram for the model. P<0.05 was chosen as the level of significance. The two variables with the highest relative contribution in predicting mortality were minimum platelet count and hyperphosphatemia (Table 2).

Figure 1.

Receiver operating curve of (A) the random forest model; (B) the Charlson Comorbidity Index (CCI) model; (C) the Elixhauser model.

Figure 2.

Classifier score histogram.

Table 2.

Random forest modeling results with coefficients and statistical significance

Variable	Coefficient	P value
Wound location	0.3436	0.0001
Gender	0.1184	0.0097
Age	0.2948	0.0034
Admission type	0.4583	0.0001
Minimum platelet count	0.8922	0.0001
Hyperphosphatemia	0.9321	0.0001

Discussion

Managing open wound patients in the ICU presents a unique challenge since their condition may have an impact on survival. Our data suggest that using only six readily available variables, clinicians may be able to predict with great accuracy which patients are at higher risk of dying in the ICU.

The CCI and the Elixhauser models compute risk scores using ICD-9-CM codes. CCI uses severity of symptoms in its calculation while Elixhauser uses comorbidity categories without severity. Both of these models have been applied to the ICU stetting but never specifically to the open wound patient population specifically. Our predictive model outperformed both CCI and Elixhauser with fewer variables, indicating its potentially high practical clinical utility.

Patients with ICD-9-CM codes 870.0 to 906.0 included in this study may not have had their open-wound(s) as the reason for their ICU admission. However, understanding which variables can contribute to mortality in patients with open wounds such as pressure ulcers in the ICU may be used to inform the development of targeted public health measures and potentially enhance discussions around wound morbidity (17). These wounds can lead to multiple complications including gangrene, hemorrhage and infection requiring lower-extremity amputations. The disabilities caused by those wounds worsen their healing resulting in a vicious cycle (18).

Even though our random forest model was developed using ICU patient data, further research could investigate its use in different settings. At the threshold of 0.13, our random forest model has a sensitivity of 98% and a specificity of 85%. With a high specificity and specificity, our model could be useful in prioritizing admission to the ICU by lowering the chances of false positives.

One potential limitation to this study is possible class imbalance affecting the model’s accuracy. At present, the random forest model presented here is unvalidated in a different hospital ICU. Future studies could validate the model with prospective cohorts, increasing the number of patients to minimize class imbalance. In this study, the model was compared to the CCI and the Elixhauser indexes since they are based on the ICD-9-CM diagnosis codes for each patient. The Elixhauser score estimates the comorbidity burden but not the acute fluctuation in the patient’s illness. Further comparison could test the random forest model against other commonly used algorithms such as APACHE-II and SOFA to estimate the model efficiency and time to completion when compared to the gold standard. The random forest model should also be assessed in racially and geographically diverse US populations to assess if the predictive model still outperforms the CCI and Elixhauser models. The goal of predictive models in the healthcare setting is to improve patient care. Machine learning should be used as adjunct and not as a substitute to clinical decision making.

Conclusions

Comorbidities continue be labeled as the leading factors of mortality in patients with open wound diagnoses. The random forest model developed in this study predicts the risk of mortality among those patients in the ICU. Our model suggests that using six readily available variables only, clinicians may be able to use predictive modeling to identify patients who are high risk for mortality in the ICU.