Predictive Accuracy of Medical Transport Information for In-Hospital Mortality

INTRODUCTION

One in twenty [1] hospital admissions results in an interhospital transfer, amounting to approximately 1.7 million patients being transferred annually. Inter-hospital transfers are growing in number due to increasing consolidation of healthcare systems [2] and specialty services that are concentrated at urban academic medical centers. Transferred patients strain healthcare systems’ resources due to increased lengths of stay, increased resource utilization, and increased mortality. Efforts to ameliorate these burdens are receiving increased attention.

There is strong national consensus about the need to develop useful clinical decision support tools to improve healthcare delivery by leveraging the vast amount of electronic medical record (EMR) data to guide clinicians, patients, and families in making difficult healthcare decisions [3–7]. Currently, limited evidence and no national guidelines [8] exist to support decision makers when considering if they should transfer a patient to another hospital, a decision that can carry significant financial burdens for individuals (i.e. patients) and health systems.

Interhospital transfer patients present several challenges when attempting to include and analyze data to either refine current risk prediction models, or build a clinical decision support system capable of using real-time data. A primary challenge is the lack of access to transport EMRs, precluding incorporating transport data into any hospital-based decision support system. Another problem is the improper application of risk mortality prediction tools such as the Acute Physiology and Chronic Health Evaluation (APACHE) IV tool [9], widely used as the national benchmark for mortality prediction and resource use. The APACHE IV tool is designed for use with the patient’s first 24 hours of physiologic data, skewing results when applied to transported patients who are often >24 hours post-admission from their incident admission at the sending hospital.

Because of these reasons it is not possible to compare transferred versus non-transferred patients using current scoring systems [10–12], or to evaluate the prognostic value of information associated with patients’ transfer encounters. The purpose of this paper is hence to assess if common data elements from medical transport provides incremental prognostic value on post-transfer mortality, both individually and in combination with the APACHE IV risk model estimates.

MATERIALS AND METHODS

Patients transported within the Cleveland Clinic health system, that consists of a quaternary main campus in Cleveland and eight surrounding community hospitals, by the hospital-based Critical Care Transport team from 2010 to 2015 where included in the study. The primary source of data used in this study was The Cleveland Clinic Transport Data Mart (CCTDM) which is an IRB-approved (IRB# 14-1556) EMR-based data repository that includes all patients undergoing medical transport to or from a Cleveland Clinic Health System facility. The primary data domains within the CCTDM include all health system based hospital encounters, all variables included in APACHE prediction modelling, and common transport data elements.

The APACHE IV data and associated risk score is obtained for all transported patients according to standard protocol, although this information is unavailable within the registry for a small number of patients. For this study, we analyzed all transported patients with valid APACHE IV data and associated CCTDM data. Only the most recent transport encounter was analyzed for each patient.

The data were randomly partitioned into an 75% model development cohort and a (remaining) 25% test cohort prior to analysis, in order to minimize the likelihood of over-fitting the models to a specific training dataset [13].

Generally, we evaluated the accuracy of probability estimates for in-hospital mortality through two lenses: discrimination and calibration. Discrimination was characterized using concordance indices (or C-statistics). C-statistics range from 0.5 to 1.0, where a value of 0.5 indicates no discriminative ability for a risk score (i.e., similar to flipping a coin) and a value of 1.0 indicates perfect separation of events from non-events according to the risk score. Together with a point estimate for the C-statistics, we estimated bootstrap 95% confidence intervals. Calibration was assessed graphically using the method of Dalton [14].

Three models were evaluated on calibration and discrimination: 1) the original APACHE IV predicted probability of mortality; 2) the predicted probability of mortality arising from a logistic regression model that incorporated (only) predictors from the CCTDM (see below); and 3) the predicted probability of mortality arising from a logistic regression model that incorporated the APACHE IV score in conjunction with the predictors from the CCTDM.

RESULTS

The transport registry included a total of 12,359 total records, corresponding to 11,218 unique patients. Taking the most recent admission per patient, 4,016 were then removed due to unavailable transport data – most often related to no patient transfer. An additional 170 patients with partial missing data were removed, leaving a final analyzed sample of 7,032 patients. Descriptive statistics on these 7,032 patients are given in Table 1.

Table 1.

Summary of baseline patient characteristics.

Age (years, mean ± sd)	60.6 ± 16.5
Male Sex (percent)	52.4
Race (percent)
White/Caucasian	74.8
Black/African/Haitian	17.3
Other/Unknown	7.8
APACHE IV Probability of In-Hospital Mortality (median [q1, q3])	0.12 [0.05, 0.29]
Source of Admission (percent)
ED	42.8
Gen Floor	0.8
ICU	22.5
SDU/Telemetry Unit	0.6
Non-Specific	33.3
Diabetes Mellitus (percent)	25.6
AIDS (percent)	0.3
COPD (percent)
None	78.9
No Limitations	10.8
Moderate	1.1
Severe	9.2
Dialysis (percent)	4.7
Heart Rate (BPM, mean ± sd)	97.7 ± 30.5
Hematocrit (%, mean ± sd)	31.8 ± 7.6
Mean Arterial Pressure (mmHg, mean ± sd)	91.6 ± 37.4
Potassium (mEq/L median [q1, q3])	3.8 [3.4, 4.3]
Respiratory Rate (breaths/min, mean ± sd)	34.2 ± 13.7
Serum Albumin (g/dL, median [q1, q3])	3.1 [2.5, 3.8]
Serum Bicarbonate (mEq/L, median [q1, q3])	22.0 [19.0, 25.0]
Serum Bilirubin mg/dL, median [q1, q3])	0.6 [0.4, 1.1]
Serum BUN (mg/dL, median [q1, q3])	22.0 [14.0, 37.0]
Serum Creatinine (mg/dL, median [q1, q3])	0.9 [0.7, 1.7]
Serum Na (mEq/L, mean ± sd)	137.6 ± 6.1
Serum WBC x 103 cells/μL, median [q1, q3])	10.2 [7.2, 15.8]
Reason For Transport (percent)
Unknown	54.3
Care Not Available	28.5
Protocol (Autolaunch)	8.8
Other	8.4
Time En Route To Completed (min, median [q1, q3])	71.0 [0.0, 103.0]
Time Arrived To Completed (min, median [q1, q3])	52.0 [0.0, 75.0]
Time Departing Facility To Completed (min, median [q1, q3])	31.0 [0.0, 48.0]
Transport Modality (percent)
Adult Ground	21.2
Adult Rotor	17.2
Non CCT Transport	15.1
Adult Fixed Wing	1.1
Unknown/Other	45.4
Hospital Referring Unit (percent)
ED	50.1
ICU	38.7
CCU	2.7
Other/Unknown	8.5
Arterial Line Placed (percent)	14.8
Intubation (percent)	1.7
Central Line Placed (percent)	0.7
Cardiac Assist Device Used (percent)	0.3
Pacer Used (percent)	<0.1

Our randomly-selected training cohort contained 5,274patients and our test cohort contained 1,758 patients. After fitting the logistic regression models and applying the models to the test cohort to obtain predicted probabilities, we found C-statistics [95% confidence intervals] of 0.854 [0.830 – 0.877] for the original APACHE-IV model; 0.640 [0.606 – 0.673] for the model that involved only variables associated with the transport encounter; and 0.854 [0.831 – 0.876] for the model that included both the APACHE-IV probability of mortality and the transport variables. Odds ratio estimates from the two models we estimated are given in Table 2.

Table 2.

Transport Only and Transport with APACHE Regression Models

Variable	Transport-Only Model	Transport + APACHE-IV Model
APACHE-IV Probability of Mortality (per 10% increase)	n/a	1.85 [1.78, 1.93]
Time between Arrival and Completion (per 10 minutes)	1.01 [1.00, 1.02]	1.01 [1.00, 1.02]
Arterial Line Placed	1.91 [1.58, 2.31]	0.96 [0.76, 1.22]
Cardiac Assist Device Used	2.49 [0.71, 8.76]	2.26 [0.49, 10.3]
Central Line Placed	3.28 [1.51, 7.10]	2.67 [0.97, 7.33]
Time between Departure and Completion (per 10 minutes)	0.99 [0.98, 1.00]	0.99 [0.98, 1.00]
Time between En-Route and Completion (per 10 minutes)	1.01 [1.00, 1.02]	1.01 [1.00, 1.02]
Intubation	1.80 [1.15, 2.82]	0.64 [0.38,.10]
Mode Completed: Adult Ground	(reference)	(reference)
Mode Completed: Adult Fixed Wing	1.28 [0.67, 2.42]	1.59 [0.76, 3.32]
Mode Completed: Adult Rotor	0.98 [0.77, 1.25]	0.82 [0.61, 1.09]
Mode Completed: Non CCT Transport	0.81 [0.61, 1.07]	1.06 [0.77, 1.47]
Mode Completed: Unknown/Other	1.02 [0.77, 1.36]	1.01 [0.72, 1.42]
Reason for Transport: Unknown	(reference)	(reference)
Reason for Transport: Care Not Available	0.85 [0.64, 1.11]	0.87 [0.62, 1.21]
Reason for Transport: Other	0.97 [0.69, 1.36]	1.08 [0.72, 1.61]
Reason for Transport: Protocol (Autolaunch)	0.86 [0.60, 1.24]	0.59 [0.38, 0.92]
Referring Unit: ED	(reference)	(reference)
Referring Unit: CCU	1.68 [1.10, 2.55]	1.79 [1.10, 2.92]
Referring Unit: ICU	1.64 [1.40, 1.92]	1.35 [1.11, 1.63]
Referring Unit: Other/Unknown	0.82 [0.60, 1.12]	1.05 [0.74, 1.49]

Odds ratios and approximate 95% confidence intervals for effects within a) the model using only predictors obtained from transport encounter data and b) the model using both the APACHE-IV probability of mortality score and the predictors from the transport encounter. The intercept for the transport-only model was −1.826 and the intercept term for the model using both APACHE-IV and transport data was −3.265.

Calibration plots for these three models, based on test-cohort data, are given in Figures 1–3. These plots indicate that: i) calibration was generally good for the APACHE-IV model, save for some under-estimation among patients with extremely high predicted probabilities of mortality (say, >0.8); ii) calibration was good for the transport-only model, although the distribution of predicted risk was rather condensed around values between 0.1 and 0.3; and iii) the combined model had good calibration (save for some over-estimation among patients with predicted probabilities below about 0.04) and a more dispersed distribution of predicted risk.

Figure 1.

Figure 3.

DISCUSSION

We found that transport-related predictors did not add to the predictive accuracy of the APACHE-IV model. When modelled alone, common transport variables had poorer accuracy than the current APACHE-IV prediction model. Significant predictors of mortality post-transfer from the CCTDM included procedures (placement of an arterial line, placement of a central catheter, or intubation) and being transferred from an intensive care unit – regardless of intensive care unit type.

Our findings that procedures performed by transport crews predict increased mortality are potentially useful in consideration of studies that have shown that patients transported by critical care transport teams show improved prediction scores, or less likelihood of dying, upon arrival at the receiving institution [12]. However, the improved mortality scores upon arrival are not correlated with increased rates of survival and thus represents an intermittent improvement in physiologic stability without concomitant change in survival trajectory. Although transport teams may provide short-term improvement in a patient’s physiologic status, the overall impact of transport may be diminished when considering the patient’s overall clinical trajectory.

Additionally, the finding that procedures performed by transport teams significantly predict post-transfer mortality are likely due to the procedure variables acting as a surrogate indicator for patient severity of illness. Although the procedures are performed and whether potentially appropriate interventions are administered or not (e.g. arterial line placement and antihypertensive medication administration) associated reductions in mortality are not realized. While noted limitations of current studies include not having access to data on previous interventions and the patient’s response to treatment[15–17] at the sending hospital and during transport, the anticipated improved predictive accuracy of these additional predictors may be of limited value during the transport phase.

After combining the CCTDM and APACHE data, admission source remained one of only two significant predictors in the final combined model. Admission source is widely supported as a significant predictor of post-transfer mortality in multiple patient populations [15, 17–21]. Similar to transport team procedures, while controlling for admission source via a single categorical variable may generally account for some “undefined, unmeasured, source of severity” of illness, or poor response to care [15] at the sending hospital, such an approach is not sensitive to potentially-useful details which may underlie observed clinical differences between patients from different admission sources. Further, admission source is not included in the other most commonly used Mortality Probability Model III [22, 23]. Just as procedures performed by the transport team is a significant predictor, the significance of admission source requires further investigation to discern if it is a surrogate marker for severity of illness and patient trajectory that may be unaffected by transfer and admission to higher levels of care.

Only one variable predicted improved mortality after transfer, that being the reason for transport as autolaunch. Autolaunch is the name of the expedited transfer process that is initiated when a patient presenting with a time-sensitive condition (i.e. myocardial infraction, trauma, stroke, aortic syndrome) is transferred after one phone call which immediately dispatches a helicopter that transfers the patiently directly to a waiting team treatment team (i.e. catheterization lab, operating room) at the receiving hospital. These results are consistent with studies that have shown a survival advantage for those experiencing time-sensitive emergencies and receive earlier treatment [24, 25].

There are several limitations of this study. The primary limitation is that the data are limited to one health system, and as such are not generalizable beyond it. Second, only records that contained complete data were used potentially introducing a selection bias. Lastly, this study did not include all transport related variables, due to data incompleteness, potentially limiting the impact that transport data may have on post-transport prediction modelling.

Future Directions

There are several significant findings from this study that inform next steps for investigation and clinical considerations. First, this is the first study with adequate power to assess the influence that transport has on post-transfer outcome. The most significant finding is that mode of transport, often the primary focus of most medical transfer studies, has no influence on post-transfer outcome [26–28]. Another significant finding is that transport time intervals (e.g. time between arrival and completion) where also not significant predictors of post-transfer mortality, countering previous findings that support time as an independent predictor [29]. The previous two findings, when taken in combination, directly challenge the primary assumption that more severely ill patients need to be moved by helicopter because it reduces out of hospital time and therefore improves patient outcomes.

Second, the primary finding that transport variables do not add prognostic value is an important finding that addresses one of the two major gaps in research of transported patient outcomes—those gaps being what happens to the patient prior to transfer and during transfer. These findings support focusing on the pre-transfer period specifically. Focusing on the pretransfer period is significant for several reasons. There is a major push to leverage the now widely available electronic medical record data to achieve a learning health care system—or the concept of using clinical data and analyzing it in near real-time to improve clinical practice on an individual patient basis [30]. Using data from the pre-transport phase is necessary to drive a clinical decision support system that would support a clinician deciding whether to transport a patient because that is the only data that is available at the time the decision is being made, simplifying future data modelling approaches.

Third, the finding that patients experiencing a time-sensitive emergency and undergo an autolaunch transfer exhibit improved mortality supports focusing future data modelling efforts on prospectively identifying patients that will benefit from earlier transfer to a high level of care for those that may be likely to fail conventional therapy or require aggressive therapy and interventions. Identifying patients that may or will require aggressive care earlier, and sending them to the tertiary center before they decompensate to the point of requiring tertiary services, can be one approach to reducing mortality for patients requiring transfer. Recent applications of telemedicine monitoring of community hospital ICUs by tertiary care providers has provided support of the usefulness of expert providers identifying patients that require transfer sooner [31, 32].

CONCLUSIONS

In Conclusion, we found that common transport variables did not in isolation, or in combination with APACHE-IV variables add to predictive accuracy of transferred patients post-transport mortality.

Future work should focus on the pre-transfer hospitalization and identifying factors that predict which patients require earlier transfer to higher levels of care.

Figure 2.

Highlights.

• Interhospital transfer patients experience increased mortality compared to patients that present directly to the same hospital and did not undergo interhospital transfer.

• Transport data in combination with APACHE data does not increase post-transport mortality prognostication.

• Patients transferred from sending hospital ICUs experience higher mortality rates.

• Patients transferred via expedited helicopter transfer protocols (Autolaunch) experience increased survival post-transfer.

• Future research should focus on pre-transport data.