Predicting Hospital Admission in a Pediatric Emergency Department using an Artificial Neural Network

Abstract

Hospital admission delays in the Emergency Department (ED) reduce capacity and contribute to the ED’s diversion problem. We evaluated the accuracy of an Artificial Neural Network for the early prediction of hospital admission using data from 43,077 pediatric ED encounters. We used 9 variables commonly available in the ED setting. The area under the receiver operating characteristic curve was 0.897 (95% CI: 0.887–0.896). The instrument demonstrated high accuracy and may be used to alert clinicians to initiate admission processes earlier during a patient’s ED encounter.

INTRODUCTION

Overcrowding continues to be a problem faced by hospitals around the country. It no longer affects just urban academic hospitals, but also suburban and rural hospitals. One of the main factors contributing to overcrowding is the shortage of available inpatient beds. This results in increased length of stay for patients in the ED waiting to be admitted to the hospital (“boarding”). The increasing number of “boarding” patients reduces the overall throughput of the ED by consuming ED resources and staff time, which can restrict access for new patients seeking emergency care. Because earlier initiation of admission procedures may lead to a decreased burden of ED diversion, we developed and evaluated an Artificial Neural Network (ANN) to predict hospital admission of pediatric patients. The instrument requires no additional data input from users and applies data elements usually available in real-time in many ED settings.

METHODS

The pediatric ED at Vanderbilt University Medical Center is an academic, Level 1 Trauma center with an annual patient volume of >40,000. The ED staff use a computerized whiteboard for patient tracking and work-flow management. We used available ED whiteboard data from a 14-month period to develop a fully connected feed forward ANN with one hidden layer. We excluded patients who left the ED without being seen, left against medical advice, died in the ED, or transferred to another facility. The ANN considered only 9 input data variables that were available early during a patient’s ED encounter. The final network model included age, gender, ICD-9 coded chief complaint, the 5-level Emergency Severity Index measure for acuity, type of consult, the presence of at least one laboratory test, at least one radiology test, and an ECG exam, and mode of arrival (ambulance, helicopter, car, or other). The outcome variable was hospital admission. Missing variables were encoded and kept in the data set. We used data from >2 months (4/10/04 to 6/30/04) to create and train the ANN and data from 2 months (7/1/04 to 8/31/04) to monitor generalization during training. The final network was validated against data from a 10 month period (9/1/04 to 6/30/05). We used area under the receiver operator characteristic curve (AUC) to evaluate the system’s performance. We evaluated the system’s operational characteristics at fixed sensitivity levels of 90% and 95%.

RESULTS

The hospital admission rate was 15% for the train and validation data sets and 16% for the test set. Complete data were available for 6 out of 9 variables; the remaining 3 variables had missing information ranging from 0.04% for age to 4.5% for acuity. The network contained 15 hidden neurons. The AUC for the training set was 0.909 (95% CI: 0.900–0.917), for the test set 0.907 (95% CI: 0.896–0.916), and for the validation set 0.897 (95% CI: 0.887–0.896). Table 1 lists the model’s test characteristics at fixed 90% and 95% sensitivity levels for the independent validation set.

Table 1.

Test characteristics at fixed sensitivity levels

Sensitivity (fixed)	Specificity	Predictive value pos	Predictive value neg.
90%	70.0%	35.3%	97.5%
95%	59.4%	29.8%	98.5%

DISCUSSION

The ANN had a strong ability to discriminate between patients. Since the variables used by the network are already captured by hospital information technology environments, the network has the ability to be integrated into existing patient tracking and work flow systems. The system may help predict and anticipate expected, near-term ED workload and hospital bed needs.