Predicting Hospital Admission and Returns to the Emergency Department for Elderly Patients

Abstract

Objectives

Methods to accurately identify elderly patients with a high likelihood of hospital admission or subsequent return to the emergency department (ED) might facilitate the development of interventions to expedite the admission process, improve patient care, and reduce overcrowding. This study sought to identify variables found among elderly ED patients that could predict either hospital admission or return to the ED.

Methods

All visits by patients 75 years of age or older during 2007 at an academic ED serving a large community of elderly were reviewed. Clinical and demographic data were used to construct regression models to predict admission or ED return. These models were then validated in a second group of patients 75 and older who presented during two 1-month periods in 2008.

Results

Of 4,873 visits, 3,188 resulted in admission (65.4%). Regression modeling identified five variables statistically related to the probability of admission: age, triage score, heart rate, diastolic blood pressure, and chief complaint. Upon validation, the c-statistic of the receiver operating characteristic (ROC) curve was 0.73, moderately predictive of admission. We were unable to produce models that predicted ED return for these elderly patients.

Conclusions

A derived and validated triage-based model is presented that provides a moderately accurate probability of hospital admission of elderly patients. If validated experimentally, this model might expedite the admission process for elderly ED patients. Our models failed, as have others, to accurately predict ED return among elderly patients, underscoring the challenge of identifying those individuals at risk for early ED returns.

The Institute of Medicine has recently reported that our national health care system is unprepared to provide for the future social and health care needs of our elderly.¹ This is especially relevant in the emergency department (ED), where elderly patients undergo more diagnostic testing and have longer length of stays than younger patients.^2,3 Taking these factors into account, one potential way to decompress overcrowded EDs might be to identify a subset of elderly patients for whom the likelihood of admission is very high, allowing the ordering of an inpatient bed at triage and avoiding delays in admission. A recent study has argued that the Emergency Severity Index (ESI, version 3) has moderate validity in predicting hospitalizations (c-statistic = 0.77) and is associated with increased resource utilization and 1-year survival in patients 65 years of age and older.⁴ Based on their results, the authors of that study suggest that the ESI score alone could be used to identify elderly patients at high risk for admission or death. By inference, if these results can be generalized to the EDs of other institutions, then the ESI score might serve as a useful means to rapidly identify some of those elderly patients for whom an expedited admission process would shorten ED resource utilization.

Another possible method to improve ED overcrowding might be to reduce the high rate of return visits by elderly patients. In a systematic review of interventions to reduce ED visits among elderly, McCusker and Verdon⁵ found that only two of nine trials of interventions offered to elderly ED patients reported decreased ED return visits and that these reported reductions were of only “borderline” statistical significance. In a second systematic review of interventions to improve outcomes of elderly discharged from the ED, Hastings and Heflin⁶ found only one nonrandomized trial of an “ED-based planning discharge program” that was successful at decreasing return ED visits and then only at 8 and 14 days. Based on these results, it remains uncertain whether a more selective intervention that targets only elderly patients at increased risk for return visits to the ED might successfully decrease ED utilization. If a profile of elderly patients at high risk for return visits to the ED could be developed, then these selected elderly patients might comprise a group that could serve as the target of future interventions to reduce ED utilization. Therefore, in this study we sought to identify a set of variables in patients 75 years of age or older, available at the time of triage, that would predict 1) the probability of a patient’s admission to the hospital or 2) a return visit to the ED within 30 days of an initial visit.

METHODS

Study Design

We conducted a retrospective study of consecutive ED visits by patients 75 years and over during a 1-year period to identify triage-based information that predicted patients’ hospital admission or return to the ED within 30 days of initial visit. This age group was chosen because these patients have the highest degree of comorbidity and frailty and, in our area, constitute a significant percentage of our ED population. Moreover, this age group is the subject of separate analyses in the National Hospital Ambulatory Medical Care Survey (NHAMCS).⁷ The results of the models developed from this derivation data set were then validated by applying them to a second group of patients who presented to the same ED during two 1-month periods during the following 1-year period. The institutional review board of the University of North Carolina at Chapel Hill approved this study.

Study Setting and Population

The site of this study was the ED of a tertiary care, Level 1 trauma center serving rural and urban populations in North Carolina. The hospital operates 724 beds, is associated with a medical school, and is a major referral center. In fiscal year 2007, the hospital had 741,980 clinic visits, 31,934 hospital discharges, and 61,200 ED visits.⁸

North Carolina has experienced a substantial increase in its elderly population. The population of persons over age 65 years has increased by 21% from 1990 to 2000, while the population of elderly aged 75 and above increased by 36.4% from 319,000 to 435,000 during that same time period.⁹ Future projections suggest that this trend will continue as the proportion of persons over age 65 years is expected to increase from 11.9% in 2002 to 15.0% in 2020.¹⁰

Selection of Subjects

The charts of all patients aged 75 years and above presenting to the UNC Hospitals ED from January 1, 2007, to December 31, 2007, were selected for analysis. A second block of charts obtained for all patients aged 75 years and over presenting to the same ED from June 1 to June 30, 2008, and December 1 to December 31, 2008, was assembled in a similar manner. For both blocks of charts, records were also examined to identify those study patients who sought care in the 30 days preceding the block period’s end date and who returned to the ED within 30 days. The first block of patient data served as the derivation set and the second block as the validation set.¹¹

Study Protocol and Measurement

Data were obtained from several sources: 1) T-System (Dallas, TX), which records clinical information pertaining to the patients’ ED visit; 2) Webcis, the electronic medical record system developed and used by UNC Hospitals; and 3) International Classification of Diseases Version 9 (ICD-9) coding records processed for each visit.

Records were evaluated and data abstracted for each patient’s demographic (age, sex, race/ethnicity), insurance (Medicare/Medicaid status), chief complaint, vital signs, ESI triage level (version 4.1),^12,13 and disposition (admitted, transferred, discharged). Chief complaints were coded independently by two reviewers according to the Reason for Visit Classification schema, developed for use with the NHAMCS by the National Center for Health Statistics.¹⁴ Because there could be up to three chief complaints recorded for each ED visit, calculation of kappa scores was considered unsuitable and, therefore, we chose to calculate interobserver matching scores for the chief complaints. For these calculations, we considered a match to represent agreement for a chief complaint only when both observers ranked a given chief complaint similarly. For example, chest pain was considered a match only when both observers ranked chest pain as the first chief complaint, even though the second observer might have included chest pain in other positions on his/her list. An admission was defined as a decision to admit the patient to an inpatient floor, psychiatry, intermediate care, or intensive care unit bed or if the patient went directly to surgery from the ED. Patients who passed through the ED for direct admission or as an inbound hospital transfer, but were not registered and did not receive medical evaluation in the ED, were excluded from this study. Charlson Comorbidity Index scores were calculated for each visit, based on the first 10 ICD-9 codes associated with each visit, according to a method developed by Stagg.¹⁵ The first set of vital signs listed in each patient’s chart was abstracted from the electronic medical record, using a specially developed computer program.

Data Analysis

Emergency department data from all sources were combined into two databases for the two study periods. Using STATA version 10.1 (College Station, TX), a logistic regression analysis was performed on the calendar year 2007 database to identify those factors most predictive of a patient’s admission. Through an iterative process, independent variables were removed from the logistic regression and the model was reduced until a final model was achieved.¹⁶ All removed variables were examined by a likelihood ratio test for the appropriateness of their exclusion from the model.

Using Microsoft Excel (Redmond, WA), the final logistic regression equation, developed from the 2007 data, was then applied to the 2008 data set to predict probabilities of admission for these patients, in a blinded manner. These probabilities of admission were changed into the dichotomous variable “admitted” or “not admitted” by calculating the number of projected admissions at various thresholds of admission certainty from 5% to 95%. These projected admission rates were then compared to the actual dispositions of the patients. With this information, the sensitivity and specificity of the model were calculated at each threshold, allowing the construction of a receiver operating characteristic (ROC) curve. For this ROC curve, the c-statistic was calculated to obtain an estimate of the power of the admission prediction model.

In a similar manner, logistic regression models were developed to predict the following conditions: overall probability of a patient’s return to the ED within 30 days of the initial visit; probability of a patient’s admission and subsequent return to the ED within 30 days of the initial visit; and finally, probability of a patient’s discharge from the ED and subsequent return to the ED within 30 days. These models were also applied to the 2008 data in a blinded manner to permit comparison of their predictions to actual patient outcomes. ROC curves and c-statistics were also calculated for these models.

RESULTS

Between January 1, 2007 and December 31, 2007, there were 4,873 visits by 3,079 patients aged 75 years and above at the study site’s ED. None of these visits or patients were excluded from our analysis.

Table 1 lists the characteristics of all patients, as well as the characteristics of patients listed according to their disposition, including demographic information, insurance status, Charlson comorbidity score, ESI triage score, vital signs taken at triage, and the most commonly given chief complaints. The median age of patients enrolled in this study was 82 years, 60.8% of patients were female, and the majority of patients (79.1%) were enrolled in Medicare. Initial vital signs are shown in Table 1. The most common presenting chief complaints were 1) accident including fall, 2) shortness of breath, 3) chest pain, 4) altered level of consciousness, and 5) abdominal pain. The interobserver matching scores for the first chief complaint in the 2007 data set were 85.6% (±1.0%) and 82.1% (±2.6%) in the 2008 data set, indicating excellent coding agreement. The admission rate for all patients 75 years of age or older in 2007 was 65.4%.

Table 1.

Demographic and Presenting Characteristics of the Study Patients (2007)

Characteristic	All Visits (N = 4,873)	Admitted Visits (N = 3,188)	Not Admitted Visits (N = 1,685)
Sex, n (%)
Male	1,909 (39.2)	1,259 (39.5)	650 (38.6)
Female	2,964 (60.8)	1,929 (60.5)	1,035 (61.4)
Age (yr), median	82	82	81
Race or ethnic group, n (%)*
Black or African American	893 (18.3)	574 (18.0)	319 (18.9)
White	3,452 (70.8)	2,296 (72.0)	1,156 (68.6)
Hispanic or Latino	40 (0.8)	26 (0.8)	14 (0.8)
Asian	25 (0.5)	14 (0.4)	11 (0.6)
American Indian/Alaska Native	3 (<0.1)	3 (0.1)	0 (0)
Unknown or not reported	460 (9.4)	275 (8.6)	185 (11.0)
Insurance status, n (%)
Medicare	3,857 (79.2)	2,503 (78.5)	1,353 (80.3)
Medicaid	576 (11.8)	382 (12.0)	194 (11.5)
Initial vital signs, mean (n) ±SD
Systolic blood pressure, mm Hg	143 (4,703) ±30.3	141 (3,066) ±30.6	148 (1,637) ±29.1
Diastolic blood pressure, mm Hg	74 (4,703) ±16.5	73 (3,066) ±17.3	76 (1,637) ±14.7
Pulse, beats/min	81 (4,771) ±19.5	83 (3,121) ±20.9	78 (1,650) ±15.9
Respiratory rate, breaths/min	19 (4,748) ±4.5	19 (3,118) ±4.9	18 (1,630) ±3.5
Oxygen saturation, %	97 (4,203) ±4.3	97 (2,981) ±4.4	97 (1,222) ±4.0
Temperature, °C	36.5 (4,455) ±0.7	36.5 (2,880) ±0.7	36.4 (1,575) ±0.6
Charlson comorbidity score, mean (±SD)	0.79 (±0.86)	0.80 (±0.87)	0.77 (±0.86)
ESI level, mean (±SD)†	2.7 (±0.7)	2.5 (±0.6)	3.2 (±0.7)
Most common chief complaints, frequency (%)
Accident, NOS (includes fall)	582 (11.9)	300 (9.4)	282 (16.7)
Shortness of breath	417 (8.6)	364 (11.4)	53 (3.2)
Chest pain	368 (7.6)	323 (10.1)	45 (2.7)
Altered level of consciousness, NOS	326 (6.7)	265 (8.3)	61 (3.6)
Abdominal pain, cramps, spasms, NOS	235 (4.8)	158 (5.0)	77 (4.6)

^*Race or ethnic group as recorded in the patient’s chart.

^†ESI score has five levels: 1 = most acute and 5 = least acute.

ESI = Emergency Severity Index; NOS = not otherwise specified.

The variables that predicted admission in the final logistic regression model are listed in Table 2, along with their associated odds ratios (ORs). Age, heart rate, diastolic blood pressure, ESI triage score, and the presence of one of several chief complaints were important in predicting the probability of a patient’s admission. Of note, sex, race, insurance status, and level of comorbidity, as represented by the Charlson comorbidity score, were not predictive of a patient’s probability of admission in the final model. In this model (Table 2), increased patient age and heart rate were associated with higher odds of admission, as were lower diastolic blood pressure and triage score. Several chief complaints, such as stroke, blood in stool, and pneumonia, were associated with higher odds of admission, while several chief complaints, such as hematuria, dysuria, or high blood pressure, were associated with lower odds of admission.

Table 2.

ORs for Variables Predictive of Admission

Variable	OR*	95% CI	p-value†
Age	1.02	1.01–1.04	0.001
ESI level	0.29	0.25–0.33	0.000
Heart rate	1.01	1.01–1.01	0.000
Diastolic blood pressure	0.99	0.98–0.99	0.000
Chief complaints
General weakness	2.00	1.42–2.83	0.000
Fainting (syncope)	3.06	1.95–4.79	0.000
Chest pain	2.89	2.03–4.13	0.000
Neurologic weakness	6.05	2.15–17.05	0.001
Shortness of breath	3.27	2.37–4.50	0.000
Labored or difficult breathing	2.86	1.59–5.18	0.000
Vomiting	2.89	1.97–4.24	0.000
Abdominal pain	1.75	1.30–2.37	0.000
Decreased appetite	3.65	1.22–10.91	0.021
Blood in stool	6.93	1.58–30.44	0.010
Blood in urine	0.47	0.25–0.87	0.016
Painful urination	0.09	0.01–0.77	0.027
Anemia	4.71	1.36–16.34	0.015
High blood pressure	0.43	0.24–0.76	0.004
Nonischemic heart disease	6.18	2.17–17.62	0.001
Cerebrovascular disease (stroke)	5.49	2.89–10.43	0.000
Pneumonia	7.27	1.68–31.37	0.008
Psychiatric evaluation needed	2.47	1.18–5.21	0.017
Leg/hip fracture/dislocation	6.33	2.43–16.54	0.000
Head/neck laceration	0.28	0.12–0.68	0.004
Head/neck/facial injury	0.49	0.29–0.82	0.007
Hip injury	4.70	2.42–9.13	0.000
Altered level of consciousness	2.11	1.53–2.90	0.000
Presenting for lab abnormality	2.50	1.37–4.54	0.003

^*These values represent the log β (ORs) derived from the regression analysis.

^†These values represent the associated significance (p-value) that the regression model assigned to each variable.

ESI = Emergency Severity Index; OR = odds ratio.

The variables that were included in the final logistic regression models that were developed to predict return to the ED in 30 days, admission and subsequent return to the ED in 30 days, and discharge and subsequent return to the ED in 30 days are listed in Table 3. Race, insurance status, and Charlson comorbidity score were not found to be predictive of a patient’s probability of experiencing any of these outcomes. In each of these three models, several patient variables were associated with increased odds of experiencing each respective outcome (Table 3). For example, presentations with a general viral infection or complaining of stomach or abdominal pain were associated with higher odds of returning to the ED within 30 days. Other conditions, such as hypertension, or leg or hip fracture were associated with lower odds of returning to the ED within 30 days.

Table 3.

ORs for Variables Predictive of Return Within 30 Days, Admit and Return Within 30 Days, and Discharge and Return Within 30 Days

Variable	Return Within 30 Days (Regardless of Disposition)			Admit and Return Within 30 Days			Discharge and Return Within 30 Days
	OR*	95% CI	p-value†	OR*	95% CI	p-value†	OR*	95% CI	p-value†
Sex	1.31	1.12–1.53	0.00
ESI				0.68	0.58–0.79	0.00	2.18	1.85–2.56	0.00
Temperature	0.89	0.79–1.00	0.06
Diastolic blood pressure	0.99	0.99–1.00	0.00	0.99	0.99–1.00	0.00
General weakness				1.57	1.06–2.32	0.03
Chest pain				1.55	1.14–2.12	0.01
Allergy, NOS							5.44	1.33–22.28	0.02
Disorders of speech, speech disturbance				5.67	1.25–25.80	0.03
Epistaxis							3.39	1.59–7.24	0.00
Dyspnea				1.73	1.09–2.75	0.02
General viral infection (flu)	9.37	0.85–103.82	0.07
Symptoms referable to lips	10.26	0.93–113.51	0.06
Stomach and abdominal pain	6.03	1.34–27.12	0.02				5.72	1.09–29.90	0.04
Lower abdominal pain							4.18	1.13–15.57	0.03
Abdominal distension				12.23	2.45–61.16	0.00
Urinary tract infection				3.00	1.18–7.66	0.02
Infection of skin of hand, arm, or finger							6.37	1.17–34.66	0.03
Foot or toe swelling							7.67	1.78–33.04	0.01
Hypertension	0.41	0.16–1.02	0.05
Need for tube insertion	2.20	1.11–4.35	0.02				3.31	1.55–7.07	0.00
Leg/hip fracture	0.27	0.06–1.11	0.07
Accident, NOS (includes fall)							1.48	1.10–1.99	0.01
Altered level of consciousness				1.40	1.01–1.95	0.04

^*These values represent the log β (ORs) derived from the regression analysis.

^†These values represent the associated significance (p-value) that the regression model assigned to each variable.

ESI = Emergency Severity Index; NOS = not otherwise specified; OR = odds ratio.

Between June 1 and June 30, 2008, and December 1 and December 31, 2008, there were 843 visits made by 731 patients to the study site’s ED. The characteristics of this patient population are similar to those of the 2007 population. These patients were also admitted at a rate similar to that of those patients seen in 2007 (65.3%).

Figure 1 displays the ROC curves for the logistic regression equations applied to the 843 patient visits in 2008 for the four clinical outcomes of interest. The c-statistic for the admission prediction equation was 0.73 (95% confidence interval [CI] = 0.69 to 0.76), indicating a moderate predictive power. The c-statistic for the ROC curve using the ESI triage score alone as a predictor of admission was 0.68 (95% CI = 0.65 to 0.72), indicating a weaker predictive power. (To demonstrate how the admission prediction model might perform in our ED, see Data Supplement S1, available as supporting information in the online version of this paper.) The c-statistics for the remaining regression equations that modeled returns to the ED within 30 days for all patients, and for those admitted and discharged from the ED, were below 0.65, indicating poor predictive power.

Figure 1.

ROC curves for predictive models

DISCUSSION

Our work shows that it is possible to model, with a moderate degree of assurance, the factors associated with an elderly patient’s admission to the hospital from the ED at the University of North Carolina Hospitals. Although this admission model is moderately accurate in our own institution, and has a c-statistic as good as or better than other reported regression models, it must be emphasized that neither this model nor any other prediction model reported to date is sufficiently powerful to be used as a clinical tool for deciding which patients should be admitted or discharged from the ED. Our prediction tool, however, as developed from our logistic regression model, may allow clinicians to identify a subset of elderly patients at triage that are very likely to be admitted to the hospital. This information, if coupled with a system of early bed requests, expedited testing, and early notification of an inpatient admitting team, could serve to shorten the admission process by several hours for some of these elderly, frail patients. Although the c-statistic for our model predicts likelihood ratios of only modest strength, and its application in our ED with an admission rate of 65% appears to provide only a modest improvement in predicting the probability of admission, it must be remembered that currently none of those elderly patients destined for admission are offered expedited bed placement. Thus, this model can improve bed placement for some, but not all, patients 75 years of age and older (please see Data Supplement S1 for an example).

It is noteworthy that the ESI triage algorithm, although an important part of our model, is less powerful than our model in predicting the probability of admission to our hospital for elderly ED patients. Although Baumann and Strout⁴ report that the ESI score was highly predictive of admission (c-statistic = 0.77), the ESI score alone did not perform as well as an admission prediction tool in our patient population. This difference in applicability may relate to several factors, such as the higher age of our study population, a different application of the triage algorithm in our hospital ED (ESI version 3 vs. version 4), or other unknown factors. Also, in the study by Baumann and Strout, the ESI algorithm was not compared with other models to determine its singular superiority over other prediction instruments. Finally, it is important to remember that the ESI score was developed not to predict admission, but to prioritize incoming patients at initial triage and identify those patients who cannot wait to be seen.¹² For this reason, the ESI score may not serve as the single best predictor of hospital admission.

Our work also demonstrates that it may not be possible to reliably model the probability of a patient’s return to the ED within 30 days, a patient’s return to the ED within 30 days after discharge from the ED, or a patient’s return to the ED within 30 days after admission to the hospital. The rate of ED return visits by elderly patients is substantial, exceeding 40% at 1 year in some settings.¹⁷ The reasons for these high rates of return visits are uncertain but may depend on a number of factors, such as chronic medical conditions with high relapse rates, inadequate primary care, a patient’s social and/or psychological characteristics, and poor quality of care at the initial visit.¹⁷ A recent study of a large administrative database has suggested that the high rates of return visits by elderly to the ED or to the hospital after hospital discharge are associated with certain ICD codes and the failure of discharged patients to obtain a timely follow-up visit with a primary physician.¹⁸ Unfortunately, the calculated ORs in that study, although quite informative for the elderly population at large, are too low to be predictive of an individual’s return visit to the ED. Moreover, many of these administrative variables are not available at triage or at discharge from the ED.

Our study is not the first to examine patient characteristics searching for methods to identify elderly patients at risk for high future health care utilization rates,^19–27 although many of these studies evaluated risk of future hospitalization and not ED use specifically. McCusker et al.¹⁷ analyzed data from four Montreal hospitals to develop a model (Identification of Seniors at Risk [ISAR]) for predicting ED return visits at 30 days. The final ISAR model produced an area under the ROC curve of 0.63, demonstrating insufficient power to accurately predict return visits. These observations, when coupled to our study, support our conclusion that it is difficult to produce models with patient information available during the ED evaluation that can reliably predict return visits. This finding has an important policy implication, as it suggests that using return ED visits as an ED quality indicator may be inappropriate because of the difficulty in identifying those likely to return.

LIMITATIONS

First, our work is centered in one study site, which may constrain the generalizability of the research findings. Second, our model does not address the appropriateness of an admission, only the propensity of our emergency physicians to admit patients with certain characteristics. Thus, this model may perform differently in places where rates of admission of elderly patients differ from the rates described in our ED here. Third, our study relies upon retrospective data, rather than data collected from a cohort followed prospectively. However, because the information we used to predict admission is not subjective, we do not think that a prospective approach would yield substantially different results. Finally, the tool developed in our work to predict admission to the hospital has not been subjected to a formal trial of its ability to improve ED throughput or decrease ED length of stay.

CONCLUSIONS

The results of our study suggest that more proximal events, such as patient admission, may be predicted with modest accuracy based on variables readily available to emergency staff at triage, but that more distal events, such as patient returns to the ED, may not be reliably predicted. Our admission prediction tool may facilitate the patient admission process by identifying a subset of elderly patients early on in their ED course who are at high likelihood of being admitted, allowing the ordering of an inpatient bed at triage. The failure of our models to accurately predict ED returns among elderly patients underscores the challenge of identifying these at-risk individuals and may help guide the direction of future research.