Appropriateness of Disposition Following Telemedicine Consultations in Rural Emergency Departments

Abstract

Objectives:

To compare the appropriateness of hospital admission in eight rural emergency departments among a cohort of acutely ill and injured children who receive telemedicine consultations from pediatric critical care physicians to a cohort of similar children who receive telephone consultations from the same group of physicians.

Design:

Retrospective cohort study between January 2003 and May 2012.

Setting:

Eight rural emergency departments in Northern California.

Patients:

Acutely ill and injured children triaged to the highest-level triage category who received either telemedicine or telephone consultations.

Interventions:

Telemedicine and telephone consultations.

Measurements and Main Results:

We compared the overall and stratified observed-to-expected hospital admission ratios between telemedicine and telephone cohorts by calculating the risk of admission using the second generation of Pediatric Risk of Admission score and the Revised Pediatric Emergency Assessment Tool. A total of 138 charts were reviewed; 74 children received telemedicine consultations and 64 received telephone consultations. The telemedicine cohort had fewer hospital admissions compared with the telephone cohort (59.5% vs 87.5%; p < 0.05). Although the telemedicine cohort had lower observed-to-expected admission ratios than the telephone cohort, these differences were not statistically different (Pediatric Risk of Admission II, 2.36 vs 2.58; Revised Pediatric Emergency Assessment Tool, 2.34 vs 2.57). This result did not change when the cohorts were stratified into low (below median) and high (above median) risk of admission cohorts, using either Pediatric Risk of Admission II (low risk, 18.25 vs 22.81; high risk, 1.40 vs 1.54) or Revised Pediatric Emergency Assessment Tool (low risk, 5.35 vs 5.94; high risk, 1.51 vs 1.81).

Conclusions:

Although the overall admission rate among patients receiving telemedicine consultations was lower than that among patients receiving telephone consultations, there were no statistically significant differences between the observed-to-expected admission ratios using Pediatric Risk of Admission II and Revised Pediatric Emergency Assessment Tool. Our findings may be reassuring in the context of previous research, suggesting that telemedicine specialty consultations can aid in the delivery of more appropriate, safer, and higher quality of care.

Rural emergency departments (EDs) treat higher proportions of children, on average, than suburban and urban EDs (1); however, pediatric healthcare resources, including subspecialty pediatricians, are often less available in these facilities (2–4). Rural EDs are less likely to have pediatric trauma services and also have limited access to pediatric sub-specialists and recommended pediatric equipment and supplies (3, 5, 6). Pediatric emergency services are difficult to maintain in rural hospitals because of low patient volumes, difficulty in recruiting and retaining healthcare providers, and unfavorable economies of scale (7–9). These factors may explain published reports that some care provided to pediatric patients in rural EDs more frequently involves incorrect diagnoses, inappropriate treatments, and suboptimal medical management (5, 6, 10). Particularly for acutely ill and injured children, these problems may contribute to the higher observed mortality rates seen among children treated in rural areas (11, 12). Given these concerns, rural EDs may be cautious and may tend to admit and/or transfer children who are not seriously ill. This approach, however, can sometimes result in unnecessary patient admissions, transfers, or overuse of expensive transport modalities (13).

In previous studies, the use of telemedicine to provide pediatric specialist consultations in rural EDs has been shown to improve the overall quality of care provided to acutely ill and injured children (9, 14, 15). The use of telemedicine consultations has also been shown to result in higher patient satisfaction with care and fewer physician-related medication errors compared to consultations provided using the telephone (16–19). However, the impact of telemedicine consultations on the appropriateness of ED disposition decisions remains unknown. To better understand the impact of telemedicine consultations on admission and transfer decisions among acutely ill and injured children treated in rural EDs, we used two previously published and validated pediatric risk-of-admission algorithms to compare ratios of observed-to-expected numbers of hospital admissions (O/E admission ratios) between acutely ill and injured children who received telemedicine consultations and those who received telephone consultations. We hypothesized that those children who received telemedicine consultations in eight rural EDs in Northern California would have more favorable (lower) O/E admission ratios than those who received telephone consultations in the same EDs, after adjusting for patient characteristics.

MATERIALS AND METHODS

Study Design and Setting

This study was part of a larger program designed to deliver immediate consultations to acutely ill and injured children presenting to rural EDs from pediatric critical care physicians using telemedicine and to investigate its impact on patient outcomes such as quality of care and medication errors. We conducted a retrospective chart review of the Pediatric Critical Care Telemedicine Program at the University of California Davis Children’s Hospital (UC Davis Children’s Hospital). Eight rural EDs having access to both telemedicine and telephone consultations from UC Davis Children’s Hospital provided the data for this study. All participating EDs were located in designated rural areas, as defined by California’s Office of Statewide Health Planning and Development (20) and the Federal Center for Medicare & Medicaid Services (21), and were within UC Davis Children’s Hospital’s referral region. These hospitals were also located in underserved communities according to the Health Resources and Services Administration’s definition of Health Professional Shortage Areas, Medically Underserved Areas, and Medically Underserved Populations (21, 22). The EDs participating in the telemedicine program are relatively small, with a total annual patient volume between 4,000 and 10,000 visits and an average annual volume of children triaged to the highest level triage category (acutely ill and injured) between 10 and 30.

Telemedicine Equipment

The telemedicine program used pole-mounted telemedicine systems with a turnkey videoconferencing unit (either Polycom or Tandberg-Cisco), a flat-screen high-resolution monitor, and an uninterrupted power supply. The videoconferencing unit provided high-definition two-way video using a camera with remote zoom, pan, and tilt capabilities. Installation of the telemedicine units took place at the eight participating EDs on a rolling basis between 2003 and 2007.

Telemedicine and Telephone Consultations

Pediatric critical care physicians were available for both telemedicine and telephone consultations around the clock. At each participating ED, the treating physician had the ultimate authority to decide which patient needed a consultation and whether the consultation was to be obtained using telemedicine or telephone. If the treating rural ED physician desired a consultation, the pediatric critical care physician would be contacted by pager and provide the consultation either by telemedicine or telephone, at the discretion of the rural ED provider. Telemedicine consultations consisted of live, interactive, audiovisual communications between the remote ED physician and nurse, the pediatric patient, the parent/guardian, and the Children’s Hospital pediatric critical care physician.

Selection of Patients

Between January 2003 and May 2012, we included patients between the ages of 1 day and 17 years at the participating EDs who received either a telemedicine or telephone consultation from any of the pediatric critical care physicians. We included patients if they were triaged at the highest acuity level of a three-level triage classification. This was defined as those who were acutely ill and injured, with emergency conditions requiring immediate physician involvement. All participating EDs had similar three-level triage classifications. For all eligible children, we sampled and performed detailed reviews of consecutive medical records for those who received telemedicine and telephone consultations during the study period.

Outcome Measures

For all eligible patients, we calculated the risk of admission using the second generation of Pediatric Risk of Admission (PRISA II) and the Revised Pediatric Emergency Assessment Tool (RePEAT) (23, 24). The PRISA II algorithm estimates the risk of “mandatory admission” based on a previously validated list of services usually provided in acute care hospitals, whereas the RePEAT algorithm estimates the risk of admission based on admissions from a large multicenter sample of pediatric ED visits (23, 25). The PRISA II algorithm uses physiologic and therapeutic data collected during the ED stay, including laboratory results and oxygen requirement. The RePEAT algorithm uses data collected at the time of initial ED triage to predict patient dispositions.

In our study, consistent with PRISA II and RePEAT, we defined admissions to include both local admissions to the rural hospital and transfers to a referring higher level of care for admission. In addition, we also calculated the mean summary PRISA II scores for both the telemedicine and telephone cohorts. However, we did not calculate the summary RePEAT scores because, in addition to the variables required to calculate the risk of admission, the algorithm includes a variable for whether or not a specialty consultation was obtained, and by definition, all patients included in our study required a specialty consultation.

Our primary outcome measures were the PRISA II O/E admission ratio and the RePEAT O/E admission ratio. Our secondary outcome measures were the stratified O/E admission ratios using the median probabilities for the PRISA II risk of admission and the RePEAT risk of admission. Because we were applying the PRISA II and RePEAT algorithms to a sample of ED patients with an inherently high risk of admission (i.e., patients for which the rural ED was seeking assistance from a pediatric critical care physician), we expected O/E admission ratios to be greater than one. We considered lower O/E admission ratios to be more favorable.

Statistical Analysis

We compared descriptive measurements using a chi-square test and Student t test where appropriate. We compared observed and expected admissions using Poisson regression, considering observed admissions as the counted observations in our analyses. We performed all analyses with STATA v12 (StataCorp LP, College Station, TX). All statistical tests were two-sided at a significance level of 0.05. This study was approved by the UC Davis Human Subjects Review Board.

RESULTS

During the study period, 138 pediatric patients in the eight participating EDs received either telemedicine or telephone consultations. Among these, 74 received telemedicine consultations and 64 received telephone consultations. As shown in Table 1, children who received telemedicine consultations were younger (3.7 yr vs 5.4 yr; p < 0.05) and were less likely to have been brought into the ED by emergency medical services (16.2% vs 31.2%; p < 0.05). Without risk adjustment, children receiving telemedicine consultations were more likely to have been discharged home (29.7% vs 6.3%; p < 0.05) and less likely to have been transferred to a higher level hospital for admission (59.5% vs 87.5%; p < 0.05).

TABLE 1.

Demographic Characteristics of Children Receiving Telemedicine and Telephone Pediatric Critical Care Consultations

Parameter	Total (n = 138)	Telemedicine (n = 74)	Telephone (n = 64)
Age, mean (sd)a	4.5 (4.8)	3.7 (4.2)	5.4 (5.2)
Gender, n (%)
Male	83 (60.1)	46 (62.2)	37 (57.8)
Female	55 (39.9)	28 (37.8)	27 (42.2)
Ethnicity, n (%)
Hispanic	36 (26.1)	20 (27.0)	16 (25.0)
Non-Hispanic	102 (73.9)	54 (73.0)	48 (75.0)
Patients arriving by emergency medical services, n (%)a
No	106 (76.8)	62 (83.8)	44 (68.8)
Yes	32 (23.2)	12 (16.2)	20 (31.2)
Minor injuryb, n (%)
No	109 (79.0)	58 (78.4)	51 (79.7)
Yes	29 (21.0)	16 (21.6)	13 (20.3)
Disposition of care, n (%)a
Discharged homea	26 (18.8)	22 (29.7)	4 (6.3)
Admitted to the rural hospital	12 (8.7)	8 (10.8)	4 (6.3)
Transferred for admissiona	100 (72.5)	44 (59.5)	56 (87.5)
Transferred to PICU	86 (85.2)	41 (89.1)	45 (81.8)

^ap < 0.05.

^bStrain, sprain, abrasion, laceration, and fracture without associated cranial, thoracic, or abdominal injury, defined by the second generation of pediatric risk of admission score (23).

Table 2 describes the mean PRISA II scores, the mean PRISA II risk of mandatory admission, and the mean RePEAT risk of admission for children included in the telemedicine and telephone cohorts. The cohorts of patients who received telemedicine or telephone consultations had similar mean PRISA II scores and mean PRISA II risk of admission. However, patients who received telemedicine consultations had a significantly lower mean RePEAT risk of admission than those who received telephone consultations (0.28 vs 0.37; p < 0.05).

TABLE 2.

Pediatric Risk of Admission II and Revised Pediatric Emergency Assessment Tool Risk of Admission Among Telemedicine and Telephone Cohorts

Admission Risk Parameter	Total	Telemedicine	Telephone
	Mean (sd)	Mean (sd)	Mean (sd)
PRISA II score (23)	12.1 (12.3)	11.1 (12.2)	13.3 (12.4)
PRISA II risk of mandatory admission	0.33 (0.35)	0.30 (0.34)	0.36 (0.35)
Revised Pediatric Emergency Assessment Tool (24) risk of admissiona	0.32 (0.23)	0.28 (0.23)	0.37 (0.22)

PRISA = Pediatric Risk of Admission.

^ap < 0.05.

Table 3 shows the overall and stratified O/E admission ratios using the PRISA II median risk of mandatory admission and the RePEAT median risk of admission. Children who received telemedicine consultations had minimally and nonsignificantly lower overall O/E admission ratios (PRISA II, 2.36 vs 2.58; p = 0.46; RePEAT, 2.34 vs 2.57; p = 0.63). When the sample was stratified at the median risk of mandatory admission (PRISA II) of 16%, patients receiving telemedicine consultations had nonsignificantly lower stratified O/E admission ratios than patients receiving telephone consultations (18.22 vs 22.83, p = 0.44 and 1.40 vs 1.54, p = 0.71, respectively). Similarly, children who received telemedicine consultations had minimally and nonsignificantly lower stratified O/E admission ratios below and above the 28% median risk of admission using RePEAT (5.35 vs 5.94, p = 0.71 and 1.51 vs 1.81, p = 0.49, respectively).

TABLE 3.

Observed and Expected Admission Ratios Using Pediatric Risk of Admission II and Revised Pediatric Emergency Assessment Tool for Patients Receiving Telemedicine and Telephone Pediatric Critical Care Consultations

Risk of Admission	Telemedicine			Telephone
	Observed Admissions	Expected Admissions	O/E Admission Ratio (95 % CI)	Observed Admissions	Expected Admissions	O/E Admission Ratio (95% CI)
Pediatric Risk of Admission II (23)
Overall	52	22.02	2.36 (1.80–3.10)	60	23.28	2.58 (2.00–3.32)
≤ 0.16	23	1.26	18.25 (12.13–27.47)	26	1.14	22.81 (15.5–33.5)
> 0.16	29	20.75	1.40 (0.97–2.01)	34	22.14	1.54 (1.10–2.15)
Revised Pediatric Emergency Assessment Tool (24)
Overall	52	22.20	2.34 (1.78–3.07)	60	23.36	2.57 (1.99–3.31)
≤ 0.28	29	5.42	5.35 (3.72–7.70)	24	4.04	5.94 (3.98–8.86)
> 0.28	23	15.20	1.51 (1.01–2.28)	36	19.85	1.81 (1.31–2.51)

O/E = observed to expected.

DISCUSSION

In our study, acutely ill and injured children in rural EDs who received telemedicine consultations from pediatric critical care physicians were younger, more likely to have been discharged, and less likely to have been transferred to a higher level hospital for admission than children who received telephone consultations. However, because children who received telephone consultations had a slightly higher risk of predicted admission than those who received telemedicine consultations using the PRISA II and RePEAT algorithms, there were no statistically significant differences in O/E admission ratios.

Several studies have reported subjective data demonstrating a tendency of nontertiary EDs to “overtriage” pediatric patients, resulting in a range of inappropriate patient transfer and admission rates from 13.5% to 30% (26–28). With this in mind, telemedicine has been proposed as a means of involving pediatric specialists in the ED care to more appropriately direct patient assessments, triage, and disposition decisions, including the use of emergency medical services and transportation modalities for pediatric transfers (29, 30). Kofos et al (31) demonstrated that children can receive accurate evaluations from pediatric specialists when using telemedicine, possibly increasing the likelihood for more appropriate transport decisions. Labarbera et al (30) found that after the initiation of a telemedicine program from a tertiary children’s hospital to a community hospital ED, there was a significant reduction in the frequency of transfer, and among those requiring transfer, a reduction in the proportion of patients ultimately diverted from the PICU to the pediatric ward. Other studies in a variety of clinical settings have concluded that telemedicine would result in a reduction of patient transfers and ambulance utilization, thus reducing unnecessary transfers and costs (32, 33). Our finding that telemedicine consultations were associated with fewer admissions and/or transfers for admission suggests that telemedicine may increase the accuracy of patients’ disposition decisions as well as reduce the frequency of medically unnecessary admissions. Emergency telemedicine could therefore reduce rural disparities in access to specialist physicians (34), improve children’s quality of care at small hospitals (16, 17, 35), and increase accuracy of triage and transport decisions to referral hospitals (36, 37).

Our study has several limitations. First, telemedicine consultations were not randomly assigned among patients, leading to measured and unmeasured differences in cohort case-mix. For example, physicians working in the rural EDs may have preferentially used the telephone for patients that they were confident needed admission and/or transfer, or for patients that were so ill, they considered the use of telemedicine too onerous. Patients receiving telemedicine consultations tended to be younger and have a lower mean predicted risk of admission, suggesting that these patients may have been less ill and less in need of admission and/or transfer. However, our risk adjustment using both PRISA II and RePEAT should theoretically adjust for these and other unmeasured differences. Another potential limitation of using the PRISA II and RePEAT algorithms was the lack of documentation regarding triage findings, physiologic variables, and detailed laboratory results for each patient. Consistent with the algorithms, we assumed all missing values were normal, which could underestimate patients’ severity of illness.

Additional limitations include the fact that participating rural EDs in our study that used telemedicine may not be representative of other EDs, limiting the external validity of our findings. As well, the sample size included in our analyses was relatively small and likely limited our ability to detect statistically significant differences between cohorts. To have been able to detect a difference of O/E admission ratios between our telemedicine and telephone cohorts with a power of 80%, we would have needed more than 300 pediatric patients in each cohort. This limitation is partly a result of the fact that we intentionally implemented our telemedicine program in low-volume rural EDs where specialist consultations could have a greater impact. Finally, we made the assumption that lower O/E ratios are more favorable, which could be questioned, particularly as we were unable to assess whether patients experienced postdischarge health problems or required additional clinic or hospital visits.

Nevertheless, there are several strengths to our study. To the best of our knowledge, this is the first study that has evaluated the impact of telemedicine consultations on the ratios of observed to the expected hospital admissions for children presenting to rural EDs. Previous studies have reported higher rates of nonurgent healthcare utilization and inappropriate admissions from non-children’s hospital EDs, but our study was underpowered to evaluate whether telemedicine consultation can address these inefficiencies (38–40). However, our findings may be reassuring in the context of previous research suggesting that telemedicine specialty consultations can result in more appropriate, safer, and higher quality of care (41).

CONCLUSIONS

We found that children receiving telemedicine consultations in eight rural EDs from a group of pediatric critical care physicians had lower rates of local rural hospital admission and transfer for higher level hospital admission than children receiving telephone consultations. However, we did not find a statistically significant difference in O/E admission ratios between the two cohorts. Future research is needed to better understand the impact of telemedicine consultations on clinical outcomes for pediatric patients in ED and other settings.