Economic Evaluation of Pediatric Telemedicine Consultations to Rural Emergency Departments

Abstract

Background.

Comprehensive economic evaluations have not been conducted on telemedicine consultations to children in rural emergency departments (EDs).

Objective.

We conducted an economic evaluation to estimate the cost, effectiveness, and return on investment (ROI) of telemedicine consultations provided to health care providers of acutely ill and injured children in rural EDs compared with telephone consultations from a health care payer prospective.

Methods.

We built a decision model with parameters from primary programmatic data, national data, and the literature. We performed a base-case cost-effectiveness analysis (CEA), a probabilistic CEA with Monte Carlo simulation, and ROI estimation when CEA suggested cost-saving. The CEA was based on program effectiveness, derived from transfer decisions following telemedicine and telephone consultations.

Results.

The average cost for a telemedicine consultation was $3641 per child/ED/year in 2013 US dollars. Telemedicine consultations resulted in 31% fewer patient transfers compared with telephone consultations and a cost reduction of $4662 per child/ED/year. Our probabilistic CEA demonstrated telemedicine consultations were less costly than telephone consultations in 57% of simulation iterations. The ROI was calculated to be 1.28 ($4662/$3641) from the base-case analysis and estimated to be 1.96 from the probabilistic analysis, suggesting a $1.96 return for each dollar invested in telemedicine. Treating 10 acutely ill and injured children at each rural ED with telemedicine resulted in an annual cost-savings of $46,620 per ED.

Limitations.

Telephone and telemedicine consultations were not randomly assigned, potentially resulting in biased results.

Conclusions.

From a health care payer perspective, telemedicine consultations to health care providers of acutely ill and injured children presenting to rural EDs are cost-saving (base-case and more than half of Monte Carlo simulation iterations) or cost-effective compared with telephone consultations.

Children living in rural communities often have poor access to subspecialty physicians due to geographical barriers.^1,2 This lack of access to pediatric subspecialists, particularly for children presenting to rural emergency departments (EDs), may result in unnecessary patient admissions, transfers, or overuse of expensive transport modalities, such as rotor and fixed-wing air ambulances.³ One potential solution to this problem is the use of telemedicine, which allows consulting specialists to make more comprehensive medical evaluations and better informed recommendations remotely than would be possible with telephone consultations. As a result, the use of telemedicine could affect disposition decisions, thereby reducing patient transfers from rural EDs to higher level hospitals, potentially reducing overall health care costs.^4,5

Economic evaluation is becoming particularly important in comparing models of care. Few studies have evaluated the costs and effectiveness of telemedicine to deliver pediatric subspecialty consultations.^6,7 In fact, systematic reviews of cost-effectiveness studies on telemedicine have reported that there was insufficient evidence to prove telemedicine was cost-effective compared with conventional standards of care, because most of these studies did not consider relevant factors important in conducting high-quality cost-effectiveness analyses.^8,9 Some studies conducting economic evaluations on how telemedicine can reduce patient transfers have lacked a clear perspective to frame the valuation of incurred and averted costs.⁸ To our knowledge, no economic evaluation has been conducted to evaluate the use of telemedicine in providing pediatric subspecialist consultations to health care providers of acutely ill and injured children treated in rural EDs.

In this study, we sought to conduct an economic evaluation of a telemedicine program that provides pediatric critical care consultations to health care providers of acutely ill and injured children presenting to rural EDs. As a reference or a status quo, our economic evaluation compared similar acuity-triaged patients who received telephone consultations, the current standard of care. The goals were to conduct a cost analysis, a cost-effectiveness analysis (CEA), and return-on-investment (ROI) estimation of telemedicine consultations compared with the telephone consultations from a health care payer perspective.

METHODS

Overview of the Pediatric Critical Care Telemedicine Program

We conducted a retrospective review of the Pediatric Critical Care Telemedicine Program at the University of California, Davis Children’s Hospital (UC Davis Children’s Hospital). The program’s goal is to deliver immediate pediatric critical care consultations using telemedicine to health care providers of acutely ill and injured children (younger than 18 years) presenting to the region’s rural EDs. Our study examined 8 rural EDs in Northern California, where telemedicine was deployed between January 2003 and December 2009. The EDs participating in the telemedicine program were relatively small, with a total annual patient volume between 4000 and 10,000 visits and an average annual volume of acutely ill and injured children between 10 and 30.

Following telemedicine deployment, the physicians staffing the rural EDs were encouraged to use telemedicine consultations for pediatric patients presenting in the highest (sickest) triage category. All participating EDs had similar 3-level triage classification systems, with the highest level defined as those who were acutely ill and injured requiring immediate physician involvement. The treating ED physician had the authority to decide whether a patient needed a pediatric critical care consultation by telemedicine or telephone. If the treating rural ED physician desired a consultation, the pediatric critical care physician was contacted by pager, and either telemedicine or telephone consultation was provided. Telemedicine consultations consisted of live, interactive, and high-quality audiovisual communications between the remote ED physician and nurse, the pediatric patient, the parent, and the Children’s Hospital pediatric critical care physician.^10,11

We determined 2 key parameters from the program: 1) telemedicine operational cost and 2) patient transfer rates. To obtain telemedicine operational cost, we collected actual cost information on equipment, maintenance, and technical support. To obtain patient transfer rates, we reviewed all telephone and telemedicine consultations made to the Children’s Hospital using the hospital’s transfer center database, as well as a comprehensive review of all participating rural ED log and transfer books. We reviewed all medical records to obtain demographic information, including age, sex, race, and ethnicity, as well as discharge diagnosis and variables required to calculate the Pediatric Risk of Admission II (PRISA II) score.¹²

Cost Analysis

We calculated all medical expenditures from a health care payer perspective, which included 4 cost components per pediatric patient per year for each participating ED: telemedicine operational cost, ED visit cost, patient transfer cost, and hospital treatment cost. The telemedicine operational cost was the incremental cost over the baseline costs of telephone consultations.

The telemedicine operational cost was calculated as

$$\frac{\left[\text{Cost of telemedicine equipment and maintenance}/\text{ED}/\text{year}\right]+\left[\text{Cost of technical support}/\text{ED}/\text{year}\right]}{\#\text{ of pateints receiving telemedicine consultations}/\text{ED}/\text{year}}$$

where the cost of telemedicine equipment and maintenance was calculated as

$$\frac{\text{Cost of telemedicine equipment and maintenance}/\text{ED}}{\text{Year of medical equipment depreciation}}$$

and the cost of technical support was calculated as

$$\left[\text{Technician on call wage}/\text{hour}\right]\times \left[\text{Hours of technical support}/\text{ED}/\text{year}\right].$$

We assumed a constant depreciation of telemedicine equipment over 5 years.¹³

The average rural ED visit charge was calculated by summing the total facility and physician charges from the 2009 State Inpatient Databases (SID)¹⁴ from the Healthcare Cost and Utilization Project (HCUP).¹⁵ The SID contributes data to the Nationwide Emergency Department Sample (NEDS)¹⁶ and contains information on patients receiving treatment in the ED and admitted to the same hospital. The NEDS is a national database that collects data on hospital-based ED visits, including ED charges. We calculated the average rural ED visit cost using the cost-to-charge information from the 2009 HCUP, designed to be used with the SID and the Kids’ Inpatient Database (KID).¹⁷ The KID is a national database that collects administrative data, including hospital charge data, on a national probability sample of inpatient stays for children. For patients who were transferred from the rural EDs, we obtained the average patient transfer costs for the use of an air ambulance or ground ambulance from previously published data.¹⁸ For patients who were admitted to a hospital, we estimated the average hospital treatment costs for 5 common diagnoses: asthma, bronchiolitis, dehydration, fever, and pneumonia. We selected these diagnoses because rural clinicians and hospitals can sometimes manage these conditions without an obligate transfer to a higher level hospital.^19,20 We also calculated the average hospital treatment costs from the charge information and the cost-to-charge ratios from the 2009 HCUP.²¹ We calculated these average hospital treatment costs based on where the patients were admitted (rural, community, and tertiary hospitals).

Decision model

Figure 1 shows our decision model to evaluate the costs and effectiveness of telemedicine consultations relative to telephone consultations provided by pediatric critical care physicians among similar acuity-triaged children. The decision to transfer a patient, the mode of transport (air or ground ambulance), and location of hospital admission (rural, community, or tertiary hospital) were determined by the rural ED physician after considering the consulting pediatric critical care physician’s recommendations. We estimated the total costs for the 12 scenarios (Figure 1) differentiated by the consultation type, patient disposition, transfer modality, and the hospital of admission. For example, the total cost estimate for scenario 1 was the sum of the telemedicine operational cost, the rural ED visit cost, the transfer cost for helicopter, and the hospital treatment cost in the tertiary hospital.

Figure 1.

Decision tree with observed patient transfer rates and cost components in the base-case analysis. *The effectiveness measure was avoided patient transfers after children received telemedicine or telephone consultations. 0 = patient transfers, 1 = avoided patient transfers. ^†The total cost estimate for scenario 1: telemedicine operational cost + rural emergency department (ED) visit cost + air ambulance transfer cost + tertiary hospital treatment cost. The total cost estimate for scenario 2: telemedicine operational cost + rural ED visit cost + air ambulance transfer cost + community hospital treatment cost. The total cost estimate for scenario 3: telemedicine operational cost + rural ED visit cost + ground ambulance transfer cost + tertiary hospital treatment cost. The total cost estimate for scenario 4: telemedicine operational cost + rural ED visit cost + ground ambulance transfer cost + community hospital treatment cost. The total cost estimate for scenario 5: telemedicine operational cost + rural ED visit cost + rural hospital treatment cost. The total cost estimate for scenario 6: telemedicine operational cost + rural ED visit cost. The total cost estimate for scenario 7: rural ED visit cost + air ambulance transfer cost + tertiary hospital treatment cost. The total cost estimate for scenario 8: rural ED visit cost + air ambulance transfer cost + community hospital treatment cost. The total cost estimate for scenario 9: rural ED visit cost + ground ambulance transfer cost + tertiary hospital treatment cost. The total cost estimate for scenario 10: rural ED visit cost + ground ambulance transfer cost + community hospital treatment cost. The total cost estimate for scenario 11: rural ED visit cost + rural hospital treatment cost. The total cost estimate for scenario 12: rural ED visit cost.

Cost-Effectiveness Analysis

We evaluated the cost-effectiveness of telemedicine consultations compared with telephone consultations using the decision model. We performed 2 types of CEA: 1) a conventional CEA with base-case and sensitivity analyses and 2) a probabilistic CEA with Monte Carlo simulation to measure the incremental cost-effectiveness ratio (ICER) expressed as the net cost per transfer avoided. We used avoided patient transfer as the effectiveness measure to be consistent with previously published economic evaluation studies in telemedicine.^7,22–24 Table 1 describes the model parameters used in the base-case and probabilistic analyses.

Table 1.

Parameters Used in the Economic Evaluation of Telemedicine Consultations for Children Presenting to Rural Emergency Departments (EDs)

Parameter	Point Estimatea	Distribution	Distribution Parameters
Transfer rateb
Telephone consultations
Baseline transfer rate (P_t_ph)	88%	Beta	n = 64, α = 56, β = 8c
Transfer rate with air ambulance (P_a_ph)	45%	Beta	n = 56, α = 25, β = 31
To tertiary hospital (P_at_ph)	80%	Beta	n = 25, α = 20, β = 5
Transfer rate with ground ambulance to tertiary hospital (P_gt_ph)	58%	Beta	n = 31, α = 18, β = 13
Admission rate to rural hospital (P_ad_ph)	50%	Beta	n = 8, α = 4, β = 4
Telemedicine consultations
Telemedicine transfer rate (P_t_tele)	61%	Beta	n = 71, α = 43, β = 28
Transfer rate with air ambulance (P_a_tele)	46%	Beta	n = 43, α = 20, β = 23
To tertiary hospital (P_at_tele)	85%	Beta	n = 20, α = 17, β = 3
Transfer rate with ground ambulance to tertiary hospital (P_gt_tele)	87%	Beta	n = 23, α = 20, β = 3
Admission rate to rural hospital (P_ad_tele)	25%	Beta	n = 28, α = 7, β = 21
Transfer reduction rate	31%	Not applicable	Not applicable
Costd
Telemedicine operational cost per child/ED/yeare	$3641	Gamma	$5193 (SD)
ED visit costf	$1237	Gamma	$912 (SD)
Transportation costg
Air ambulance	$18,872	Gamma	α = 1, λ= 1/18,872h
Ground ambulance	$2555	Gamma	α = 1, λ = 1/2555h
Hospital treatment cost per child/hospitalization/yeari
Rural hospital	$6369	Gamma	$5918 (SD)
Community hospital	$7256	Gamma	$17,443 (SD)
Tertiary hospital	$16,422	Gamma	$39,647 (SD)

^a.Used in both base-case and probabilistic analyses.

^b.Derived from the Pediatric Critical Care Telemedicine Program.

^c.α = r, β = n − r, where r represents the number of successes and n represents the number of trials.

^d.2013 US dollar value.

^e.Defined as [cost of telemedicine equipment and maintenance/ED/year ($3004) + cost of technical support/ED/year ($2675)] / # of patients receiving telemedicine consultations/ED/year. Derived from the Pediatric Critical Care Telemedicine Program.

^f.Derived from the State Inpatient Database.

^g.Derived from the previously published literature.¹⁸

^h.Derived from the previously published book.²⁶

^i.Derived from the Kids’ Inpatient Database.

The “baseline transfer rate” and “telemedicine transfer rate” indicated the proportion of pediatric patients triaged at the highest acuity level who were transferred from the participating rural ED to a higher level hospital following a telephone or telemedicine consultation, respectively. We defined the “effectiveness of telemedicine” as the transfer reduction rate, calculated as

$$\frac{\text{Baseline trandfer rate }-\text{ Telemedicine transfer rate}}{\text{Baseline transfer rate}}.$$

We obtained the cost measures used for the CEA from the cost analysis. The ICERs represented the incremental cost per patient transfer avoided from the rural ED to a higher level hospital. The ICERs were calculated as

$$\frac{\left[\text{Costs of care for a patient receiving a telemedicine consultation}\right]-\left[\text{Costs of care for a patient receiving a telephone consultation}\right]}{\left[\#\text{of patients not transferred after a telemedicine consultation}\right]-\left[\#\text{of patients not transferred after a telephone consultation}\right]}.$$

Assuming a positive denominator, positive ICERs indicated costs were greater with telemedicine consultations, while negative ICERs indicated telemedicine consultations were cost-saving compared with telephone consultations. A threshold ICER value represents society’s maximum willingness to pay, in the form of medical expenditures, for telemedicine consultations to avoid a patient transfer. If the estimated ICER was less than a threshold value, it would imply that telemedicine consultations were preferred to—and more effective than—telephone consultations, considering the societal willingness to pay. Currently, however, there is neither an established nor published ICER threshold value representing society’s willingness to pay to avoid a patient transfer.

Base-case analysis and sensitivity analysis

We used point estimates for each model parameter (Table 1) for the base-case analysis. First, we constructed a tornado diagram to identify the most important and sensitive parameters, included in the base-case analysis, that had most influence on the ICER. The diagram is a set of 1-way sensitivity analyses presented in a single graph. The wider the bar on the tornado diagram, the larger the potential influence on the ICER. For all the transfer rates, we applied the minimum and maximum values derived from the programmatic data as the ranges. For the hospital treatment costs, we applied the mean ±1 standard deviation derived from the KID as the ranges. We did not include the interfacility transportation costs in the tornado diagram because of the limited published data and the inability to obtain reliable standard deviations. Second, we performed 1-way sensitivity analysis and 2-way sensitivity analysis to evaluate how the ICER was influenced by changing 2 key parameters: the effectiveness of telemedicine and the number of acutely ill and injured children who received telemedicine consultations per ED per year. In the 1-way sensitivity analysis, the ICER was influenced by varying only 1 parameter. In the 2-way sensitivity analysis, the ICER was influenced by varying both parameters simultaneously.

Probabilistic analysis with Monte Carlo simulation

We performed the Monte Carlo simulation with a hypothetical cohort (5000 iterations) to estimate the ICER by addressing the uncertainty of multiple model parameters simultaneously. We applied a beta distribution for all transfer rates using the integer parameter only option in the TreeAgePro 2014 software (Williamstown, Massachusetts, USA). The values α and β in the beta distribution were calculated as

$$\alpha =r,\beta =n-r,$$

where n represents the number of successes among n trials.

For example, for the baseline transfer rate (P_t_ph), r = 56 and n = 64, and for the telemedicine transfer rate (P_t_tele), r = 43 and n = 71 (Table 1).

We applied a gamma distribution for all of the cost parameters. The values α and β in the gamma distribution were calculated as²⁵

$$\alpha =\frac{{\mu }^2}{S^2},\beta =\frac{S^2}{{\mu }^2},$$

where μ = mean, S = standard deviation.

Since the air and ground ambulance interfacility transportation costs were obtained from a single publication without standard deviation information, we assumed the standard deviation to be the same value as the mean.²⁶ We interpreted the simulation results under different assumptions for the societal willingness to pay to avoid a patient transfer.

Return on Investment

When our CEA results demonstrated that telemedicine consultations were cost-saving under the base-case and/or probabilistic analyses, we calculated the ROI as the ratio of the savings from telemedicine consultations to the cost of telemedicine consultations. We calculated the amount saved as the expected reduction in medical expenditures following telemedicine consultations compared with telephone consultations. Specifically, the amount saved was calculated as

$$\left[\text{Medical expenditures following a telephone consultation}/\text{patient}/\text{ED}/\text{year}\right]-\left[\text{Medical expenditures following a telemedicine consultation}/\text{patient}/\text{ED}/\text{year}\right]$$

The ROI was calculated as

$$\frac{\text{Amount saved from telemedicine consultations per child}/\text{ED}/\text{year}}{\text{Telemedicine operational cost per child}/\text{ED}/\text{year}}$$

All dollar values were adjusted to 2013 US dollars (USD) using the medical care commodities expenditure category of the Consumer Price Index from the US Bureau of Labor Statistics.²⁷ We performed all analyses using TreeAge Pro 2014. This study was approved by the UC Davis Human Subjects Review Board.

RESULTS

Patient Demographics

Our primary data included 135 children who presented to the 8 rural EDs in the highest triage category and received either a telemedicine or telephone consultation between January 2003 and December 2009 (Table 2). Of these children, 71 (52.6%) received telemedicine consultations and 64 (47.4%) received telephone consultations. The children who received telemedicine consultations were younger than those who received telephone consultations (3.7 v. 5.4 years; P < 0.05). The 2 groups were otherwise similar with regard to their sex, race and ethnicity, and diagnosis at discharge.

Table 2.

Demographics of Patients Who Received Telemedicine and Telephone Consultations in Rural Emergency Departments

	Overall (n = 135)	Telemedicine (n = 71)	Telephone (n = 64)	P Value
Age, mean (SD), ya	4.5 (4.8)	3.7 (4.2)	5.4 (5.2)	0.04
Sex, n (%)
Male	81 (60.0)	44 (62.0)	37 (57.8)	0.62
Female	54 (40.0)	27 (38.0)	27 (42.2)
Race, n (%)
Caucasian	67 (49.6)	38 (53.5)	29 (45.3)	0.34
Non-Caucasian	68 (50.4)	33 (46.5)	35 (54.7)
Ethnicity, n (%)
Hispanic	34 (25.2)	18 (25.4)	16 (25.0)	0.96
Non-Hispanic	101 (74.8)	53 (74.6)	48 (75.0)
Discharge diagnosis, n (%)
Asthma	2 (1.5)	0	2 (3.1)	0.55
Bronchiolitis	20 (14.8)	10 (14.1)	10 (15.6)
Dehydration	25 (18.5)	13 (18.3)	12 (18.8)
Fever	28 (20.7)	18 (25.4)	10 (15.6)
Pneumonia	12 (8.9)	6 (8.5)	6 (9.4)
Other	48 (35.6)	24 (33.8)	24 (37.5)
PRISA II score, mean (SD)	12.1 (12.3)	11.0 (12.1)	13.3 (12.4)	0.29

PRISA II, Pediatric Risk of Admission II.

^a.P < 0.05 is statistically significant.

Cost Analysis

The cost for telemedicine equipment and maintenance was $3004 per ED, per year, and the cost for technical support was $2675 per ED, per year. The average annual telemedicine consultation cost was $3641 per consultation, per child, per ED and ranged from $1656 to $28,399 among the 8 participating EDs.

Cost-Effectiveness Analysis

Base-case analysis

Sixty-one percent of the children who received telemedicine consultations and 88% of children who received telephone consultations were transferred to a higher level hospital. Based on these data, the effectiveness of telemedicine consultations was 31%. The negative ICER under the base-case analysis (Table 3) indicated telemedicine consultations were cost-saving compared with telephone consultations.

Table 3.

One-Way Sensitivity Analysis for Patients Who Received Telemedicine Consultations Compared with Telephone Consultations in Rural Emergency Departments

Parameter: Effectiveness of Telemedicinea	Consultation	Costb	No. (%) of Patients Not Transferredc	ICERd (USD/Transfer Avoided)
10%	Telephone	$2,348,680	13 (20.3)
	Telemedicine	$2,725,630	21 (29.6)	$43,080
15%	Telephone	$2,348,680	13 (20.3)
	Telemedicine	$2,610,154	26 (36.6)	$19,921
20%	Telephone	$2,348,680	13 (20.3)
	Telemedicine	$2,496,788	30 (42.3)	$8342
25%	Telephone	$2,348,680	13 (20.3)
	Telemedicine	$2,379,203	34 (47.9)	$1295
31% (Base-case analysis)	Telemedicine	$2,263,728	39 (56.3)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$3236e
35%	Telemedicine	$2,148,252	43 (60.6)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$6544
40%	Telemedicine	$2,032,777	48 (67.6)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$9025
45%	Telemedicine	$1,917,301	52 (73.3)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$10,955
50%	Telemedicine	$1,801,826	56 (78.9)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$12,500
55%	Telemedicine	$1,686,350	61 (85.9)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$13,763
60%	Telemedicine	$1,570,875	65 (91.5)	(cost-saving)
	Telephone	$2,348,680	13 (20.3)	−$14,815

^a.Transfer reduction rate using telemedicine, % = 100% * [baseline transfer rate − telemedicine transfer rate]/baseline transfer rate.

^b.2013 US dollar (USD) value.

^c.Children who received telemedicine consultations: n = 71; children who received telephone consultations: n = 64.

^d.ICER (incremental cost-effectiveness ratio) = [Costs of care for a patient receiving a telemedicine consultation] − [Costs of care for a patient receiving a telephone consultation]/[# of patients not transferred after a telemedicine consultation] − [# of patients not transferred after a telephone consultation].

^e.A negative ICER indicated telemedicine consultations were cost-saving compared with telephone consultations. The negative ICER value itself did not represent the amount of cost-saving.

One-way sensitivity analysis

From the tornado diagram (Figure 2), the parameter with the largest impact on the ICER was the effectiveness of telemedicine. The parameters on the transfer rate with air ambulance with telephone consultations (P_a_ph), the transfer rate with ground ambulance to tertiary hospital with telephone consultations (P_gt_ph), and the transfer rate with air ambulance to tertiary hospital with telephone consultations (P_at_ph) had the larger impacts on the ICER.

Figure 2.

Tornado diagram representing the influence on the incremental cost-effectiveness ratios from a set of 1-way sensitivity analyses. The variable with the widest bar on the top indicates it has the largest potential influence on the incremental cost-effectiveness ratio. USD, US dollars.

Table 3 also demonstrates that the ICER was sensitive to changes in the effectiveness of telemedicine. The more effective telemedicine was at reducing transfer rates, the lower the ICER (i.e., preferring telemedicine consultations). Assuming a modest reduction in transfer rates following telemedicine consultations, or a telemedicine effectiveness of 25%, telemedicine consultations were more costly than telephone consultations by $1295 per patient transfer avoided. When the effectiveness of telemedicine exceeded 26.5%, telemedicine consultations were always cost-saving, resulting in fewer patient transfers and lower overall costs.

Two-way sensitivity analysis

Figure 3 shows our 2-way sensitivity analysis, varying the 2 key parameters: 1) the effectiveness of telemedicine and 2) the number of acutely ill and injured children who received telemedicine consultations per ED per year. This figure indicates that telemedicine consultations were preferred because they were cost-saving (compared with telephone consultations) for more than 90% of the feasible combinations of these 2 key parameters. Figure 3 also suggests 2 thresholds for the cost-effectiveness of telemedicine. First, when the effectiveness of telemedicine exceeded 24%, telemedicine consultations were preferred to telephone consultations (i.e., telemedicine consultations were more effective and less costly and hence cost-saving) as long as there were more than 2 children receiving a telemedicine consultation per ED per year. Second, if the effectiveness of telemedicine exceeded 14% and more than 10 acutely ill and injured children received telemedicine consultations per year, telemedicine was preferred and resulted in cost-savings.

Figure 3.

Two-way sensitivity analysis evaluating telemedicine effectiveness and patient volume when the willingness to pay is $0. ED, emergency department.

Probabilistic analysis with Mont Carlo simulation

Assuming a willingness to pay of $0 to avoid 1 patient transfer, telemedicine consultations were cost-saving and hence preferred in 57% of 5000 iterations of the Monte Carlo simulation. When the willingness to pay increased from $0 to $5000, the likelihood that telemedicine was preferred increased from 57% to 71%.

Return on Investment

Table 4 summarizes the ROI results of our telemedicine program. In the base-case analysis, the average amount saved from the use of telemedicine due to a relevant reduction in medical expenditures was $4662 per consultation while the cost was $3641 per child, per ED, per year. As a result, the ROI was 1.28 ($4662/$3641). The estimated mean ROI from the probabilistic analysis was 1.96 (95% confidence interval: 0.17, 6.24), suggesting a $1.96 return for each dollar invested in telemedicine. When 10 acutely ill and injured children were treated in the rural ED each year using telemedicine, the estimated cost-savings for each ED was $46,620 per year.

Table 4.

Return on Investment (ROI) of Telemedicine Consultations for Children Presenting to Rural Emergency Departments (EDs)

	Base-Case Analysis	Probabilistic Analysis Mean (95% Confidence Interval)
Amount saved from telemedicine consultations ($ per child/ED/year)a	$4662	$6614 ($618, $20,206)
Cost of telemedicine consultations ($ per child/ED/year)b	$3641	$4023 ($1960, $8455)
ROI	1.28	1.96 (0.17, 6.24)

^a.The amount saved from telemedicine consultations was the expected reduction in medical expenditure. Specifically, it was calculated as medical expenditure following a telephone consultation – medical expenditure following a telemedicine consultation.

^b.Telemedicine operational cost = $3641 per child/ED/year (2013 US dollar value).

DISCUSSION

In this study, using data specific to the UC Davis Pediatric Critical Care Telemedicine Program serving small rural EDs, we estimated the average cost of a pediatric critical care telemedicine consultation to be $3641 per child, per ED, per year. However, considering this cost in combination with the costs resulting from patient transfer decisions (i.e., costs of transport and hospitalization at the referral hospital), we determined that for a variety of scenarios, the telemedicine program was more effective and resulted in lower overall health care costs from a health care payer perspective compared with telephone consultations provided among similar acuity-triaged patients. Because telemedicine consultations were cost-saving under both base-case CEA and more than half of the probabilistic CEA (57%), we reported the ROI as the primary result. Our calculated ROI was 1.28 with an annual saving of $46,620 when 10 children received telemedicine consultations at a rural ED.

Our findings that telemedicine consultations resulted in a reduction of patient transfers and cost-savings are similar to other studies evaluating the impact of telemedicine consultations in other clinical settings. Others have reported a cost-saving of $4838 (2013 USD) per avoided transfer among infants in a telecardiology program,⁷ $47,440 (2013 USD) per hospital per year among acute ischemic stroke patients in a telestroke program,²⁸ and $48,435 (2013 USD) per health system per year among infants in an acute care telemedicine program.²⁹ Our ROI findings were also similar to the evaluation of a tele–mental health program conducted by Lokkerbol et al,³⁰ who also found telemedicine to be favorable and cost-effective compared with telephone consultations.

There are several notable strengths to our study. First, this is the first economic study to evaluate the costs, effectiveness, and ROI of a pediatric telemedicine program providing pediatric critical care consultations to children treated in rural EDs. Second, the patient transfer rates were calculated based on actual experience from the telemedicine program, and hospital costs were based on regional and national data sets, respectively. The use of actual data in our model parameters was therefore more likely to produce results with good internal and external validity. Third, our telemedicine program was cost-saving or cost-effective due to a reduction in patient transfers from rural EDs to higher level hospitals, which is consistent with clinicians’ views that patients from remote EDs are often overtransferred to be on the side of caution due to the lack of access to specialists.^3,31,32 Finally, we used probabilistic CEA with Monte Carlo simulation to evaluate the parameter uncertainties of our results.

Limitations

Our study has several limitations. First, the patients included were not randomly assigned to receive telephone or telemedicine consultations, which could have resulted in selection bias; however, we inclusively sampled all acutely ill and injured children who were in the highest triage level, and our 2 cohorts had similar diagnostic profiles. Second, our results might not have accurately represented other pediatric telemedicine programs providing similar clinical services to other EDs across the nation. Furthermore, because there may have been missed opportunities to obtain more telemedicine consultations, the telemedicine operational costs could have been better shared and therefore the telemedicine operational costs lowered. The reason for the relatively low telemedicine utilization is because our program focused on small, underserved rural hospitals with low pediatric ED volumes. Finally, while the use of telemedicine resulted in a reduction of the overall transfer rate and potentially more informed transfer decisions, the absence of patient follow-up data did not allow us to assess whether patients experienced postdischarge health problems or required additional hospital or clinic visits as a result of not being transferred.

CONCLUSIONS

Our study demonstrates that providing pediatric critical care consultations to acutely ill and injured children presenting to rural EDs using telemedicine results in fewer interfacility transports and reduces overall health care costs. We found telemedicine consultations to be cost-saving (base-case and more than half of Monte Carlo simulation iterations) or cost-effective from a health care payer perspective compared with telephone consultations, the current standard of care. Our study findings have important implications for clinicians, health administrators, and policy makers considering implementing similar telemedicine care models for children living in rural communities facing disparities in access to specialist physicians.