Monte Carlo Simulation Modeling of a Regional Stroke Team’s Use of Telemedicine

Abstract

Objectives

The objective of this study was to evaluate operational policies that may improve the proportion of eligible stroke patients within a population who would receive intravenous recombinant tissue plasminogen activator (rt-PA), and minimize time to treatment in eligible patients.

Methods

In the context of a regional stroke team, the authors examined the effects of staff location and telemedicine deployment policies on the timeliness of thrombolytic treatment, and estimated the efficacy and cost-effectiveness of six different policies. A process map comprising the steps from recognition of stroke symptoms to intravenous administration of rt-PA was constructed using data from published literature combined with expert opinion. Six scenarios were investigated: telemedicine deployment (none, all, or outer-ring hospitals only); and, staff location (center of region or anywhere in region). Physician locations were randomly generated based on their zip codes of residence and work. The outcomes of interest were onset-to-treatment (OTT) time, door-to-needle (DTN) time, and the proportion of patients treated within three hours. A Monte Carlo simulation of the stroke team care-delivery system was constructed based on a primary dataset of 121 ischemic stroke patients who were potentially eligible for treatment with rt-PA.

Results

With the physician located randomly in the region, deploying telemedicine at all hospitals in the region (compared with partial or no telemedicine) would result in the highest rates of treatment within three hours (80% vs. 75% vs. 70%) and the shortest OTT (148 vs. 164 vs. 176 minutes), and DTN (45 vs. 61 vs. 73 minutes) times. However, locating the on-call physician centrally coupled with partial telemedicine deployment (five of the 17 hospitals) would be most cost-effective with comparable eligibility and treatment times.

Conclusions

Given the potential societal benefits, continued efforts to deploy telemedicine appear warranted. Aligning the incentives between those who would have to fund the up-front technology investments and those who will benefit over time from reduced ongoing health care expenses will be necessary to fully realize the benefits of telemedicine for stroke care.

INTRODUCTION

Appropriate treatment of ischemic stroke requires temporal urgency. Every 15-minute reduction in delay to treatment with recombinant tissue plasminogen activator (rt-PA) results in increased odds (OR, 1.04; 95% CI = 1.03 to 1.05; P < 0.001) of the patient being independent at hospital discharge.^1,2 Despite this urgency, many patients do not get proper stroke care in a timely manner. At presumably highly motivated centers that participate in the American Stroke Association’s (ASA) Get with the Guidelines quality initiative, only half of all rt-PA treated patients received treatment within the recommended 60 minutes from hospital arrival after a quality improvement intervention; just 26.5% achieved this goal pre-intervention.³

One approach to increasing the responsiveness of medical centers to stroke patients is to organize regional stroke teams offering clinical and technical support. In the Greater Cincinnati area, the stroke team has a stroke physician on call 24/7. Once notified of a potential candidate for treatment, the on-call physician typically travels to the hospital where the patient is located in order to provide care, while other clinical and diagnostic workup proceeds. Although travel time from the stroke physician’s location to the patient’s bedside occurs in parallel with diagnostic and imaging work, long travel times have the potential to delay care.

To provide treatment more rapidly, health care providers are turning to advanced telemedicine technologies. Telestroke provides stroke team physicians with enhanced communication with remote patients by providing a two-way, audio-visual connection with integrated electronic medical information, scans, and tests results, as well as clinical assessment tools. Telestroke can facilitate timely rt-PA treatment without lowering the quality of care.⁴ However, the technology can be expensive, and deploying it at all care sites may not be financially viable, despite evidence that telestroke can be cost-effective in the long term.⁶ It seems likely that, under a constrained budget, equipping all hospitals in a region with telestroke units may be cost-prohibitive. Therefore, perhaps the farthest hospitals in a region, with the longest stroke physician travel times, should be the first locations to receive telestroke units. The travel distance to and, hence, time-to-treatment for patients at, sites without telemedicine will be affected by where the stroke physician is located when the call is received. It might be assumed that, if the stroke physician is located centrally, travel time is reduced across sites. Whether this holds true given the distribution of where stroke patients are treated is unknown.

We hypothesized that deploying telemedicine at a subset of five outlying hospitals in our region could be more cost-effective than deploying telemedicine at all hospitals in the region. We also hypothesized that the proportion of patients who could receive treatment within three hours would be increased. Finally, we expected that onset-to-treatment time would be reduced when the stroke physician was centrally located compared to when the stroke physician was not centrally located.

METHODS

Study Design

This was a computer simulation study using Monte Carlo methodology. The study was funded in part by an unrestricted investigator initiated grant from Genentech, Inc. Genentech played no role in design, data acquisition, simulations or drafting/revision of the manuscript. Since only previously de-identified data and simulation techniques were used, the study was deemed non-Human Subjects research by the University of Cincinnati Institutional Review Board.

Study Protocol

A high-level process map of the stroke care process from stroke onset to rt-PA treatment was first developed (Figure 1). Then, a Monte Carlo simulation of the stroke-team care-delivery system was constructed based on a primary dataset of 121 ischemic stroke patients who were residents of the Greater Cincinnati/Northern Kentucky Region during 2005, had a confirmed symptom onset time, presented within 4.5 hours of onset to a local study ED, and had no contraindications to receiving rt-PA. This region, which is representative of the United States in terms of age distribution, racial composition, level of education, and median household income, includes 17 acute-care hospitals, all served by a single, highly-experienced stroke team that has offered acute stroke treatment and management for over 20 years. The primary dataset was obtained from a population-based, epidemiology study of stroke, the Greater Cincinnati/Northern Kentucky Stroke Study (GCNKSS), which is described in detail previously.⁶ In brief, study nurses and physicians use comprehensive medical record review methodology to collect detailed clinical information for every hospitalized stroke for all residents of the region. We used these data to construct a model to estimate the effects of different operational policies on time-to-treatment within the population. Specifically, we modeled onset-to-treatment (OTT) time, door-to-needle (DTN) time, and the proportion of eligible patients receiving rt-PA within three hours of stroke onset.

Figure 1.

High-level process map from stroke onset to rt-PA administration

SOURCE: Authors’ depiction of normative ischemic stroke care process.

Notes: Sizes of activity blocks are not scaled to represent time durations.

Process Map

The normative process modeled here starts from the time the patient recognizes the stroke has occurred (the recognition time). The patient then either takes a personal vehicle or calls an ambulance to obtain care. If the latter, a dispatch notice is then sent. An emergency medical services (EMS) team travels to the patient location, prepares the patient for transfer, and transports the patient to a nearby hospital. The EMS team may or may not pre-notify the receiving hospital prior to arrival.⁵ After the patient arrives at the hospital, whether by ambulance or personal vehicle, the ED staff perform an initial work-up. If the patient is recognized as a possible stroke patient, ED staff notify the stroke team. In cases where EMS preemptively notifies the hospital, ED staff may notify the stroke team and may facilitate an immediate computed tomography (CT) scan. While the patient may have blood work and a CT scan done, the determination of eligibility and administration of rt-PA begins only when the stroke team physician evaluates the patient. Once the stroke team physician is notified, he or she can travel to the hospital, or set up a telemedicine consultation. For rt-PA-eligible patients, the medicine is either prepared at bedside or through the pharmacy (depending on hospital policy); once it is prepared, treatment may commence.

Sampled and Simulated Patient Populations

Table 1 summarizes the variables that were extracted from the GCNKSS dataset, and the statistical expressions that best describe the data. Arena Input Analyzer (Rockwell Automation, Inc. Milwaukee, WI) was employed to find the probability density function that fits the empirical data best for each variable. In cases where we did not have data, we relied on expert opinion to estimate minima, maxima, and modes of the variables of interest, and then built triangular distributions for those variables (identified by * in Table 1).

Table 1.

Process variables and their best-fit probability density functions

Variable	Probability distribution
Recognition time duration	Weibull (24.6, 0.479)
% Calling an ambulance	88% (deterministic)
Call ambulance duration	Gamma (1.41,1.41)-0.5
EMS patient prep time duration	Weibull (15, 2.09)
Patient handover duration	Gamma (4.31, 1.45)
EMS pre-notification rate5	73% (deterministic)
Workup duration	Weibull (18.4,0.85)
Duration from CT order to reading	Gamma (10, 1.78)
Bedside prep duration*	Triangular (2,5,8)
Pharmacy prep duration*	Triangular (5,10,15)
Additional tests and evaluations*	Triangular (2,5,8)
Telemedicine setup time duration*	Triangular (2,5,8)
Delay in calling stroke team after patient arrival*	Uniform (5,15)
Probability of stroke team at base hospital*	0.42 (10 hrs/day)
ED triage time duration*	Triangular (15,30,45)

^*Probability distributions per expert estimates (no * indicates distribution is based on 2005 GCNKSS data⁶)

CT = computed tomography; EMS = emergency medical services; GCNKSS = Greater Cincinnati/Northern Kentucky Stroke Study

An essential element of this analysis is identifying the location and travel times for both patients and stroke team physicians. For travel-time calculations, ArcGIS (ESRI Redlands, CA) was used to randomly generate hypothetical patient locations throughout the five counties of Hamilton and Clermont in southwest Ohio; and Kenton, Boone, and Campbell in northern Kentucky. One hundred random locations within each of the 92 standard zip codes were generated and identified by latitude and longitude and the nearest street address.

We note that 12 of the Ohio zip codes and five of the Kentucky zip codes cross the boundaries of our five-county region into adjacent counties. The goal of the GCNKSS is to determine population-based incidence of stroke; therefore, it does not include cases from the adjacent counties. Its stroke team, however, is consulted for all potential cases of stroke that present to the region’s hospitals, irrespective of a patient’s residence. Therefore, we included the entire areas of these zip codes for the simulation. Three additional Ohio zip codes that are primarily associated with adjacent counties and have less than 2% of their populations in Hamilton or Clermont counties were not included in the simulation.

Figure 2 shows the map of the geographic sampling frame and the 9,200 simulated patient locations. This pool of 9,200 locations was then used as a sampling frame for both patients and physician locations, with physicians’ locations limited to those zip codes in which they live and work. The Google Maps application program interface was used to generate estimated travel durations for each of the patient and stroke physicians going to each hospital; code was written using Visual Basic for Applications to generate batch routing within Microsoft Excel. Travel time estimates were based on early afternoon weekday traffic densities, representative of “typical” travel times. This decreases the potential for extreme outliers due to rush hour.

Figure 2.

Seventeen hospital locations and 9,200 randomly generated patient locations in the Greater Cincinnati/Northern Kentucky regional sampling frame.

SOURCE: Authors’ data, generated using ArcGIS software.

Notes: Hospital locations and zip code boundaries accurate as of 10/31/2014.

Monte Carlo Simulation

We examined two factors of interest. The first factor was telemedicine availability at various hospitals. Three different deployment policies were compared: 1) no telemedicine in the region; 2) telemedicine in all hospitals throughout the region; 3) telemedicine only in outer-ring hospitals.

The second factor was the location of stroke team physicians while on call. We considered two policies: 1) stroke team physicians were based in their home zip code, and 2) stroke team physicians were located within a 15-minute driving radius of the center of the region. Full-factorial combination of these policies resulted in six distinct scenarios. The performance of each scenario was estimated using a Monte Carlo computer simulation model. The desired margin of error for comparing sample proportions was 0.01, requiring 7,000 simulated observations per scenario.

To ensure reliable analysis of results, variance reduction was used to decrease statistical “noise” (unexplained variance) in the output measure of performance. This noise reduction helps in better capturing the effect of the two operational factors (telemedicine policy and physician location policy). The Monte Carlo simulation of the care process was modeled in Microsoft Excel and the Common Random Generation method was used for variance reduction.⁷

Below is a brief overview of one replication of the simulation model; the process was repeated for each patient until the desired number of replications was achieved for each scenario. Note that all of these steps can be found in the process map shown in Figure 1. All time durations were generated using the expressions shown in Table 1. The model was verified and validated against the 2005 GCNKSS data⁶ and expert opinion.

Stroke onset (Time 0) 1Generate a patient location. Generate a uniform random number and look up corresponding location in the sampling dataset. 2Generate recognition duration. This is the amount of time between onset and the patient recognizing he requires medical attention. 3Determine destination hospital. This was randomly selected from the three closest hospitals, where the chance of being selected was weighted based on proximity to the patient’s recognition location. 4Generate traveling mode: personal vehicle or ambulance. We assumed all patients traveled directly to a hospital and did not seek care elsewhere first. 5Generate travel time from patient location to destination hospital. If patient travels by ambulance, follow steps 6–10: 6Generate “call ambulance” time. 7Indicate ambulance-to-patient travel time assuming ambulance originates from destination hospital. 8Generate EMS on-scene time. 9Generate EMS pre-notification8: Yes/No; if yes, initial workup duration, door-to-imaging time, and time-to-stroke team notification were adjusted to reflect the reduced times resulting from the EMS pre-notification. Identify patient-to-hospital travel time.

ED arrival/Triage: 10If patient is arriving by ambulance, generate patient-handover duration. 11If patient is arriving by personal vehicle, generate triage delay. 12Generate initial workup time. 13Generate the time stamp for when ED staff at destination hospital calls stroke team. 14Generate location of stroke team physician according to the scenario policy. 15Generate mode by which stroke team provides care: telemedicine or traveling to patient’s bedside. If traveling, look up travel time from stroke team location to destination hospital. If using telemedicine, generate telemedicine setup duration. 16Generate CT duration: time from CT order to when preliminary results are available.

Stroke team care begins 17Generate duration for final tests and evaluation. 18Generate rt-PA preparation time; either bedside preparation or pharmacy preparation, per each hospital’s policy. 19Collect time of treatment for patient; calculate the OTT time and DTN time.

Repeat steps (1) to (19) for n patients, where n is the number of replications needed to achieve target margins of error on output measures of performance (in our case, 7,000 patients per scenario).

Return-on-Investment Analysis

The final component of our analysis was to roughly estimate the economic return, in terms of payback period, for a region should it decide to deploy telemedicine at some or all hospitals in the region. We considered two scenarios consistent with the prior model assumptions: 1) partial deployment, where only the outermost five hospitals receive telemedicine, and 2) full deployment, where all hospitals in the region receive telemedicine. Recent cost-effectiveness studies estimate that treatment with rt-PA within three hours of stroke onset results in an average life-time societal savings of $25,000 per patient.⁹ Combining that figure with our model’s output and a range of telemedicine costs from $1,000 per location to $50,000 per location, yielded overall payback curves that indicate how long it is likely to take to recoup the cost of the telemedicine in terms of reductions in stroke-related morbidity and mortality. All technology costs were assumed to occur up front; ongoing maintenance and repair expenditures were assumed to be zero.

Sensitivity Analyses

As a supplement to the above analyses, we considered several important assumptions and examined how the results were affected if these assumptions were incorrect. First, we assessed the effect on the results if the “workup duration” estimate had a distribution that was either longer or shorter than our assumed baseline. Second, we considered what effect a longer telemedicine setup duration might have on the results. Third, we considered both shorter and longer ED triage durations. Fourth, we tested the sensitivity of our results to traffic pace (typical vs. “rush hour”) and ambulance travel mode (normal vs. “lights and sirens”) assumptions. These particular sensitivity analyses were chosen due to the size of their durations and the potential for our baseline values to lack generalizability. They were then compared to the least-restrictive baseline values (i.e., physicians permitted to take call from home).

RESULTS

The numerical results are provided in Table 2. When the stroke team physician is located anywhere randomly in the region, the proportion of the simulated stroke population treated within three hours of symptom onset increases when comparing no telemedicine to partial telemedicine deployment to total telemedicine deployment (70.3% vs. 74.9% vs. 80.0%, respectively). When telemedicine is partially deployed in the region, locating the stroke team physician within the zip code of the hospital near the center of the region resulted in a reduction in OTT time compared to when the physician is located anywhere (157.7 vs. 164.4 minutes).

Table 2.

Performance outcomes of the simulated scenarios

Scenario		% rt-PA within 3 Hours	Onset-to- Treatment Time (min)	Door-to- Needle Time (min)
Stroke Physician Location	Telemedicine Deployment	Percentage (±95% HW)	Mean (±95% HW)	Mean (±95% HW)	Average Annual Cost Savings*
Home	None	69.5 (±1)	175.6 (±2.5)	72.7 (±0.5)	Base case
	Partial	74.4 (±1)	164.4 (±2.5)	61.4 (±0.6)	$114,500
	All	79.7 (±1)	147.5 (±2.5)	44.6 (±0.6)	$242,500
Central	None	73.2 (±1)	168.9 (±2.5)	65.9 (±0.5)	$72,500
	Partial	77.6 (±1)	157.7 (±2.5)	54.8 (±0.5)	$180,750
	All	80.0 (±1)	147.5 (±2.5)	44.6 (±0.6)	$242,500

HW = half-width; AIS = Abbreviated Injury Scale

^*Per 100 AIS patients in the region per year

When all hospitals in the region have telemedicine, the location of the stroke team physician has no effect on OTT time, DTN time, or the proportion of patients treated within three hours. This is because telemedicine eliminates the need for physical travel in most cases. However, if there is no telemedicine in the region or telemedicine has only been partially deployed, locating the stroke team physician centrally significantly improved all three performance metrics.

Partial deployment of telemedicine improved DTN time to a mean of 61.4 minutes, compared to 72.7 minutes with no telemedicine. Having telemedicine at all hospitals improved the DTN time to an average of 45 minutes, well below the 60-minute ASA recommendation.¹⁰

The results of the return on investment analysis are shown in Figure 3. Partial deployment and full deployment, each assuming centrally located stroke team physicians due its better performance as described above, both have payback periods under four years throughout the range of technology costs that were considered. However, the partial deployment strategy yielded a shorter payback period regardless of the cost of the technology. If the physician is not centrally located, the payback period for the partial deployment policy is lengthened but does not change for the full deployment policy.

Figure 3.

Telemedicine payback periods based on cost per site and deployment strategy.

SOURCE: Authors’ calculations based on simulation results

Notes: Annual discount rate assumed to be 3%.

The results of the sensitivity analyses are shown in Tables 3 and 4. None of the deviations from our original assumptions produced changes in the percent-treated metric large enough to alter the conclusions based upon the baseline parameters.

Table 3.

Sensitivity of % treated to probability distribution assumptions

Case	Workup (Weibull)	Telemedicine setup (Triangular)	ED triage (Triangular)	% treated within 3 hours
				None	Partial	All
Base case	(18.4,0.85)	(2,5,8)	(15,30,45)	69.5	74.4	79.7
Case 1	(14.7,0.85)	-	-	71.5	76.3	81.7
Case 2	(23.0,0.85)	-	-	69.1	73.4	78.1
Case 3	-	(4,10,16)	-	70.3	74.	79.8
Case 4	-	-	(10,20,30)	70.7	75.2	80.2
Case 5	-	-	(22.5,45,67.5)	69.5	74.3	79.7

A cell value of - indicates the base case value was used

All % treated values have 95% half-widths of ±1%

Table 4.

Sensitivity of % treated to traffic pace assumptions

Case	Traffic Conditions		% Treated Within 3 Hours
	Lights & Siren*	Rush Hour**	None	Partial	All
Base case	0	0	69.5	74.4	79.7
Case 1	0	1	71.9	76.3	80.9
Case 2	1	1	68.0	73.74	80.0

^*Ambulance travels 12% faster with lights and siren on

^**All traffic moves 10% slower during rush hour

All % treated values have 95% half-widths of ±1%

DISCUSSION

In this study, we simulated the effects of various policies for the deployment of telemedicine and stroke team physician location on 1) the proportion of ischemic stroke patients able to be treated within three hours, 2) symptom onset-to-treatment time, 3) door-to-needle time, and 4) the payback period for telemedicine deployment in a well-characterized population that has been served by a single regional stroke team for decades. We found that the proportion of rt-PA-eligible patients who would be treated within three hours of symptom onset improved with partial telemedicine deployment, and improved further with full telemedicine deployment. Treatment times would also be reduced. When the stroke team physician is located closer to the center of the region, the improvements resulting from partial deployment are even greater. We note that these findings apply to optimization of care for the entire region and do not focus on the performance of isolated hospitals in the region.

Since reducing the time-to-treatment for stroke victims is essential to achieving the best possible patient outcomes,² and the deployment of telemedicine systems is limited by its cost, it is clear that better understanding of how technology and personnel policies influence system performance is needed. We have demonstrated that combining regional clinical data with simulation techniques can identify effective policies for deploying telemedicine (telestroke robots or tablets) and clinical staff (physicians), and we provide evidence that proper deployment of these resources can significantly improve the system’s ability to be responsive to patients’ needs.

We also examined the payback periods associated with deployment of telemedicine by estimating the amount of time that would be needed to recoup the telemedicine expense through improvements in stroke-related societal expenses. Partial telemedicine deployment at the hospitals located most remotely in the region, with the physician located centrally during the call period, was found to be most cost-effective. Full deployment was estimated to provide better care in terms of shortest treatment times and highest proportion treated, but that investment policy suffers from diminishing returns. Deploying telemedicine at the hospitals furthest away from the central hospital eliminated the longest travel times first, so deployments at closer-in hospitals provided comparatively smaller improvements in system performance. However, given the overt benefit of reduced DTN times with telemedicine deployed everywhere, efforts to markedly reduce the current costs of telemedicine have the potential to improve the outcomes of stroke patients by increasing the proportion of rt-PA treated patients who are treated quickly.

Two prior studies have used decision-analytic models to estimate the cost-effectiveness of telestroke networks compared with usual care.^5,11 Our findings complement these prior studies and add an estimate of payback periods depending on physician staffing and telemedicine deployment policies.

LIMITATIONS

Limitations of our study include the inability to generalize our findings to other regional stroke teams. While others have examined the effectiveness of telemedicine technology and the challenges in its implementation, and have offered recommendations on how to embed this technology into practice,^12–14 the regional differences in systems of stroke care suggest there is no prescriptive generalizable framework for designing a universal telestroke network. However, lessons may be gained from our system where we are able to model how best to care for all stroke patients within a population cared for by a single team of physicians. Moreover, our approach may be easily customized to other health systems.

Another limitation is that our process map cannot capture all possible steps and potential process variations within acute stroke care. However, we used a high-level map that is generally representative of the steps typically involved in the acute care process leading up to rt-PA administration, and we incorporated variation into estimated process times to reflect the variation in processes that occur at the patient level. Finally, our model made the simplifying assumption that all telemedicine costs were incurred up front and that ongoing maintenance and repair expenditures would be zero. We recognize this is unlikely, but we posit that, due to the assumption of a non-zero discounting factor, spreading the costs over time would only improve the return on investment by reducing the payback period.

The datedness of our primary dataset is also a limitation. However, we believe the use of the primary dataset to estimate the variables in Table 1 coupled with the sensitivity analyses in Tables 3 and 4 fairly reflects the potential variations in practice between 2005 and the present.

Last, our goal was to estimate the cost-effectiveness and impact of various policies of telemedicine deployment on time to treatment for all potentially eligible stroke patients within the population. Countless additional sensitivity analyses may be performed targeting questions specific to a certain region or health system. For instance, telemedicine consultations have previously been reported to result in more accurate decision-making than telephone consultations.¹⁵ As such, systems that still rely primarily on telephone consultations may consider the cost-effectiveness of switching to telemedicine using these data and our model.

CONCLUSIONS

Deploying telemedicine at all regional hospitals resulted in the highest rates of rt-PA treatment within three hours, and the shortest time to treatment. However, central location of the on-call stroke team physician coupled with partial telemedicine deployment was the most cost-effective strategy. Given the potential societal benefit, efforts to markedly reduce the upfront and sustaining costs of telemedicine deployment are warranted, and would mitigate having to specify the physical locations for on-call stroke team physicians. Aligning the incentives between those who would have to fund the up-front technology investments and those who will benefit over time from reduced ongoing health care expenses will be necessary to fully realize the benefits of telemedicine technology for stroke care.