Decreased length of stay after addition of healthcare provider in emergency department triage: a comparison between computer-simulated and real-world interventions

Abstract

Objective

(1) To determine the effects of adding a provider in triage on average length of stay (LOS) and proportion of patients with >6 h LOS. (2) To assess the accuracy of computer simulation in predicting the magnitude of such effects on these metrics.

Methods

A group-level quasi-experimental trial comparing the St. Louis Veterans Affairs Medical Center emergency department (1) before intervention, (2) after institution of provider in triage, and discrete event simulation (DES) models of similar (3) ‘before’ and (4) ‘after’ conditions. The outcome measures were daily mean LOS and percentage of patients with LOS >6 h.

Results

The DES-modelled intervention predicted a decrease in the %6-hour LOS from 19.0% to 13.1%, and a drop in the daily mean LOS from 249 to 200 min (p<0.0001). Following (actual) intervention, the number of patients with LOS >6 h decreased from 19.9% to 14.3% (p<0.0001), with the daily mean LOS decreasing from 247 to 210 min (p<0.0001).

Conclusion

Physician and mid-level provider coverage at triage significantly reduced emergency department LOS in this setting. DES accurately predicted the magnitude of this effect. These results suggest further work in the generalisability of triage providers and in the utility of DES for predicting quantitative effects of process changes.

Background

The United States maintains a national network of hospitals and clinics, using a single-payer, single-employer model, dedicated to providing care to former members of the United States Armed Forces. Collectively, this network is known as the Veterans Health Administration (VHA), which is a subsidiary of the cabinet-level Department of Veterans Affairs. In these respects, the VHA bears some resemblance to the British National Health Service and other nationalised healthcare systems in social democracies. It is common in these systems to provide performance measures, such as the National Health Service's 4-hour rule, as a large-scale means of standardised care.

Given that long emergency department (ED) stays are known to be associated with increased risks of adverse events,¹ the VHA holds a national performance measure that no more than 10% of patients should spend >6 h in the ED from arrival to disposition (discharge, transfer or admission). Moreover, waiting times are a major reason for patients leaving without being seen (LWBS) by a healthcare provider^{2 3} and, likewise, improvements in waiting times are associated with decreases in LWBS.^{4 5} The preponderance of evidence suggests that patients who leave without being seen are similar with regard to illness and acuity to patients who remain for care and, therefore, may have similar risk of adverse events after leaving.^{6 7} Case reports of front-end interventions to improve ED length of stay (LOS) and LWBS have demonstrated success.⁸ However, the generalisability of such interventions remains in question.

Most interventions to reduce ED LOS involve costly process interventions—for example, changes in staffing, facilities, computer or laboratory systems, and so on. Moreover, successful interventions generally require buy-in from clinical staff; as such, a failed intervention may diminish staff morale and might lower the likelihood of success for future interventions. A rapid, accurate, low-cost, low-risk means of testing and evaluating such interventions would be highly advantageous.

Discrete event simulation (DES) is a technology that facilitates analysis of non-linear interactions between variables and their intermediary agents and is therefore highly suited for the complex, dynamic system of ED patient flow. Such models are useful in guiding process interventions in a low-cost, minimal-risk manner. However, published reports of DES in the ED are essentially case reports; it is not clear how changes in metrics predicted by DES compare quantitatively with real-world interventions.

Computer simulation of clinical ED patient flow was first proposed in the economics literature as early as 1975,⁹ but was not described in the medical literature until 1989,¹⁰ without further mention until Coats and Michalis introduced a proof-of-concept model in 2001.¹¹ Since that time, DES has been described as a means of analysing patient flow,^12–14 predicting demand for services and addressing the related problems of crowding,^15–17 inpatient boarding^{18 19} and evaluating various other interventions in patients requiring emergency services.^{20 21} However, the authors know of no published studies reporting the comparison of real-world implementation with the results predicted a priori by simulation in the ED.

This study represents an attempt to assess reduction in the daily mean LOS, and the proportion of patients with LOS >6 h, by employing a triage intervention consisting of adding a physician and mid-level provider (currently a nurse practitioner, but potentially also a physician's assistant) in triage, consolidating Fast Track into triage and discharging low-acuity patients directly from triage whenever possible. Fast Track is a separate, dedicated portion of the ED designed to handle lower-acuity patients. Because of the restricted nature of this cohort, it is generally possible to maintain a higher patient to nurse ratio and have the primary provider as a mid-level provider (in the case of St. Louis Veterans Affairs Medical Center (VAMC), a nurse practitioner). Acuity is defined according to the Emergency Severity Index (ESI), which sorts patients by both severity and expected resources needed to complete the visit. ESI 1 represents patients in need of resuscitation, ESI 2 represents patients in dire emergent condition, and ESI 3–5 represent patients in urgent to non-urgent condition and requiring higher to lower numbers of resources to treat. ESI levels and procedures are defined in the Agency for Healthcare Research and Quality's Emergency Severity Index Implementation Handbook.²² The intervention was devised by ARA-R based on personal communication and collaboration with other Veterans Administration ED directors. Additionally, we aim to determine how accurately changes in clinically useful throughput metrics are predicted by the DES model, by comparing simulation-based prediction with the results of a real-world trial of the proposed intervention.

Methods

Design

The design of the study was to develop and validate a DES of the ED, and to use this tool to predict the effectiveness of the proposed change to triage services, followed by a real-world implementation of the strategy. The pre-intervention triage system is denoted below as the ‘control’, and the post-intervention triage system is denoted as the ‘test’. The computer simulation was first modelled and then validated against the real-world control; then it was used to predict the test. Then, upon generation of a favourable prediction, the triage intervention was adopted in the real-world ED. Finally, post-hoc analysis of the result from the real-world test was compared with the simulated prediction, to determine precisely how accurate the simulation was in predicting the consequences of adopting the intervention. Thus, there are four total instantiations of the study: computer simulation before and after and the real-world system, before and after.

Setting

The St. Louis VAMC is a Level 3 Trauma Centre with 120 inpatient beds. Level 3 Trauma Centres are characterised by having less than full specialist availability, but full resources for resuscitation, and ICU. The ED includes 14 patient beds, of which two are dedicated for mental health patients, and receives approximately 20 000 patient visits annually. The study intervention was approved by hospital administration and declared exempt by the Institutional Review Board of the St. Louis VAMC. The intervention was implemented in the real-world ED on 19 September 2010 for a trial period of 1 month, until 18 October 2010. Data collected for this study represent this trial period. Comparison data were drawn from 19 September 2009 to 30 October 2009, to be used for pre-intervention validation data, which are, respectively, the ‘trial period’ and the ‘test period’. With the exception of ARA-R, providers were unaware that intervention results would be compared with the simulated prediction.

Control condition (pre-intervention)

Prior to the intervention under study, data were collected from daily aggregated patient encounter sheets for all patients presenting for care between 19 September 2009 and 30 October 2009—hereafter referred to as the ‘control period’. During this trial period, patients presenting to the ED between 08:00 and 16:00 were seen first by a ‘pre-triage’ nurse responsible for directing patients in urgent or emergent need directly to the ED treatment area; others, with less urgent needs, were directed to resources outside of the ED (eg, pharmacy, primary care or other clinics). Outside of these hours, all patients presented directly to triage.

The remaining patients were then directed to a triage room, where they were interviewed and assessed by a dedicated triage nurse. As a result of this evaluation, the triage nurse assigned an ESI and directed the patient to the main ED (ESI 3 or sometimes 4), Fast Track (ESI 4 and 5) or discharged the patient directly from triage (a small subset of ESI 5). Those with ESI 1 and 2 were directed to the ED treatment area, effectively bypassing triage. Whether a patient with ESI 4 is directed to the ED or to Fast Track was determined by the triage nurse's assessment. Similarly, whether a patient with ESI 5 was dispositioned from triage or sent to Fast Track was determined by the triage nurse (figure 1,).

Figure 1.

Process flowchart for emergency department (ED) patients, from arrival to disposition—pre-intervention conditions. Decision points are depicted as diamonds; percentages indicate the percentage of patients assigned to each pathway, based on staff discretion or Emergency Severity Index (ESI) (in the real ED) or random assignment (in the discrete event simulation model).

The main ED treatment area was then reserved for care of patients of moderate to high acuity (ESI 1–3) and included standard ED treatment, including laboratories, medications, full radiological services and other services typical of a Level 3 Trauma Centre (figure 3). This area was staffed by four attending physicians, in addition to the lone mid-level provider dedicated to the Fast Track area.

Figure 3.

Common elements of patient flow (‘ED Treatment Bed’), pre- and post-intervention conditions.

Test condition (post-intervention)

Following the intervention, patients were seen first by the pre-triage nurse from 08:00 to 16:00 and presented directly to triage at other times, in a similar manner. Patients determined to be ESI 1 or 2 at this stage are brought directly to an ED bed, bypassing the full triage protocol. Patients continuing on to triage were registered and then divided between two providers, both present at triage: a physician, who evaluates higher-acuity patients (ie, ESI 3), and a mid-level provider, who evaluates lower-acuity patients (ESI 4 and 5). These providers communicate as necessary and may reassign patients to one another during this triage process.

All ESI 4 and ESI 5 patients are treated and discharged from the triage area, rather than being assigned to a separate ‘Fast Track’. Any patients requiring services that cannot be provided by providers at triage are assigned ED beds, and may be reassigned a lower ESI. As many ESI 3 patients as possible (ie, all those who can safely and comfortably wait in the waiting room rather than an ED bed) were also evaluated entirely from the triage area, rather than being assigned to a separate ‘Fast Track’. The remaining high-acuity patients were transferred promptly to ED treatment beds for further care (figure 2). As a result of this change, Fast Track was eliminated as a separate treatment path and consolidated with triage, as was the mid-level provider formerly assigned to those patients.

Figure 2.

Process flowchart for emergency department (ED) patients—post-intervention conditions. Decision points are depicted as diamonds; percentages indicate the percentage of patients assigned to each pathway, based on staff discretion or Emergency Severity Index (ESI) (in the real ED) or random assignment (in the discrete event simulation model). MD, attending physician; MLP, mid-level provider (nurse practitioner); RN, nursing staff assigned to the triage area.

Further care in the ED treatment area was performed as before (figure 3), with the exception that only three attending physicians were assigned to this area.

Simulations

As described above, a computer simulation was developed to model the ED as it existed prior to any intervention (‘control’). Then, the simulation was updated with the proposed intervention (‘test’), to determine the consequences of the proposed changes. The triage processes were mapped by study investigators, in flowcharts representing both pre-intervention (figure 1) and post-intervention (figure 2) conditions, as well as the internal ED processes common to each of them (figure 3). Probability frequencies for decision trees were calculated based on data collected from actual patient visits during the test period. Process turnaround times for each step in the model were abstracted from daily ED status reports in the case of processes estimable from those (admission/discharge rates, ESI distributions, arrival rates), and for smaller processes (provider time with patient, nurse time with patient) estimated from interviews with experienced staff members responsible for those processes.

Simulations of this event sequence were created as DES models using commercially available software for this purpose (AnyLogic Professional 6.4). Two DES models were created, prior to actual intervention in the real world, one simulating the ‘pre-Intervention’ flowcharts (figures 1 and 3) and the other simulating the ‘post-Intervention’ flowcharts (figures 2 and 3). Simulations were constructed according to a standard four-step process:

1. The system was decomposed into its constituent elements: entities (patients); resources (physicians, nurses, etc.); locations (exam rooms, triage rooms, etc.); and path networks (hallways).

2. The flow was mapped according to the elements identified in step 1.

3. The system was integrated into the AnyLogic 6.4 architecture, detailing how entities consume resources at locations, and then proceed to the next location according to the flowcharts. Proportional likelihoods of following particular paths at decision trees were assigned according to probability frequencies described above.

4. The system underwent face validation with members of the ED staff, who confirmed that processes were accurately mimicked by the simulation, and was then validated against a test data set as follows: The pre-intervention simulation was run for a 6-week period, then repeated for a total of 10 instantiations. LOS data so obtained were averaged between these 10 runs. The post-intervention simulation was also run for ten 6-week runs and the data were averaged similarly.

Data analysis

Similar data from the real-world ED were collected both prior to and pursuant to implementation of the change in triage process. Pre-intervention data were collected from all patient visits during the test period. Post-intervention data were collected from all patient visits during the trial period. Comparisons between mean daily LOS were performed with unpaired, two-tailed Student t tests; comparisons between %6-hour LOS were performed with two-proportion Z tests.

Results

During the pre-intervention period, the real-world ED saw 2194 patient visits over a 6-week period. Daily mean throughput time (arrival to disposition) was 247 min (SD 39.8), with 437 (19.9%) visits taking >6 h. The pre-intervention simulation run saw 2178 patient visits over a similar time period. Simulated mean throughput time was 249 min (SD 39.7), with 413 (19.0%) visits taking >6 h. There was no statistically significant difference between mean LOS (p=0.694) or %6-hour LOS (p=0.909), between the real-world situation and the simulation.

The post-intervention DES model reported 2154 patient visits over a 6-week period, with a daily mean throughput time of 200 min (SD 19.0). Of these, 282 visits took >6 h (13.1%), for a relative reduction of 31.1% compared with the pre-intervention simulation.

During the post-intervention period, the real-world ED saw 1699 patient visits over a 1-month period. Daily mean throughput time was 210 min (SD 16.6). Mean throughput time was significantly different between pre- and post-intervention samples (p<0.0001). It should be noted that the decrease in patients is due to the real-world intervention trial period being 30 days, rather than the 42 days of the other periods, and not a decrease in daily census. During the post-intervention period, there were 243 (14.3%) visits that took >6 h, which represents a relative reduction of 28.2% compared with the pre-intervention, real-world ED (p=0.045). Five hundred and seventy-seven patients with acuity ESI 4 or ESI 5 were discharged directly from triage under the new flow model, rather than being sent to ‘Fast Track’ under the pre-intervention model.

There was no statistically significant difference between the post-intervention states in the simulated and real-world mean LOS (p=0.499) and %6-hour LOS (p=0.880). All mean daily LOS values were verified to fit normal distributions using STAT::FIT (Geer Mountain Software Corporation, South Kent, Connecticut, USA).

Discussion

The reassignment of a physician and nurse practitioner to triage, coupled with the consolidation of Fast Track into triage, appears to have been effective in reducing the ED patient turnaround time in two ways. First, provider coverage allowed for a large percentage of patients to be discharged directly from triage. Because provider coverage was accessible earlier in the emergency visit process, some treatment decisions could be made earlier in the patient encounter, resulting in shorter stays for some patients. This was critical in decreasing the number of visits that took >6 h; previously, some patients would experience prolonged delays, while higher-acuity patients were treated first. Second, by reducing the number of patients requiring beds in the ED, more resources were available for those patients who did require such beds.

It has been suggested, but not easily demonstrated, that provider coverage in triage²³ and discharge from triage²⁴ have the potential to improve the ED throughput. DES was instrumental in predicting the consequences of such interventions prior to implementation, and such predictions were confirmed to be highly accurate, when compared with the actual ED.

DES holds promise as a low-cost, low-risk method for evaluating clinical process changes in the ED. While numerous case studies exist demonstrating qualitatively that hypotheses generated by DES models can yield effective interventions, it is unclear how accurate these models are in this regard and, therefore, how trustworthy such models are in ranking the effectiveness of such process changes. This study shows that, in our institution, a DES model was highly accurate in predicting the decrease in mean LOS and the percentage of patients with LOS >6 h.

While DES cannot provide a single optimised state of a department, it is extremely versatile at testing hypotheses related to process change. This study demonstrates the potential for DES in testing and accurately predicting the results from a complex intervention involving changes in staffing (adding a physician and mid-level provider to triage), upstream process (changing the underlying criteria by which patients will be seen in triage) and downstream process (changing the treatment location of patients following triage).

DES has a long history of use in medical systems for the analysis and prediction of changes to system dynamics and policy. However, outcomes of implementations based upon simulated predictions are rarely reported in the medical literature. The simulation was able, in this case, to provide valuable insight into the likely outcome of the proposed triage intervention, and the intervention was adopted in large part due to the simulated results. One perceived strength of the simulation was its capacity for graphical representation of the ED, thereby allowing stakeholders unfamiliar with simulation methods (ie, ED administration) to demonstrate the viability of the intervention. The outcome of the intervention was to significantly improve system performance and was consistent with the predicted results.

Our validation methods—that is, comparing LOS and %6-hour LOS data between the DES model and the real-world data—represents an additional degree of robustness compared with LOS validation techniques which are seen in the literature.²⁵ Accurately capturing the %6-hour LOS provides a reflection of the high end of the distribution of patient times, rather than restricting the validation to aggregated averages over time.

Limitations

As with any simulation, it is impossible to capture every potential occurrence in the ED; therefore, rather than attempting to incorporate every possible variation in care, the flow is designed to represent the general practice of emergency care in the St. Louis VAMC ED.

It should be noted that these results are based on limited real-world sample sizes. The reference data set represents only a 6-week period, during which there were no extraordinary or unexpected demands on the ED facility or seasonal variation in patient arrivals. We could not necessarily guarantee such a high degree of predictability over a longer, or more heterogeneous, period of time.

Individual process times (eg, patient time with a physician in an exam room) were estimated based on interviews with ED staff and modelled with triangular distributions. While this is common practice, it does allow the introduction of recall bias. Nevertheless, the similarity of results in the real-world and simulated data suggests that this bias was insignificant to the present study. It is possible that this bias introduces a similar systematic error in pre- and post-intervention simulations and, therefore, may cancel out when comparing between pre- and post-interventional conditions.

Protected health information, such as patient identity and demographics (age, sex, ethnicity and gender), was not available. As such, it was not possible to assess the effects of patient characteristics on throughput data. The accuracy of the DES model before and after intervention suggests that patient characteristics other than illness severity (eg, ESI) are unlikely to influence results.

The duration of the simulated period, as well as pre- and post-intervention control period, was determined as a quality assurance project. As a result, there was a discrepancy between the duration of the predictions made in simulation and the real-world trial period. The authors recommend that simulation-tested interventions be implemented under the same conditions as those simulated, and for the same length of time.

Conclusions

To improve healthcare delivery in the ED of the St. Louis VAMC, we designed, simulated and adopted an intervention consisting of adding a physician and mid-level provider in triage, absorbing Fast Track into triage and discharging low-acuity patients directly from triage whenever possible. This resulted in a mean reduction in LOS by 15%, with a 28.2% reduction in patients with LOS >6 h. Our DES models were effective in quantitatively predicting these results. Further work is suggested to demonstrate (1) the applicability of such changes in patient flow in other EDs and (2) that DES is similarly accurate in predicting the results of other process changes, in this and other EDs. Future work is indicated to employ DES in the capacity of identification of crowding causes and the testing of proposed interventions designed to ameliorate those factors.