Abstract
Study Objective:
To evaluate if in situ (on site) simulation training is associated with increased telemedicine use for patients presenting to rural emergency departments (EDs) with severe sepsis and septic shock. Secondarily, to evaluate the association between simulation training and telehealth with acute sepsis bundle (SEP-1) compliance and mortality.
Methods:
This was a quasi-experimental study of patients presenting to two rural EDs with severe sepsis and/or septic shock before and after rollout of in situ simulation training that included education on sepsis management and the use of telehealth. Unadjusted and adjusted analyses were conducted to describe the association of simulation training with sepsis process of care markers and with mortality.
Results:
The study included 1753 patients, from two rural EDs, 629 presented pre-training and 1124 presented post-training. There were no differences in patient characteristics between the two groups. Compliance with several SEP-1 bundle components improved post-training: antibiotics within 3 hours, IV fluid administration, repeat lactic acid assessment, and vasopressor administration. The use of telemedicine increased from 2% to 5% post-training. Use of telemedicine was associated with increases in repeat lactic acid assessment and reassessment for septic shock. We did not demonstrate an improvement in mortality across either of the two group comparisons.
Conclusion:
We demonstrate an association between simulation and improved care delivery. Implementing an in situ simulation curriculum in rural EDs was associated with a small increase in the use of telemedicine and improvements in sepsis process of care markers, but did not demonstrate improvement in mortality. The small increase in telemedicine limited conclusions on its impact.
INTRODUCTION:
Background
Rural hospitals experience unique difficulties with effective and accurate application of evidence-based patient safety solutions.1–4 Patient safety concerns arising from the identification and management of acute time-sensitive critical conditions, such as severe sepsis and septic shock, in rural Emergency Departments (ED) are especially concerning given the high probability of harm or death when errors occur. Sepsis is a major healthcare problem worldwide, affecting over 750,000 Americans each year, and the incidence continues to rise.5, 6 Once sepsis evolves to severe sepsis and septic shock, mortality increases to 50–60%.5, 6 Sepsis is one of the most challenging, complex, and time-sensitive disease states clinicians will encounter. Well-established, evidence-based guidelines for identification and safe management of acute, time-sensitive critical conditions in the ED, such as acute sepsis and the acute sepsis bundle (SEP-1),7, 8 have been developed in academic centers. There remain significant disparities in effective implementation and application of these guidelines and outcomes in rural EDs.2, 3, 9, 10
New technologies, such as telemedicine, provide a unique approach for delivering consistent, safe care to patients across settings: from large academic to small rural hospitals.11, 12 Telemedicine offers many benefits: improved access for both patients and physicians, cost efficiencies, and improved quality and safety.13 Telemedicine programs vary widely, but typically include audio, video, and electronic communications with a remote, centralized support center where nurses and physicians provide monitoring and consult support. Telemedicine use has recently grown in light of the COVID-19 pandemic, but at present, telemedicine is most commonly used and studied in hospitals in the electronic intensive care unit (eICU) environment; and eICU has been shown to improve compliance with care protocols, improve quality and patient safety, decrease hospital and ICU length of stay, and reduce patient mortality.14–18 Specifically, when implemented with a care bundle, eICU has been shown to reduce adjusted mortality by 29.5%, relative to a control population,15 and substantially reduce medication errors in a pediatric population by 7%.19 ED telemedicine implementation has been studied on a limited basis,11, 20 with evidence supporting implementation for tele-psychiatry consultation,21 and tele-stroke care.22
In theory, ED telemedicine implementation for sepsis could improve care in a number of ways. The incorporation of an eICU into the care team allows for additional checks for compliance with quality measures in real time: a collegial reminder function. If video teleconferencing is used then the eICU nurse becomes part of the monitoring process for patient deterioration, and a communication conduit to the eICU medical staff. This eICU nurse could also help mentor junior nurses at the bedside who may have less critical care experience. If patients are placed into an eICU monitoring system, then automated monitoring algorithms might also monitor patient physiology to augment the alert system.
Importance
Addressing patient safety in rural hospitals is a critical issue that affects millions of Americans across the country. Over 51 million Americans live in rural areas23 and are served by 1800 rural hospitals across the country.24 Similarly, sepsis is a highly prevalent and high mortality disease state. Implementation of telemedicine may improve rural ED sepsis care delivery, but is a complex process with multiple factors impacting adoption and sustainability.20, 25 Implementation of telemedicine in the ED is particularly complex, given the constant changes in patient volume, types of conditions, and variability in clinician expertise and staffing. In situ simulation, simulation that takes place in the live, operational, clinical setting with clinicians performing their real roles, but prior to actual implementation of the process in real clinical care, may provide the ability to train and evaluate the integration of new technology, such as telemedicine, along with care guidelines, such as the SEP-1 bundle, into clinical practice and workflow.
Goals of This Investigation
The primary objective was to evaluate if care team training using in situ simulation would be associated with increased telemedicine use for patients presenting to two rural EDs with severe sepsis and septic shock. The secondary objective was to evaluate if in situ simulation training would be associated with improved compliance with components of the SEP-1 bundle and sepsis mortality. This study was conceived, executed, and analyzed prior to the COVID-19 pandemic.
METHODS:
Study Design, Setting, and Intervention
This study is a quasi-experimental study, with an educational intervention of in situ simulation, and collection of process and outcomes data for all adult patients with severe sepsis or septic shock (by Centers for Medicare and Medicaid Services (CMS) SEP-1 definitions) presenting to two rural EDs between April 1, 2015 and March 31, 2019. The educational intervention aimed to improve care of patients with severe sepsis and septic shock. This study was part of a larger hospital-system effort to promote sepsis care and telemedicine: both sites had telemedicine capabilities, including tele-stroke and tele-psych, and received standard education on the new telemedicine service focusing on sepsis care, in addition to the educational intervention studied here.
The study took place in ED-A and ED-B. ED-A is a 42-bed health care facility in rural Illinois, with approximately 14,000 ED visits per year, approximately 18 patients with sepsis per month. ED-B is a 99-bed acute care hospital, also in rural Illinois, with approximately 20,000 ED visits per year and approximately 20 patients with sepsis every month. Both EDs are staffed by one ED physician at a time and have limited on-site critical care capabilities, but each have ICUs. The two rural hospitals are part of a large 15-hospital healthcare system with a central academic tertiary care referral system that operates a telemedicine (eICU) program to support the healthcare system.
The eICU is staffed around-the-clock by eICU nurses with intensivist back-up. Calls and telemedicine interactions are handled primarily by the eICU nurses. The eICU nurses are able to monitor patients in the rural EDs through the electronic medical record system and a “sepsis hospital” monitoring board, even prior to being consulted by the treating physician. Prior to the intervention at both sites, communication between the ED and eICU nurse could occur (initiated by either party), and ED physicians could call the intensivists for admission. For critically ill patients needing transfer to a hospital with increased critical care capabilities, during daytime hours the transferring ED physician would speak with the transfer center at the receiving institution which created a phone connection to the receiving ICU team, and after 7pm the ED physician would speak to an intensivist in the eICU. The smaller sites transfer patients in need of mechanical ventilation primarily to one of two ICUs within the system, also with eICU oversight, and this was the case prior to and during the study period. Prior to the intervention, both sites had experience using telemedicine carts for tele-psychiatry, which had sunsetted due to increased local access to psychiatric case managers, and tele-stroke which had active ongoing use.
For the telemedicine-sepsis intervention, the ED team was to engage the eICU team via a telemedicine cart that contained the camera, microphone, and necessary software. At any team member’s discretion and with verbal consent if the patient or family were able to consent, a team member would roll the cart into the patient’s room and initiate the connection to the eICU. eICU team members were also in-serviced on the process and could also initiate a request for connection. The team was encouraged to use the telemedicine support for any cases they felt were very ill, or in cases where the additional monitoring of the patient would be helpful to them, but the decision to use telemedicine support was at the discretion of the treating physician or nurse. Alternatively, the bedside nurse and eICU nurse might simply have conventional phone conversations to discuss care issues and consider additional telemedicine engagement.
The educational intervention has been described in a previous publication.26 Briefly, an in situ simulation curriculum was created and deployed in the two rural EDs with the goal of improving (1) sepsis knowledge, including familiarity of the CMS SEP-1 measure, and (2) initiation and use of telemedicine in the care of patients with severe sepsis and septic shock. The education included pre-simulation online education modules focusing on sepsis education and a 45 minute in situ simulation training event. The in situ simulation centered on a patient presenting with sepsis and evolving to septic shock, who would benefit from the use of telemedicine. The focus then turned to how to use the telemedicine technology. As we anticipated that the bedside nurse would be the primary person to interact with the telemedicine equipment, we focused on nursing participation. On-site training took approximately one month to complete at each ED over approximately four days per ED. The intervention of in situ simulation training was rolled out at different dates with an intentional time lag between the sites, ED-A completed simulation training in April 2016, ED-B completed simulation training in April 2017. Thus, the numbers of patients’ pre-post intervention differed between the sites. Training was considered complete once 80% of nurses completed the in situ simulation curriculum.
Selection of Participants
Patients were selected for inclusion in this study if they were over the age of 18 and presented to either ED-A or ED-B with severe sepsis or septic shock. Patients were first identified by a pre-existing hospital system process improvement dataset with the following inclusion criteria: (1) lactate level >2.5, (2) systemic inflammatory response syndrome (SIRS) and a low blood pressure reading while in the ED, and/or (3) left the ED with a Diagnosis Related Group (DRG) consistent with severe sepsis or septic shock (DRG 870, 871, 872). Each chart was then reviewed by the study team to ensure the patient had evidence of severe sepsis or septic shock in the ED by the CMS SEP-1 definitions. The study was IRB approved with a waiver of informed consent.
Methods of Measurement and Data Collection
Data was first obtained through an automated electronic medical record (EMR) query. Automated data points were then reviewed by research assistants (RAs) and additional EMR chart review and data collection was completed for data points not collected in the EMR, such as telemedicine use. Key data points, such as classification as severe sepsis or septic shock, and outcome variables, such as sepsis bundle component completion and mortality were all verified by chart review. The research nurse completed an independent quality assurance review of classification and outcome variables for 10% of cases, randomly selected. Reviewers were not blinded in any fashion as risk for bias during chart review for objective data process markers was felt to be low. All discrepancies from the quality assurance review, as well as any unclear cases for the RAs on initial review, were brought to the lead investigators for adjudication and consensus agreement.
Outcome Measures
Key process outcomes included telemedicine use (including both telemedicine cart use and telephone calls), compliance with SEP-1 sepsis bundle components including lactic acid completion, blood culture completion, antibiotics within 3 hours, intravenous (IV) fluid repletion for hypotension or lactic acid >4, repeat lactic acid completion for lactic acid >2, vasopressor administration for septic shock, and reassessment for septic shock. The key sepsis outcome measure was inpatient mortality and 30-day mortality.
Data Analysis
Cases were classified as pre-training and post-training depending on if they presented before or after the in situ simulation training program was completed in the particular ED. The study was powered to detect a 10% increase in SEP-1 compliance post-training, with focus on time to antibiotics, and a 10% increase in telemedicine use from use in 0% of cases prior to initiation of the telemedicine sepsis program to 10% of sepsis cases post-training. We anticipated that clinicians would not need to use telemedicine in all cases, but only the most sick and complex, hence the 10% goal. While mortality is also a reported outcome, we were unable to power the study to detect a significant mortality difference given low patient volumes in these rural EDs.
Chi-square test or Fischer’s exact test were used to describe univariate association of simulation training (pre- vs. post-) with demographics, primary and secondary outcome variables, and mortality. Adjusted analysis were performed using logistic regression, with adjustment for hospital characteristics- facility and if the patient was transferred to an outside facility or not- and patient characteristics- age, sex, and severity of illness (SOFA score)- and pre- vs post-training. Analyses were performed for both clinical outcomes (mortality) and process outcomes (telemedicine use and SEP-1 compliance). Adjusted associations were reported as odds ratios (OR) with 95% confidence intervals (95% CI). Additional secondary analyses- both unadjusted and adjusted- were completed to evaluate the impact of telemedicine and for this analysis, cases were classified as including a telemedicine activation (including either telemedicine cart use or telephone calls) or not. We considered using interrupted time series analysis to evaluate changes in telemedicine use over time, but unfortunately with low rural ED census, the number of cases did not allow for this methodology.
RESULTS
In total, 2421 patients were assessed for study inclusion and 748 were excluded because on chart review there was no evidence of severe sepsis or septic shock in the ED by SEP-1 definitions (Figure 1). The overall sample included 1753 patients, 1255 with severe sepsis and 498 with septic shock, 834 from ED-A and 919 from ED-B. Within the patient sample, 629 presented prior to the ED staff undergoing training in sepsis and telemedicine (pre-training) and 1124 presented to the ED after training was completed (post-training). There were no differences in patient characteristics between the two groups, pre- vs. post-training, (Table 1). Severity of illness was similar as well with a SOFA score of 3.0 across the two groups. Characteristics of patients for whom a telemedicine consultation was utilized in the ED vs. not were also the same. (Table 2).
Figure 1.
Patient Inclusion/Exclusion
Table 1. Unadjusted Analysis.
Unadjusted univariate associations of patient characteristics, process outcomes, and primary mortality outcomes by pre vs post- telehealth in-situ simulation sepsis training program
| Total Sample N = 1753 | Overall N = 1753 Mean (SD)/ Median (IQR) or % |
Pre-training N = 629 Mean/Median or % |
Post-training N = 1124 Mean/Median or % |
p-value |
|---|---|---|---|---|
| Facility: | ||||
| A (N = 834) | 834 (47.6%) | 174 | 660 | <0.01 |
| B (N= 919) | 919 (54.2%) | 455 | 464 | |
| Transfer for admission (yes vs no) | 317 (18.1%) | 17.2% | 18.7% | 0.35 |
| Patient Characteristics: | ||||
| Age | 69.7 (15.3) | 68.7 | 70.2 | 0.05 |
| Female | 48.3% | 51.4% | 46.6% | 0.06 |
| Infectious Source | ||||
| Urine | 32.5% | 31.3% | 33.2% | 0.42 |
| Lung | 38.5% | 38.0% | 38.8% | 0.74 |
| Intra-abdominal | 10.2% | 9.7% | 10.3% | 0.76 |
| Unknown | 14.4% | 15.3% | 13.9% | 0.43 |
| SOFA Score | 3.0 (1.0, 5.0) | 3.0 | 3.0 | 0.98 |
| Primary Process of Care Outcomes: | ||||
| Telehealth Use | 3.9% | 2.1% | 5.0% | <0.01 |
| Sepsis Bundle | ||||
| Lactic Acid | 96.6% | 96.2% | 96.8% | 0.50 |
| Blood Cultures | 90.1% | 89.4% | 90.6% | 0.41 |
| Antibiotics | 79.5% | 74.6% | 82.2% | <0.01 |
| IV Fluid* (N= 516) | 50.6% | 40.7% | 55.8% | <0.01 |
| Repeat Lactic Acid* (N=1192) | 64.4% | 36.3% | 79.1% | <0.01 |
| Vasopressors* (N=295) | 55.6% | 46.9% | 59.8% | 0.04 |
| Reassessment for Septic Shock* (N=488) | 2.1% | 0.6% | 0.8% | 0.17 |
| Primary Patient Outcomes: | ||||
| Inpatient Mortality | 10.2% | 11.0% | 9.7% | 0.40 |
| 30-Day Mortality | 19.5% | 20.0% | 19.2% | 0.68 |
Only includes patients for which this bundle component was required
Table 2. Unadjusted Analysis.
Unadjusted univariate associations of patient characteristics, process outcomes, and primary mortality outcomes with telehealth utilization.
| No Telehealth Use N = 1684 Mean/Median or N (%) |
|||
|---|---|---|---|
| Facility: | |||
| A | 790 (46.9%) | 44 (63.8%) | <0.01 |
| B | 894 (53.1%) | 25 (36.2%) | |
| Transfer for admission (yes vs no) | 1360 (18.0%) | 20 (71.0% | 0.05 |
| Patient Characteristics: | |||
| Age | 71.0 | 73.0 | 0.69 |
| Female | 48.3% | 47.8% | 0.93 |
| Infectious Source | |||
| Urine | 32.7% | 27.5% | 0.37 |
| Lung | 38.1% | 49.3% | 0.06 |
| Intra-abdominal | 10.2% | 8.7% | 0.84 |
| Unknown | 14.1% | 20.3% | 0.15 |
| SOFA Score | 3.0 | 3.0 | 0.09 |
| Process of Care Outcomes: | |||
| Sepsis Bundle | |||
| Lactic Acid | 96.5% | 98.6% | 0.73 |
| Blood Cultures | 90.1% | 89.9% | 0.84 |
| Antibiotics | 79.5% | 78.3% | 0.80 |
| IV Fluid* (N=516) | 50.0% | 58.8% | 0.32 |
| Repeat Lactic Acid* (N=1192) | 63.8% | 78.0% | 0.04 |
| Vasopressors* (N=295) | 55.0% | 63.6% | 0.51 |
| Reassessment for Septic Shock* (N=488) | 1.5% | 11.1% | <0.01 |
| Primary Outcomes: | |||
| Inpatient Mortality | 10.2% | 8.7% | 0.84 |
| 30-Day Mortality | 19.5% | 18.8% | 0.89 |
Only includes patients for which this bundle component was required
Telemedicine was utilized in 69 cases total (3.9%) and there was a significant increase in telemedicine utilization post-training (5.0% vs 2.1%, p<0.01). Note that the telemedicine program was launched in both ED-A and B, but ED-A underwent in situ simulation training first, therefore “pre-training” telemedicine use did occur in ED-B. Compliance with several SEP-1 bundle components improved post-training: antibiotics within 3 hours, IV fluid administration for patients that necessitated IV fluids by the SEP-1 rules, repeat lactic acid evaluation, and vasopressor administration (Table 1). However, utilization of telemedicine did not demonstrate an association with improved bundle compliance with the exception of repeat lactic acid assessment with an improvement from 63.8% to 78.0% (p=0.04) and reassessment for septic shock with an improvement from 1.5% to 11.1% (p<0.01) (Table 2). We did not demonstrate an improvement in inpatient or 30-day mortality across either of the two group comparisons (pre- vs post-training, no telemedicine vs telemedicine use) (Table 1 and 2). The aforementioned associations between telemedicine, in situ simulation sepsis training, and mortality (Table 3) and process outcomes- including telemedicine use- (Table 4), and telemedicine use and mortality (Table 5) all held in adjusted analyses, with the exception of the association between simulation training and improved compliance with performing a repeat lactic acid assessment (no longer statistically significant).
Table 3. Adjusted Analysis.
Adjusted analysis of association between completion of telehealth in-situ simulation sepsis training and inpatient and 30-day mortality. Odds Ratio with 95% Confidence Intervals.
| Predictors | Primary Outcomes | |
|---|---|---|
| Inpatient Mortality | 30-Day Mortality | |
| Hospital Characteristics | ||
| Facility A (vs B) | 0.76 (0.53, 1.09) | 0.79 (0.60, 1.05) |
| Transfer (yes vs no) | 1.76 (1.18, 2.62) | 1.41 (1.01, 1.97) |
| Patient Characteristics | ||
| Age | ||
| 18 – 60 years | reference | reference |
| 61 – 70 years | 1.22 (0.71, 2.09) | 1.91 (1.25, 2.92) |
| 71 – 81 years | 2.19 (1.33, 3.61) | 2.83 (1.88, 4.26) |
| 82 – 103 years | 2.13 (1.27, 3.59) | 3.71 (2.44, 5.62) |
| Female | 1.06 (0.76, 1.48) | 1.27 (0.98, 1.64) |
| SOFA Score | ||
| 0 – 1 | reference | reference |
| 2 | 2.01 (0.86, 4.66) | 1.71 (1.02, 2.86) |
| 3 – 4 | 4.41 (2.10, 9.26) | 3.09 (1.94, 4.91) |
| 5 – 19 | 13.05 (6.49, 26.24) | 9.33 (6.03, 14.45) |
| Telehealth in-situ simulation sepsis training program completed | 0.82 (0.57, 1.17) | 0.95 (0.72, 1.28) |
Table 4. Adjusted Analysis.
Adjusted analysis of association between completion of telehealth in-situ simulation sepsis training and process outcome completion (sepsis bundle components). Odds Ratio with 95% Confidence Intervals.
| Predictors | Process Outcomes | ||||
|---|---|---|---|---|---|
| Telehealth Use | Antibiotics within 3 hours | IV Fluid Resuscitation* | Repeat Lactic Acid* | Vasopressors for hypotension* | |
| N | 1725 | 1725 | 502 | 1725 | 285 |
| Hospital Characteristics | |||||
| Facility A (vs B) | 1.63 (0.95,2.79) | 1.32 (1.02,1.69) | 1.02 (0.69,1.51) | 0.79 (0.45,1.39) | 0.87 (0.48,1.61) |
| Transfer (yes vs no) | 1.45 (0.80,2.60) | 0.86 (0.63,1.17) | 1.69 (1.09,2.61) | 0.78 (0.40,1.51) | 1.03 (0.54,1.97) |
| Patient Characteristics | |||||
| Age | |||||
| 18 – 60 years | ref | ref | ref | ref | ref |
| 61 – 70 years | 1.27 (0.63,2.59) | 1.14 (0.82,1.60) | 0.88 (0.53,1.46) | 0.77 (0.36,1.64) | 1.59 (0.76,3.30) |
| 71 – 81 years | 1.26 (0.63,2.54) | 1.05 (0.76,1.44) | 0.71 (0.43,1.17) | 0.94 (0.44,2.02) | 1.13 (0.53,2.71) |
| 82 – 103 years | 0.96 (0.45,2.08) | 1.27 (0.90,1.80) | 1.01 (0.59,1.73) | 1.00 (0.45,2.24) | 0.94 (0.41,2.42) |
| Female | 1.23 (0.72,2.14) | ||||
| SOFA Score | 0.98 (0.60,1.62) | 0.77 (0.61,0.98) | 1.05 (0.69,1.44) | 0.94 (0.55,1.60) | 1.40 (1.27,1.55) ** |
| 0 – 1 | ref | ref | ref | ref | |
| 2 | 0.61 (0.26,1.44) | 0.90 (0.65,1.26) | 0.57 (0.29,1.09) | 0.83 (0.36,1.93) | |
| 3 – 4 | 1.10 (0.56,2.17) | 1.19 (0.86,1.65) | 0.72 (0.39,1.31) | 0.64 (0.30,1.36) | |
| 5 – 19 | 1.25 (0.65,2.42) | 1.34 (0.97,1.86) | 0.75 (0.43,1.30) | 0.80 (0.37,1.74) | |
| Telehealth in-situ simulation sepsis training program completed | 2.04 (1.08,3.86) | 1.43 (1.11,1.83) | 1.77 (1.18,2.64) | 1.22 (0.69,2.16) | 2.13 (1.16,3.91) |
Only includes patients for which this bundle component was required
SOFA Score continuous as the model with quartiles was unstable (did not converge).
Table 5. Adjusted Analysis.
Adjusted analysis of association between telehealth use and inpatient and 30-day mortality. Odds Ratio with 95% Confidence Intervals.
| Predictors | Primary Outcomes | |
|---|---|---|
| Inpatient Mortality | 30-Day Mortality | |
| Hospital Characteristics | ||
| Facility A (vs B) | 0.73 (0.51, 1.03) | 0.79 (0.60, 1.03) |
| Transfer (yes vs no) | 1.76 (1.18, 2.63) | 1.42 (1.02, 1.98) |
| Patient Characteristics | ||
| Age | ||
| 18 – 60 years | reference | reference |
| 61 – 70 years | 1.27 (0.74, 2.17) | 1.94 (1.26, 2.97) |
| 71 – 81 years | 2.24 (1.36, 3.69) | 2.85 (1.89, 4.30) |
| 82 – 103 years | 2.16 (1.28, 3.64) | 3.73 (2.45, 5.66) |
| Female | 1.06 (0.75, 1.48) | 1.26 (0.97, 1.64) |
| SOFA Score | ||
| 0 – 1 | reference | reference |
| 2 | 2.00 (0.86, 4.64) | 1.70 (1.01, 2.85) |
| 3 – 4 | 4.39 (2.09, 9.21) | 3.09 (1.94, 4.91) |
| 5 – 19 | 13.14 (6.53, 26.41) | 9.38 (6.06, 14.53) |
| Telehealth Use | 0.46 (0.16, 1.33) | 0.69 (0.34, 1.39) |
DISCUSSION
This is the first work to demonstrate a relationship between in situ simulation training in the rural environment and improved uptake of new technology, specifically telemedicine. We demonstrated a modest increase in utilization of telemedicine after in situ simulation training. We were also able to demonstrate a modest improvement in sepsis process of care metrics, and the SEP-1 bundle compliance after in situ simulation training, for all patients. Particular to telemedicine use, we saw an improvement in sepsis process of care metrics that focus on vigilance in reassessment (repeat lactic acid assessment and reassessment for septic shock). While we were unable to demonstrate an improvement in in-hospital or 30-day mortality, the study was not powered to detect a difference in mortality, and marginal mortality benefit is difficult to detect in the era of protocolized care27–29 and with smaller sample sizes in rural ED care. There are multiple elements to consider when discussing the results of this study: in situ simulation training, the nature of the proposed telemedicine, the adoption of the proposed telemedicine, the impacts of telemedicine, and evaluating these interventions in the rural environment.
The in situ simulation training addressed the introduction of the technology, its incorporation into care, and the care process for sepsis. The training was completed by 80% of the nursing staff at the sites, with interprofessional and physician participation welcome. The intervention was designed to have content validity by including an interprofessional team with emergency medicine expertise, and training held to a checklist to ensure the opportunity for the team to demonstrate all critical items of interest for both telemedicine and sepsis care.26 Other simulation teams, with varying degrees of repetitive training, have demonstrated a positive impact on readiness for pediatric trauma,30 pediatric emergency teams,31 advanced cardiac life support,32, 33 and demonstrated mixed results for emergent delivery.34 While we did not repeat the simulations to study readiness, ours is among an uncommon group of simulation interventions to track process and patient care outcomes in real patients after simulation.35, 36 Other examples of simulation training translating to patient care include procedures where a learner can be tracked to a specific patient, and include central line insertion training,37–39 thoracentesis,40 and lumbar puncture.41
The proposed telemedicine intervention included the ability of the team to connect to the eICU team via a telemedicine cart. While this cart was essentially the same as the tele-stroke mechanism, including the connectivity software (Vidyo©), it represented an extra step in the care of septic patients. Unlike telestroke, in which the use of the cart connected to a service that was not available otherwise (a neurologist) and led to an immediate patient care decision (thrombolytics or not), this cart application was proposed to improve care that in theory should already be ongoing. Thus, it may have been perceived as an encumbrance rather than a boon. The barriers and facilitators of the adoption of telemedicine in this setting are the subject of a planned qualitative analysis. There are additional factors that may account for the benefit of telemedicine. In this study, most of the ED-eICU connections occurred via telephone, and were facilitated by the eICU nurse placing the patient in their list of “sepsis hospital” patients within the EMR. This list provides a level of monitoring and helps the eICU team check compliance with sepsis process goals across the ED and eICU sites. In addition, the eICU and the two ED sites all had available a sepsis automated alert mechanism through the EPIC© EMR that could be used to trigger telemedicine initiation. Sepsis alerts have been shown to improve process outcomes,42, 43 but not mortality in sepsis. In the context of this study, the presence and benefits of these factors may wash out the marginal benefit of deeper telemedicine involvement, but this is difficult to assess.
Because of the limited use of telemedicine in this study, it is difficult to comment on the true impact of telemedicine in rural EDs for the improvement of sepsis care. Based on our experience of integrating the technology, deploying the simulations and tracking the outcomes, we would recommend several items be considered prior to a deployment event. First, while there are certainly issues with alarms and alarm time costs, we suggest considering sepsis automated alerts as a low-cost reminder. These should improve as machine learning techniques lead to more predictive value.44, 45 Second, consider the ease with which telemedicine can ideally be initiated and be most helpful, and how the connection is made to the nurse or provider. It is possible that telemedicine is only helpful in patients who are more critically ill and therefore a more targeted approach could be used. For example, a dedicated room in a small ED might have pre-installed cameras, and a one button touch activation for telemedicine engagement. This solution sacrifices flexibility for ease of initiation of the connection, but the connection process must be simple and fast, with easy escalation to provider support. Third, many eICUs use sophisticated physiology monitoring algorithms to monitor patients for more subtle vital sign changes that may indicate deterioration.15, 46, 47 However, the integration of ED patients into such monitoring schemes, particularly in rural EDs, is not without cost or the need for technical support. Thus, our patients did not benefit from some of the most advanced vigilance functions our eICU had to offer and the benefit of telemedicine could be even greater if these steps were taken.
While it is clearly of value to study care improvement and patient safety in the rural environment as over 51 million Americans live in rural areas, published research is limited. Research is less robust and more difficult in the rural environment, likely for several reasons including lower volumes and distance from large academic centers with research knowledge, resources, and processes already in place. In addition, while the study used established performance improvement design, tracking methods and attempted to adhere to appropriate study planning and reporting guidelines,48 it was still subject to the risks inherent in both retrospective and performance improvement designs. We attempted to control for key covariates with statistical planning and analysis, but temporal trends in sepsis care are likely factors: ongoing self-education of physicians and staff outside of our efforts, awareness efforts through professional organizations and government agencies, local mechanisms for feedback to staff on performance, among other unmeasured variables. As with any operational study, we did not operate in a vacuum and there was variation in the degree of feedback on sepsis care performance the teams received at each site with more or less emphasis at meetings depending on the site. This study is rare in that it is a multi-center study of rural EDs over a long period of time to allow for adequate patient enrollment. In addition, this study evaluated a process, telemedicine, and a training modality, in situ simulation, that are well suited to the rural ED environment and implementation in other rural EDs.
CONCLUSION
This study is unique in that we have demonstrated an association between simulation training and how care is delivered in the real patient care environment and with process care measures. Implementing an in situ simulation curriculum in rural EDs, with a goal of increasing telemedicine adoption for sepsis, led to a small increase in the use of telemedicine in these EDs. The intervention was associated with improvements in sepsis process of care components, but did not demonstrate a significant improvement in mortality.
Acknowledgements:
The authors would like to acknowledge simulation specialist, Dustin Holzwarth; Jump Simulation Research Assistants Chase Salazar, Rebecca Ebert-Allen, Kyle Mou, Gabriel Beratio; the Standardized Participant Program at Jump Simulation; OSF ConstantCare eICU staff: William Tillis, MD, Toni Bortell, RN; Sarah Metzger, RN; Lori Nelson, RN; Courtney Pilat, RN; Erika Shaw, RN, and Vicki Weatherington, RN; Rural ED leadership: Erin Nimbler, RN and Lisa Sopher, RN; and all other OSF Saint James ED and OSF St. Mary’s ED staff who participated and provided feedback during the study.
Funding:
This study has been funded by Agency for Healthcare Research and Quality Grant 1 R18 HS024027-01 “In Situ Simulation for Adoption of New Technology to Enhance Safety in Rural EDs”.
REFERENCES
- 1.Casey MM, Wakefield M, Coburn AF, Moscovice IS, Loux S. Prioritizing patient safety interventions in small and rural hospitals. Jt Comm J Qual Patient Saf. 2006;32:693–702. [DOI] [PubMed] [Google Scholar]
- 2.Coburn AF, Wakefield M, Casey M, Moscovice I, Payne S, Loux S. Assuring rural hospital patient safety: what should be the priorities? J Rural Health. 2004;20:314–326. [DOI] [PubMed] [Google Scholar]
- 3.Culler SD, Atherly A, Walczak S, et al. Urban-rural differences in the availability of hospital information technology applications: a survey of Georgia hospitals. J Rural Health. 2006;22:242–247. [DOI] [PubMed] [Google Scholar]
- 4.Tupper J, Coburn A, Loux S, Moscovice I, Klingner J, Wakefield M. Advances in Patient Safety: Strategies for Improving Patient Safety in Small Rural Hospitals. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign). Rockville (MD): Agency for Healthcare Research and Quality; (US: ); 2008. [PubMed] [Google Scholar]
- 5.Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–1310. [DOI] [PubMed] [Google Scholar]
- 6.Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35:1244–1250. [DOI] [PubMed] [Google Scholar]
- 7.Pepper DJ, Jaswal D, Sun J, Welsh J, Natanson C, Eichacker PQ. Evidence Underpinning the Centers for Medicare & Medicaid Services’ Severe Sepsis and Septic Shock Management Bundle (SEP-1) A Systematic Review. Annals of internal medicine. 2018;168:558–568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Faust JS, Weingart SD. The past, present, and future of the Centers for Medicare and Medicaid Services quality measure SEP-1: the early management bundle for severe Sepsis/Septic shock. Emergency Medicine Clinics. 2017;35:219–231. [DOI] [PubMed] [Google Scholar]
- 9.Powell ES, Khare RK, Courtney DM, Feinglass J. Volume of emergency department admissions for sepsis is related to inpatient mortality: results of a Nationwide Cross-Sectional Analysis. Crit Care Med. 2010;38:2161–2168. [DOI] [PubMed] [Google Scholar]
- 10.Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock: 2012. Crit Care Med. 2013;41:580–637. [DOI] [PubMed] [Google Scholar]
- 11.Mueller KJ, Potter AJ, MacKinney AC, Ward MM. Lessons from tele-emergency: improving care quality and health outcomes by expanding support for rural care systems. Health Aff (Millwood). 2014;33:228–234. [DOI] [PubMed] [Google Scholar]
- 12.Lilly CM, Zubrow MT, Kempner KM, et al. Critical Care Telemedicine: Evolution and State of the Art. Crit Care Med. 2014. [DOI] [PubMed] [Google Scholar]
- 13.American Telemedicine Association.
- 14.Lilly CM, Cody S, Zhao H, et al. Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. Jama. 2011;305:2175–2183. [DOI] [PubMed] [Google Scholar]
- 15.McCambridge M, Jones K, Paxton H, Baker K, Sussman EJ, Etchason J. Association of health information technology and teleintensivist coverage with decreased mortality and ventilator use in critically ill patients. Arch Intern Med. 2010;170:648–653. [DOI] [PubMed] [Google Scholar]
- 16.Willmitch B, Golembeski S, Kim SS, Nelson LD, Gidel L. Clinical outcomes after telemedicine intensive care unit implementation. Crit Care Med. 2012;40:450–454. [DOI] [PubMed] [Google Scholar]
- 17.Rincon TA, Bourke G, Seiver A. Standardizing sepsis screening and management via a tele-ICU program improves patient care. Telemed J E Health. 2011;17:560–564. [DOI] [PubMed] [Google Scholar]
- 18.Khunlertkit A, Carayon P. Contributions of tele-intensive care unit (Tele-ICU) technology to quality of care and patient safety. J Crit Care. 2013;28:315.e311–312. [DOI] [PubMed] [Google Scholar]
- 19.Dharmar M, Kuppermann N, Romano PS, et al. Telemedicine consultations and medication errors in rural emergency departments. Pediatrics. 2013;132:1090–1097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rogove HJ, McArthur D, Demaerschalk BM, Vespa PM. Barriers to telemedicine: survey of current users in acute care units. Telemed J E Health. 2012;18:48–53. [DOI] [PubMed] [Google Scholar]
- 21.Reinhardt I, Gouzoulis-Mayfrank E, Zielasek J. Use of Telepsychiatry in Emergency and Crisis Intervention: Current Evidence. Curr Psychiatry Rep. 2019;21:63. [DOI] [PubMed] [Google Scholar]
- 22.Sharma R, Zachrison KS, Viswanathan A, et al. Trends in Telestroke Care Delivery: A 15-Year Experience of an Academic Hub and Its Network of Spokes. Circulation. Cardiovascular quality and outcomes 2020;13:e005903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Johnson BE, Shifferd J. Who Lives Where: A Comprehensive Population Taxonomy of Cities, Suburbs, Exurbs, and Rural Areas in the United States. Geographical Bulletin. 2016;57. [Google Scholar]
- 24.Fast Facts on U.S. Hospitals. Vol 2020: American Hospital Association; 2020. [Google Scholar]
- 25.Moeckli J, Cram P, Cunningham C, Reisinger HS. Staff acceptance of a telemedicine intensive care unit program: a qualitative study. J Crit Care. 2013;28:890–901. [DOI] [PubMed] [Google Scholar]
- 26.Bond WF, Barker LT, Cooley KL, et al. A Simple Low-Cost Method to Integrate Telehealth Interprofessional Team Members During In Situ Simulation. Simul Healthc. 2019;14:129–136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Peake SL, Bailey M, Bellomo R, et al. Australasian resuscitation of sepsis evaluation (ARISE): A multi-centre, prospective, inception cohort study. Resuscitation. 2009;80:811–818. [DOI] [PubMed] [Google Scholar]
- 28.Yealy DM, Kellum JA, Huang DT, et al. A randomized trial of protocol-based care for early septic shock. The New England journal of medicine. 2014;370:1683–1693. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Mouncey PR, Osborn TM, Power GS, et al. Trial of early, goal-directed resuscitation for septic shock. N Engl J Med. 2015;372:1301–1311. [DOI] [PubMed] [Google Scholar]
- 30.Hunt EA, Heine M, Hohenhaus SM, Luo X, Frush KS. Simulated Pediatric Trauma Team Management: Assessment of an Educational Intervention. Pediatric Emergency Care. 2007;23. [DOI] [PubMed] [Google Scholar]
- 31.Theilen U, Fraser L, Jones P, Leonard P, Simpson D. Regular in-situ simulation training of paediatric Medical Emergency Team leads to sustained improvements in hospital response to deteriorating patients, improved outcomes in intensive care and financial savings. Resuscitation. 2017;115:61–67. [DOI] [PubMed] [Google Scholar]
- 32.Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WC. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008;133:56–61. [DOI] [PubMed] [Google Scholar]
- 33.Didwania A, McGaghie WC, Cohen ER, et al. Progress toward improving the quality of cardiac arrest medical team responses at an academic teaching hospital. Journal of graduate medical education. 2011;3:211–216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Fransen AF, van de Ven J, Schuit E, van Tetering AAC, Mol BW, Oei SG. Simulation-based team training for multi-professional obstetric care teams to improve patient outcome: a multicentre, cluster randomised controlled trial. BJOG: An International Journal of Obstetrics & Gynaecology. 2017;124:641–650. [DOI] [PubMed] [Google Scholar]
- 35.McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc. 2011;6 Suppl:S42–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB. A critical review of simulation-based mastery learning with translational outcomes. Med Educ. 2014;48:375–385. [DOI] [PubMed] [Google Scholar]
- 37.Ahya SN, Barsuk JH, Cohen ER, Tuazon J, McGaghie WC, Wayne DB. Clinical performance and skill retention after simulation-based education for nephrology fellows. Seminars in dialysis. 2012;25:470–473. [DOI] [PubMed] [Google Scholar]
- 38.Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169:1420–1423. [DOI] [PubMed] [Google Scholar]
- 39.Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697–2701. [PubMed] [Google Scholar]
- 40.Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology. 2012;79:132–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Kessler DO, Auerbach M, Pusic M, Tunik MG, Foltin JC. A randomized trial of simulation-based deliberate practice for infant lumbar puncture skills. Simul Healthc. 2011;6:197–203. [DOI] [PubMed] [Google Scholar]
- 42.Hwang MI, Powell ES, Bond WF. Sepsis Alerts in Emergency Departments: A systematic review of accuracy and quality measure impact. Western Journal of Emergency Medicine (in press). 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: a systematic review. Journal of hospital medicine. 2015;10:396–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Delahanty RJ, Alvarez J, Flynn LM, Sherwin RL, Jones SS. Development and Evaluation of a Machine Learning Model for the Early Identification of Patients at Risk for Sepsis. Annals of emergency medicine. 2019;73:334–344. [DOI] [PubMed] [Google Scholar]
- 45.Giannini HM, Ginestra JC, Chivers C, et al. A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice*. Critical Care Medicine. 2019;47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Fusaro MV, Becker C, Scurlock C. Evaluating Tele-ICU Implementation Based on Observed and Predicted ICU Mortality: A Systematic Review and Meta-Analysis*. Critical Care Medicine. 2019;47. [DOI] [PubMed] [Google Scholar]
- 47.Lilly CM, McLaughlin JM, Zhao H, Baker SP, Cody S, Irwin RS. A Multicenter Study of ICU Telemedicine Reengineering of Adult Critical Care. Chest. 2014;145:500–507. [DOI] [PubMed] [Google Scholar]
- 48.Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Quality & Safety. 2016;25:986–992. [DOI] [PMC free article] [PubMed] [Google Scholar]