Use of an Electronic Decision Support Tool Improves Management of Simulated In-Hospital Cardiac Arrest

Abstract

INTRODUCTION

Adherence to Advanced Cardiac Life Support (ACLS) guidelines during in69 hospital cardiac arrest (IHCA) is associated with improved outcomes, but current evidence shows that sub-optimal care is common. Successful execution of such protocols during IHCA requires rapid patient assessment and the performance of a number of ordered, time-sensitive interventions. Accordingly, we sought to determine whether the use of an electronic decision support tool (DST) improves performance during high-fidelity simulations of IHCA.

METHODS

After IRB approval and written informed consent was obtained, 47 senior medical students were enrolled. All participants were ACLS certified and within one month of graduation. Each participant was issued an iPod Touch device with a DST installed that contained all ACLS management algorithms. Participants managed two scenarios of IHCA and were allowed to use the DST in one scenario and prohibited from using it in the other. All participants managed the same scenarios. Simulation sessions were video recorded and graded by trained raters according to previously validated checklists.

RESULTS

Performance of correct protocol steps was significantly greater with the DST than without (84.7% v 73.8%, p< 0.001) and participants committed significantly fewer additional errors when using the DST (2.5 errors v. 3.8 errors, p< 0.012).

CONCLUSION

Use of an electronic DST provided a significant improvement in the management of simulated IHCA by senior medical students as measured by adherence to published guidelines.

INTRODUCTION

The practice of hospital-based and perioperative medicine requires the knowledge and application of many diverse acute care skills, including the management of in-hospital cardiac arrest (IHCA). The presence of Advanced Cardiac Life Support (ACLS) trained providers and adherence to published guidelines for the management of cardiac arrest are associated with improved outcomes.^1–6 However, management skills and appropriate application of ACLS guidelines have been shown to quickly fade after training.^7–9 Furthermore, current evidence shows that sub-optimal resuscitation is common in both medical and surgical patients.^3,4,10–13

During rapidly evolving or deteriorating patient conditions, there is often insufficient time for physicians to re-familiarize themselves with current guidelines. Although both paper and electronic aids may facilitate guideline adherence during the management of acute patient instabilities or cardiac arrest,^14–16 there is also evidence that cognitive aids can negatively affect the way in which providers deliver care during cardiac arrest.^17,18 Therefore, we sought to determine whether the use of an electronic decision support tool (DST) that dynamically guides a provider through American Heart Association (AHA) ACLS protocols would improve performance during high-fidelity simulations that require the management of acute dysrhythmias and IHCA.

METHODS

Study Design

After IRB approval was granted, written informed consent was obtained from 47 ACLS-certified senior medical students one month prior to graduation (26 females, 20 males). Each participant was issued an iPod Touch™ device (Apple Inc., Cupertino, CA) on which the DST was installed. They were given a brief orientation (~30 min) to the use of the DST user interface and to the simulation environment, the simulator mannequin (SimMan 2G®, Laerdal, Inc., Stavanger, Norway), and all equipment to be used during the study (defibrillator, monitors, etc).

Each participant managed two high-fidelity simulation scenarios involving IHCA in a cross-over design (see Figure 1). In order to remove bias in performance due to any potential difference in scenario difficulty or due to DST order, participants were assigned to one of four combinations as shown in Figure 1 through a random number generator. The scenarios were constructed according to AHA testing standards for Megacode scenarios where a participant manages an acute dysrhythmia in a pulsatile state, then a shockable pulseless state, and then a non-shockable pulseless state. Scenario A consisted of the patient presenting with unstable bradycardia proceeding to ventricular fibrillation and finally to asystole. Scenario B consisted of the patient presenting with a narrow complex stable tachycardia proceeding to pulseless monomorphic ventricular tachycardia and finally to PEA.

Figure 1.

This figure illustrates participant randomization and allocation. Participants were randomized to one of four pathways, representing an equivalent to randomizing both the order of DST use and the order of Megacode scenario testing. After they managed the first Megacode scenario, they then managed the remaining scenario with or without the DST, opposite from how they managed the first scenario. [DST = decision support tool; A = Megacode scenario A; B = Megacode scenario B].

We performed all simulation sessions in a setting that replicated a patient room on a general care ward at our institution. The code cart and defibrillator used during these events replicated those in clinical use at our institution, as did the available medications, intravenous (IV) fluids, IV lines, medical gases, and airway devices. Each participant was told to assume the role of the physician team leader responding to a code in the university hospital. All other roles (chest compressions, airway manager, drug administration, and defibrillator manager) were played by a standardized code team that was comprised of simulation center staff trained to respond with consistent scripted responses.

The DST was programmed for the Apple iOS and deployed on iPod™ devices, as noted above. The Appendix shows screen shots from the DST through which the user would navigate during a brief patient assessment and then management. The management steps programmed into the DST were based upon the 2005 AHA ACLS update.

Analysis and Grading

Each session was video recorded via a multi-camera system and B-line Medical® software (B-line Medical, Washington, D.C.) and then graded by two experienced raters according to previously validated scoring checklists derived from AHA training manuals.¹⁹ A score for the percentage of correct actions performed was recorded. Since only errors of omission (i.e. not doing a prescribed action) are captured in the percentage of correct steps, errors of commission were recorded in a separate error count (e.g. shocking a patient in asystole).

Global performance on each scenario was compared so that any bias due to scenario difficulty could be taken into account. With no demonstrable difference in global performance between scenario A and scenario B, further adjustment of the analysis model to account for scenario difficulty was not needed. The averaged correct actions and error count scores for each participant were used in the analysis, one from the scenario managed with the DST and one from the scenario managed without the DST. Another sensitivity analysis was also performed that took into account whether the DST was used first or second, but no difference was found. Therefore, the percent correct actions and the error counts were analyzed through an ANOVA to compare performance with and without the DST. Data are presented as Mean ± 95% CI for continuous variables and p<0.05 was considered significant.

RESULTS

Overall, as seen in Figure 2, the total number of correct checklist items during high-fidelity simulation of IHCA was significantly improved with the iOS-based DST compared to management from memory alone (78.4% v. 71.8%, P < 0.001). Further analysis by checklist item type revealed that the number of correct management steps was significantly improved with the use of the DST (84.7 v. 73.8%, P < 0.001), while the number of patient assessment steps did not appear significantly improved (73.8 v 70.4%, P = 0.103). In addition to improving the percentage of correct guideline actions performed, participants were also less likely to make errors of commission, such as inappropriate defibrillation or incorrect medication administration, when the DST was used (2.5 errors v. 3.8 errors, P< 0.012, see Figure 3). As measured by time to first defibrillation, which was similar with and without the DST (63.6 v. 65.7 seconds, P = 0.808, respectively), use of the DST did not appear to delay timely delivery of care.

Figure 2.

This figure illustrates the difference in performance based upon use of the DST as measured by the percentage of correct actions completed during a simulated IHCA. Of note, the DST did not make a significant difference in assessment of the patient, but did significantly improve adherence to the management protocol. [DST = decision support tool]

Figure 3.

This figure illustrates the difference in performance based upon use of the DST as measured by the number of errors committed during a simulated IHCA. This only involves errors of commission, such as defibrillating when not indicated. Errors of omission are accounted for in percentage of correct actions completed shown in figure 2. [DST = decision support tool]

DISCUSSION

As noted above, adherence to published ACLS guidelines is often suboptimal with respect individual aspects such as to time to defibrillation and quality of CPR during in-hospital cardiac arrest.^3,4,10,12,13 Furthermore, we have recently shown that improved overall adherence to published ACLS guidelines throughout the entire resuscitation event is associated a much greater likelihood of achieving return of spontaneous circulation (ROSC).²⁰ Thus, we investigated whether the use of an electronic DST that dynamically guides a provider through published AHA ACLS protocols would improve performance during high-fidelity simulations requiring management of acute dysrhythmias and IHCA. The results of this study demonstrate that the DST which we tested significantly improved adherence to and reduced deviations from published guidelines. Our recent findings from a retrospective analysis of IHCA at our institution suggest that the 11% absolute improvement in adherence to guidelines that we report in this simulation study would likely be clinically relevant, possibly improving the odds of return of spontaneous circulation by approximately 30%.²⁰

Additionally, the DST tested in this study advances upon the previous report of a smartphone cognitive aid used in simulations of IHCA,¹⁶ as the DST used in this study actively prompts the end-user with the proper decision points and actions to consider based upon prior decisions and management steps logged by the end-user (see Appendix for example). It is therefore a dynamic and fluid management tool for IHCA events. Concerning the use of DSTs in healthcare today, those deployed within the electronic medical record have been shown to improve adherence to guidelines and, in some reports, to improve patient outcomes in non-crisis settings.^22–24 Furthermore, the use of mobile technology (e.g. automated text messaging reminders) has been shown to improve adherence to guidelines for clinicians and patients in the management of several conditions.^25–27 Even though these DSTs and mobile technology tools are showing benefit, none of these tools are for use at the bedside in crisis situations. Our report thus advances upon what has previously been shown concerning the effect of DSTs on adherence to guidelines in high-fidelity simulation of crisis situations. Future research needs to test its application in the clinical arena.

Another important consideration with respect to testing and employing technology is that it can often change the manner in which people work, and not always in a beneficial manner. The new technology may distract and/or burden the clinician with extra work, resulting in an impediment to timely IHCA management. Zanner et al. found that a mobile phone application not only failed to improve BLS performance of laypersons, but actually slowed the performance of those with pre-existing BLS knowledge.¹⁷ Nelson et al. also found that residents using pocket cards were more likely to choose an incorrect pediatric cardiac arrest treatment algorithm, and suggested that successful cognitive aids should help guide this decision-making.¹⁸ We, therefore, used a human factors approach to streamline our dynamic DST to assist with rapid decision making and to reinforce the timely application of crucial management steps. We believe this extra attention to the human factors aspect during application development is largely responsible for our ability to demonstrate improved overall performance without slowing down IHCA management. In contrast to these previous studies, our students were able to complete a greater number of correct actions in the same amount of time while using the DST, and time to first defibrillation was similar between the groups.

Our results demonstrated that the greatest effect of the DST was on adherence to management guidelines during simulated IHCA rather than patient assessment. This is similar to a recent report by Neal et al. concerning the effect of a cognitive aid on the management of simulated local anesthetic systemic toxicity.²⁸ In that study and ours, the checklist used for grading contained steps dealing with patient assessment and patient management. In both studies, use of the cognitive aid made a significant difference in the management steps, but not in the items related to patient assessment such as systematically checking neurological responsiveness, adequacy of airway/breathing, and existence of intravenous access. The measured difference in assessment may not have been significant because assessment of patients may occur by a mental process that is more difficult to effect through the use of a cognitive aid. Alternatively, failure to detect a difference in patient assessment may be due to challenges in the video observers’ ability to grade the assessment items. Patient assessment items graded during these acute physiologic disturbances may have been adequately considered, but not verbalized by the participants, therefore limiting the grading observers’ ability to record any difference in the assessment process. Of note, to avoid any grading bias towards the DST group, the graders did not see the assessments that may have been selected on the electronic DST, but not verbalized. The treatment steps require more definitive action by the participant, which may have allowed more precise grading and easier demonstration of improvement with the DST. While it is unfortunate that we were not able to detect a significant improvement in patient assessment with the DST, it is ultimately the correct patient treatment steps that translate into better clinical outcomes.^3,4,10

There are several potential limitations of the present study that must be acknowledged. First, the study population involved ACLS-certified graduating medical students, which might be of concern with respect to generalizability of our results to more experienced clinicians. This has been partially addressed by several studies reporting ACLS certification as the factor of importance, not training level,^1,29 but medical students may also be more comfortable with the use of smartphone applications than older clinicians. Second, we isolated the performance of the team leader in this study for analysis. It may be of benefit in future studies to test the impact of a DST on the performance of the entire team.^30,31 While direction given by the team leader is important, it is the whole of the actions performed by the entire team that matter most to clinical outcomes. In fact, it may be even more beneficial to have an otherwise non-tasked team member solely focusing on navigating the DST in context to the real-time events. Third, there was a ceiling effect in this trial, such that performance with the DST did not reach 100% compliance even when all of the appropriate management steps were presented to the team leader. This highlights the need for ongoing human factors assessment of the technology, as there could be a problem with the user interface that was limiting or it is possible that not enough time was spent on orientation to the DST. Future studies are needed to elucidate how to improve adherence to 100% of all recommended protocol items and reduce the number of errors (i.e. inappropriate deviations from protocol). While we provided orientation regarding the use of the DST prior to study, we did not formally assess the students’ proper use of the DST prior the study.

CONCLUSION

The dynamic decision support tool investigated in this study is effective for increasing adherence to the ACLS protocols without causing any delay in treatment during high-fidelity simulation of IHCA. Further research needs to investigate whether this impact is achievable for team performance in the clinical environment and to confirm whether such improved clinical performance translates to improved patient outcomes.

Appendix

Shown are a series of application screenshots as an example of the user interface. Although dynamically branching logic is built in throughout the application, the first screenshot is an early prompting for clinical assessment of the pulse. If the user indicates that the patient has a pulse (finger 1), they are prompted down a pathway (starting at arrow 1) that includes different assessment and treatment items than if the user were to select a pulseless state (finger 2 and arrow 2). As the user selects assessment items and/or checks off treatments as completed/started, the application will continuously update to show the next algorithm steps with prioritization and ongoing timing cues. At any time from any screen, the clinician may indicate that the patient's rhythm has changed and therefore quickly navigate back to the basic pulse and rhythm assessment.