How Do Simulated Error Experiences Impact Attitudes Related to Error Prevention?

Abstract

Introduction:

The objective of this project was to determine whether simulated exposure to error situations changes attitudes in a way that may have a positive impact on error prevention behaviors.

Methods:

Using a stratified quasi-randomized experiment design, we compared risk perception attitudes of a control group of nursing students who received standard error education (reviewed medication error content and watched movies about error experiences) to an experimental group of students who reviewed medication error content and participated in simulated error experiences. Dependent measures included perceived memorability of the educational experience, perceived frequency of errors, and perceived caution with respect to preventing errors.

Results:

Experienced nursing students perceived the simulated error experiences to be more memorable than movies. Less experienced students perceived both simulated error experiences and movies to be highly memorable. After the intervention, compared with movie participants, simulation participants believed errors occurred more frequently. Both types of education increased the participants’ intentions to be more cautious and reported caution remained higher than baseline for medication errors 6 months after the intervention.

Conclusions:

This study provides limited evidence of an advantage of simulation over watching movies describing actual errors with respect to manipulating attitudes related to error prevention. Both interventions resulted in long-term impacts on perceived caution in medication administration. Simulated error experiences made participants more aware of how easily errors can occur, and the movie education made participants more aware of the devastating consequences of errors.

Medication and intravenous (IV) fluid administration errors^* are frequent and can be harmful.^1–4 Although it may seem simple to check a patient’s identification and the type and dose of medication or fluid administered, research suggests that it is not uncommon for clinicians to miss important information.^5–8 Performance-shaping factors that make it difficult to detect errors include repetitive work for long hours, a relatively low rate of errors (so they are not expected), lack of feedback when errors are present and missed, and competing attention demands.⁹ Systems such as bar code medication administration have been implemented to support these tasks. However, for different reasons, it is feasible and perhaps not uncommon for clinicians to “work around” systems solutions.^10,11 There is a need for improving (1) clinicians’ intentional compliance with risk reduction policies and (2) clinicians’ performance with respect to detecting errors.^6,7,29,30

Compliance with error prevention strategies is expected to be driven, in part, by whether clinicians perceive strategies to be necessary and effective.^12,13 Perceived frequency and personal susceptibility to errors are expected to influence perceptions of the need for prevention strategies. Beyond this, there is research to suggest that there are important emotional components to both intentional decision making (eg, whether or not to comply with risk reduction practices) and attentional performance (eg, the ability to detect unexpected error situations) in the context of high-risk tasks.^14–17 Loewenstein et al¹⁴ refer to this as “risk as a feeling.” Emotional memories associated with negative situations (eg, seeing a bear in the woods) seem to be more accurate and longer lasting than memories for routine situations.^15,16 In the context of gastrointestinal surgeons recognizing risky situations and avoiding errors, Moulton et al¹⁸ conceptualized that it is possible to disrupt automated processing and transition to more controlled processing (eg, “slowing down when you should”). The experience of emotional memories associated with negative situations may be an important link in building recognition of situations that require this transition.

Research has linked previous experience with adverse events and severity of consequences to seeking out and complying with risk reduction strategies.^19,20 Behaviors show greater consistency with attitudes when attitudes are attained through personal experience.²¹ Timing of experiences can also affect attitudes. Early experiences with new situations tend to bias individuals to interpret future experiences in a similar light, the Halo effect.¹⁷ For example, if trainees experience error situations early in their clinical experiences, they would be more likely to believe that errors are common and seek out confirmatory information. However, if early experiences are uneventful, trainees may dismiss later experiences with errors as being abnormal or “explain away” these occurrences.

If personal experience with error situations changes risk perception in a way that impacts behavior, it may also be true that exposure to simulated error events (eg, within a structured educational program) can impact risk perception and associated behaviors. Simulated error experiences, particularly those involving good feedback related to severity of consequences, may (1) change attitudes related to the consequences of missing an error, (2) increase the perceived frequency of errors, and (3) impact emotional memory in a way that leads to deep encoding and recall of the experience, which may support longer-term improvement in the ability to detect errors.^14,15,17 In pilot testing of simulated error experiences by the authors,²² participants expressed some level of stress, fear, or dismay when they realized that they missed identifying an embedded error. However, their reaction to these simulations was more positive than negative. They described the experience as “eye-opening” and “gave me a sense of my own vulnerabilities.” Practicing nurses commented that they would like to see this type of educational experience repeated periodically (every 6 months to a year).

The primary aim of this project was to determine whether exposure to simulated error events would change attitudes toward risk. Specifically, does exposure to simulated error situations change attitudes toward the perceived frequency of events or change reported caution in risk-related activities? Are these changes long-lasting, because one would expect whether the experience created the intended emotional memories? We sought to compare the attitudes of nursing students exposed to current educational practices, which include watching movies involving third-party stories of serious error experiences and subsequent consequences, with the attitudes of nursing students exposed to simulated error experiences. Although both movies about serious errors and simulated error situations were expected to generate emotional memories, we hypothesized that simulation would have a greater and longer-term impact because (1) participants would encode the memory in the context of a realistic and personally relevant experience and (2) participants would be exposed to their own fallibility (if errors were missed) in the context of these tasks. A long-term goal is to understand factors that will most effectively shape a program of education for error prevention and that will provide a significant and lasting impact on attitudes and behaviors to decrease rates of errors.

METHODS

Study Design

We conducted a prospective stratified quasi-randomized experiment comparing risk perception attitudes of nursing students across 2 intervention conditions. The experiment was conducted at Boise State University in Idaho in 2013 and 2014. The independent variable was intervention condition: exposure to education involving simulated error experiences or standard error education, which included watching movies about true error experiences. Participants in both conditions received educational content in the form of PowerPoint style lecture materials about medication errors to read before attending either a movie or simulation session. Dependent variables included survey-based responses to assess nursing risk attitudes and were collected before and at 3 time points after the intervention. Institutional review board approval for the experiment and for a mixed methods pilot study used to inform the design of the simulation-based intervention was obtained from Boise State University and Saint Alphonsus Regional Medical Center.

Simulation Intervention

We conducted pilot testing at simulation laboratories at the university’s nursing school and at Saint Alphonsus Regional Medical Center to identify the most effective simulated error experiences and to validate survey questions. High-fidelity simulations were designed to mimic the emotional and psychological impact of being involved in error events. We varied patient presentation, type of embedded error, number of embedded errors, whether the participant acted alone or as a team, details of the introduction (eg, related to deception), and details of debriefing. Quantitative metrics on memorability of the experience and qualitative discussion after the simulations from 17 students and 5 practicing nurses suggested that the simulated error experience was more impactful when (1) the patient was a baby, (2) participants performed alone, (3) more than 1 simulation was performed, and (4) the experience included presentation of true stories with severe consequences of errors.²² Information warning participants of the potential for nonroutine-embedded events did not impact perceived memorability of the activity or rate of detection of embedded errors. Simulations involving 2 or more embedded errors did not increase participants’ perceptions that they were being “set up” for an error.

On the basis of these pilot test findings, we selected 1 adult and 1 infant scenario, each with 2 error situations embedded, and chose to have 1 student at a time to participate in the simulation intervention. For both scenarios, participants were provided a medical record with demographics, allergies, history of present illness, medical history, current medications, significant laboratory values and diagnostics, social history, and a brief scripted nurse facilitator report (similar to a shift report) of the patient’s current status, and the tasks to be performed. The script instructed participants to perform nursing assessments, administer medications, and implement the written orders. Participants were given a phone number to call a charge nurse, pharmacist, or physician if they needed further information. If a call was made, a facilitator played the role of the relevant provider and responded to queries. The responses were not scripted but were brief and consistent. The scenarios were discontinued as soon as participants completed the key tasks and did not progress to physiologic changes based on participant actions.

In the adult scenario, an elderly man (Gaumard HAL S3101) with congestive heart failure (CHF), upper respiratory infection, and allergy to penicillin is to receive IV fluids and antibiotics. The patient’s monitor displayed the following values: heart rate, 82 beats per minute; respiratory rate, 24 breaths per minute; blood pressure, 142/80 mm Hg; temperature, 101.8°F; and SpO₂, 94%. This scenario contained 2 embedded errors: (1) IV fluids running at 20 mL/h, although the order is for fluids running at 55 mL/h and (2) an order for penicillin and the patient has a penicillin allergy. If participants queried the patient regarding allergies or other concerns, scripted responses were given by a facilitator via voice-modulated speaker. If participants missed identification of the penicillin allergy, the outcome could range from mild to severe allergic reaction. A severe allergic reaction in an elderly patient with CHF and upper respiratory infection has the potential to be fatal. If participants missed correcting the wrong fluid rate, a patient with CHF could have worsening heart failure, which could, if left unchecked, ultimately result in death.

In the infant scenario, an infant (Gaumard Newborn HAL S3010) with respiratory syncytial virus and pneumonia is to receive fluids and penicillin. The patient’s monitor displayed the following: heart rate, 164 beats per minute; respiratory rate, 54 breaths per minute; blood pressure, 92/50 mm Hg; temperature, 38.4°C; and SpO₂, 94%. This scenario contained 2 embedded errors: (1) the wrong IV fluid is running (D5 1/2 NS instead of D5W 1/4 NS) and (2) oral penicillin (Penicillin VK PO, pink liquid in a labeled oral syringe), rather than the ordered IV penicillin, which is stocked in the medication cart. The correct IV fluid solution is available at the nurses’ station. The simulator made mild crying noises for approximately 10 minutes into the scenario for a short period. If the participants do not recognize and correct the incorrect IVrate in an infant, this could cause a sodium overdose, which could lead to renal failure. If participants do not recognize the wrong route penicillin supplied and they administer an oral medication intravenously, this would be a very serious error that would prove fatal.

The students performed the adult scenario, then the infant scenario. After completion of each scenario, we informed the participant of the embedded errors and whether they were missed or appropriately identified and corrected. Participants then read a summary of the potential consequences of each of the embedded errors if they had gone uncorrected (See Appendix 1).

After both scenarios, participants were given an AHRQ Web M&M Case, Death by PCA,²³ to read. This case was selected among several previewed in pilot testing because it involved a relatively conceivable error (related to PCA dosage choice and pump programming) that resulted in the death of a healthy young person. Debriefing included a brief discussion of the participant’s experience in the simulation and the potential consequences of missing embedded errors. Participants were given an opportunity to ask questions and to comment on the experience. The facilitator kept responses to a minimum so as not to unduly influence participant attitudes. Details of the participant comments in the debriefing were not formally documented as experimental data.

Movie Intervention

Nursing education related to error reduction frequently includes material such as lecture-based content on types and frequency of errors and movies that present stories by patients and clinicians of error experiences. We designed a comparison or control condition that included watching movies with stories from third parties (patients and clinicians) about error experiences. This choice supports the comparison of (1) the simulation intervention to current practice and (2) direct personal experience with error situations (simulation) to indirect third-person delivery of error experiences (movies and common current practice). Formalizing a comparison condition, rather relying on comparison with current educational activities, allowed us to (1) enroll students at different phases of training (sixth, seventh, and eighth semester) independent of the timing of their error-related training and (2) control for threats to validity related to experiment procedures.²⁴ Both groups participated for a similar amount of time in an activity that they may reasonably expect would change them in some way (even if it was not significantly different from training they received in other settings).

On the basis of personal experiences of the authors with nursing and patient safety curriculum, we selected 3 movies presenting high-consequence medical error stories from clinician and patient advocate perspectives. We selected the following movies: (1) “Beyond Blame,”²⁵ which portrays the consequences of medical errors from the perspective of the clinicians involved; (2) a Sue Sheridan Patient Safety video,²⁶ in which Ms Sheridan describes her personal experience as a mother and wife of patients who were harmed by medical errors; and (3) Dennis Quaid’s presentation to the National Press Club,²⁷ which tells the story of the drug error that harmed his newborn twins. The movies totaled for approximately 40 minutes of content. This was less than the time spent in simulation, but we believed it to be enough for participants to perceive it to be a meaningful intervention. Participants watched the movies in sequence in a classroom in groups of 1 to 5 students.

After the movie, the facilitator conducted a minimal debriefing, simply asking participants what they thought of the movies and whether they had seen them previously. In-depth discussion was avoided to reduce any influence of the facilitator on student perceptions.

Participants

Participants included 58 prelicensure (sixth, seventh, and eight semester) nursing students. All students had experienced 2 semesters of academic work including simulation and clinical hours in local healthcare facilities, clinical rotations in the medical-surgical setting in local healthcare facilities, and basic error prevention skills education in 2 previous semesters. Participants in pilot study activities were excluded from participation. Participants were divided into strata by graduation date. Within these strata, each time 2 individuals from the same graduation class volunteered, a coin toss was used to assign one to the movie condition and one to the simulation condition. In a small number of cases (5/58), scheduling or no shows influenced the assignment to either simulation or movies. We are not aware of any specific or systematic issues that could have influenced condition assignment in the few cases where we diverged from true randomization.

Procedures

Volunteers were e-mailed consent information and a PowerPoint style presentation on medication safety. They were asked to complete an online risk perception survey. When participants arrived to participate, they were informed of the study, risks, and benefits and written consent was obtained. A script was used to ensure consistency in the language introducing the experiment. On the basis of pilot test findings, we told simulation participants that the simulations may expose them to situations that are not routine and that they should feel free to speak up or ask questions. We told participants that we were not judging or testing their knowledge or skills but were interested in whether the simulations impacted the way they think and feel about risk in nursing. Participants completed their assigned intervention, followed by the postintervention survey.

Dependent Measures

To measure the impact of the simulation and movie experiences on trainee’s attitudes, we designed a survey that evaluated (1) memorability of the experience, (2) perceived frequency of adverse events, and (3) perceived caution in comparison with peers (see Appendix 2). We chose to measure perceived memorability (#3 in Appendix 2) to evaluate our hypothesis that simulation experiences would lead to deeper encoding of the experience in memory than movie experiences. We generated the wording of the question and validated it with 17 participants in the pilot test described previously. As evidence of content validity, responses to this question corresponded with qualitative feedback of participants. This question also differentiated less experienced participants (found the experience more memorable) from more experienced participants and differentiated solo participants (found the experience more memorable) from pairs.²²

We iteratively developed the frequency and caution questions during the pilot study and during the development of a related survey of practicing nurses’ experience with adverse events and risk attitudes (not yet published). We framed these questions in the context of specific error types. The error types were chosen to generate responses that were both directly and indirectly related to intervention content and were selected through literature review, nursing interviews, and a review of the Trinity Health adverse event database. The error types were (1) medication and IV fluid errors, (2) wrong patient, wrong procedure, and wrong site errors, as well as (3) falls.

We chose to measure perceived frequency of adverse events (numbers 4–6 in Appendix 2) because decisions to comply with risk reduction strategies are likely to be grounded in perceived frequency of events. We chose to measure perceived caution in relation to peers (numbers 7–9 in Appendix 2) because it is expected to reflect both intention to comply with risk reduction strategies and likelihood of behaviors such as “slowing down when you should” in comparison to peers’ behavior as a frame of reference. Asking more directly whether one intends to comply with risk reduction practices or intends to pay attention in risky situations was not expected to yield reliable or sensitive responses.

We pilot tested the wording of both questions with 6 practicing nurses who responded to these questions (among others) and then participated in an interview evaluating whether the questions were understandable and whether the interpreted meaning matched our intended meaning. Data from a survey of 278 nurses across 3 hospitals revealed a Cronbach α value of of 0.53 for the frequency question and 0.85 for the caution question across different error types. Post hoc analysis from our experiment also indicated stability of responses over time (Cronbach α value ranged from 0.68 to 0.88 for frequency and caution questions across time within error types). Post hoc analysis indicated that the frequency and caution questions measured distinct constructs with no significant correlations between survey responses to the 2 questions.

We measured memorability of the educational experience immediately after the experience. We measured perceived frequency and caution in comparison with peers immediately after participation, 3 months after participation and 6 months after participation.

We also surveyed participants regarding the realism of the simulation and whether they felt they were being set up for errors. We documented whether participants in the simulation condition identified each of the embedded errors in the 2 scenarios.

Hypotheses and Analysis

We hypothesized that both interventions would change risk attitudes compared with baseline and that changes in attitudes would diminish over time. We hypothesized that compared with a current standard of watching movies about error experiences, simulated personal experiences with errors would have more significant and lasting impact on risk attitudes. We evaluated each of these hypotheses across the 3 metrics of perceived memorability, perceived frequency of errors, and reported caution in comparison with peers.

Comparisons across time within intervention conditions were made using nonparametric Wilcoxon signed rank tests. Comparisons between the 2 intervention conditions and between students with different experience levels or students with different numbers of missed errors in the simulation intervention were made using nonparametric Mann-Whitney U tests. Because of multiple comparisons, we report significant findings related to our hypotheses at a P value of less than 0.01. We report significant findings not related to experimental hypotheses (eg, to verify no differences between groups at baseline) at a P value of less than 0.05. Descriptive statistics (frequencies, medians, and means) were calculated for responses to survey questions and error detection performance.

RESULTS

Demographics and Baseline Survey Responses

Twenty-eight students participated in the simulation intervention and 29 students participated in the movie intervention. All 57 participants completed the pretest and posttest surveys. Although 7 participants responded about a month late (after multiple follow-up requests), all participants completed the 3-month survey. Four participants (2 movie participants and 2 simulation participants) did not complete the 6-month survey, and 2 responses were approximately a month late.

Before the intervention, the median response to expected frequency of errors in a typical 30-bed medical-surgical unit was 4 (a few times a month) for medication errors, 3 (about once a month) for wrong patient/site errors, and 3 (about once a month) for falls. The median response to perceived caution in comparison with peers was 6 (1 point more cautious on the scale than 5 “about the same”) for all 3 questions.

We analyzed pretest survey responses to determine whether the 2 intervention groups were similar at baseline. There were no significant differences between intervention groups in pretest responses to questions about how frequently errors occurred or how cautious participants thought they were in comparison with their peers. We also compared pre-test responses of the sixth-semester students (n = 32) with the seventh and eighth semester (n = 25) students to determine whether experience (classroom or clinical) may impact these attitudes. There were no differences related to caution in comparison with peers; however, the seventh- and eighth-semester students reported the frequency of medication error events to be higher than the sixth-semester students (P < 0.05).

Perceived Realism of Simulation and Simulation Error Detection Performance

Perceptions of realism and deception of the simulation intervention are presented in Table 1.

TABLE 1.

Perceptions of Realism and Deception for the Simulation Intervention

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree	No Response
I thought the simulation experience was realistic.	11	15	1	—		1
I felt as though I was being set up to make an error in a way that is not realistic.	—		3	16	8	1

For error detection performance, we did not capture data for 1 of the 28 participants. In the adult simulation scenario, 26 (96%) of the 27 participants identified the drug-allergy contraindication and 18 (66%) identified the incorrect fluid rate. In the infant scenario, 8 (30%) of the participants identified the wrong IV solution, and 18 (67%) identified the wrong penicillin route. Twenty two (81%) of the participants failed to identify at least 1 of the 4 embedded errors in the 2 scenarios. There were no significant differences between the sixth-semester students (n = 14) and seventh- and eighth-semester students combined (n = 13) with respect to error detection performance.

Memorability of the Education Experience

The simulation intervention was perceived to be more memorable than the movie condition (P < 0.005). Secondary analyses of these data revealed that this effect was driven solely by a difference in memorability of the intervention for the seventh- and eighth-semester students (P < 0.005, Fig 1). There was no difference in memorability between simulation and movies for sixth-semester students. There was no difference in memorability of the simulation intervention for students who failed to identify 0 or 1 errors (n = 14) compared with those who failed to identify 2 or 3 errors (n = 13).

FIGURE 1.

Mean responses to questions regarding perceived memorability of simulation and movie interventions by trainee experience level. *Significantly different at P < 0.005.

Perceived Frequency of Errors

After the intervention, participants in both conditions revealed changes from baseline regarding the perceived frequency of errors (Fig. 2). Simulation participants believed that medication and wrong patient/site errors occurred more frequently immediately after the education session (P < 0.01). These perceptions remained higher at 3 months for wrong patient/site errors (P < 0.01). Movie participants increased their perceptions of the frequency of falls immediately after the educational session (P < 0.01).

FIGURE 2.

Mean responses to questions regarding the perceived frequency of 3 different types of errors over time. Sim, simulation. *Significantly different than pretest scores, P < 0.01. Error bar displays standard error.

Comparing the 2 intervention conditions, immediately after the intervention, simulation participants had higher perception of frequency of medication errors (P < 0.001). Three and 6 months after the intervention, there were no significant differences between the 2 groups in the perception of frequency of errors.

Perceived Caution in High-Risk Activities

Figure 3 displays mean responses to perceived caution in activities to prevent errors. Immediately after the intervention, participants in the simulation condition increased their perceived caution for activities related to all 3 types of errors (P < 0.01). Six months after the intervention, caution was higher than baseline for simulation participants for medication errors (P < 0.01). For the movie condition, immediately after the intervention, participants increased their perceived caution for medication errors and wrong patient/site errors (P < 0.01). Six months after the movie intervention, caution was higher than baseline for medication errors (P < 0.01).

FIGURE 3.

Mean responses to perceived caution in nursing activities related to 3 different types of AEs over time. Sim, simulation. *Significantly different than pretest, P < 0.01. Error bar displays standard error.

With respect to comparisons between the 2 intervention conditions, there were no significant differences in reported caution between individuals in the movie condition and individuals in the simulation condition at any point in time.

DISCUSSION

All nursing students found the simulation experience to be highly memorable. The simulation experience, which included 2 embedded errors in 2 scenarios, exposed participants to the idea that errors are possible, to the variety of errors that can occur and to their own fallibility with respect to detecting such errors. Some embedded errors (eg, allergy-drug contraindication) were detected by more participants than others (eg, wrong saline solution running). However, most (81%) of participants missed detecting at least 1 embedded error. Sixth-semester nursing students also found the movies to be highly memorable. The movie intervention exposed participants to the extreme consequences of errors in care. The extreme emotional content of the movies may have had a strong impact on these students, who were less likely than more experienced students to already have seen these or similar movies.

Nursing students underestimated the frequency of medication errors and IV fluid errors. The median response to this question was “4 = a few times a month.” We estimate that the correct response is “6 = about once a day” (in a typical 30-bed medical-surgical unit).^1,28–31 More experienced students had a more realistic perception of the frequency of errors but still underestimated. Exposure to simulated error situations increased this perception immediately after the intervention to a near realistic interpretation, but the effect was short-lived. Movies had minimal (if any) impact on perceived frequency of events, perhaps because they focused on specific high harm events, which are rarer than errors in general. Although our prereading materials did include some statistics related to the rates of errors, this did not seem to influence the accuracy of participant responses to questions focused on evaluating rates of error in a meaningful context with respect to a specific care role. It is not clear whether a realistic interpretation of the likelihood of errors is important to promoting risk attitudes that lead to high compliance with risk reduction practices and controlled attention during high-risk activities. It also remains unclear whether simulated error situations, perhaps combined with high-quality educational content that places error statistics in perspective, may be helpful in achieving this goal.

Before the intervention, participants perceived themselves to be slightly more cautious than their peers in activities associated with error prevention. Immediately after the intervention, both groups increased their perceived caution in comparison with peers who had not participated in the intervention for medication and IV fluid administration and for checking patient identity. Six months after the intervention, perceived caution in comparison with peers remained higher for both groups for the medication/IV errors (the focus of the interventions). This provides evidence for a potential lasting impact of both interventions.

Based on our findings, there is evidence for a slight advantage of lecture and simulation-based error education over the more typical approach of lecture and movies. Further exploration of other approaches, such as using both movies and simulation, is warranted. Unfortunately, simulation education is time and resource intensive. Our simulation approach required approximately 30 minutes of technician setup time, and participants were involved for approximately an hour, including introduction, participation, and debriefing. Although this is not significantly greater than the time learners spent watching movies, in our approach, working with 1 learner at a time, the time required of facilitators or educators is much greater than the time required to moderate a large group movie session.

Although both simulations and movie experiences were memorable and impacted risk perception, it is not clear whether these attitude changes would result in behavioral or performance changes. The argument that simulation experiences would lead to deep encoding of the events in memory is somewhat supported by our findings in that simulation participants scored their memory of the experience as 8 of 9, where 9 was “one of the most memorable experiences in my education.” Unfortunately, we did not ask about the memorability of the experience at a later time, which may have provided a better indicator of whether the experience was ultimately encoded deeply. In our simulations, we stopped short of simulating extreme clinical and emotional consequences of errors, partly because of concerns for the psychological stress that this would create for participants. However, emotional and stressful situations are linked to deeply encoded memories that impact behavior.^14,15,17 It is not clear from our work whether our simulations impacted emotional memory this way. It is possible that more extreme circumstances would be required to achieve this, and if so, an important question is whether the benefits outweigh the costs.

There is evidence that attitude changes can impact intentional compliance with risk reduction strategies.^19–21,32 Our findings suggest that both movies and simulated error experiences would contribute to changes in attitudes, which would then need to be combined with education on error prevention strategies, policies, and technologies to ultimately prevent adverse events. There may be more economical approaches to integrating simulated error experiences as part of a structured education program such as adding error situations into simulations that already exist and are used for other purposes or by creating brief “in situ” experiences that involve less time and resources. It is not clear whether these types of activities would have the same impact as the tested approach. If trainees are focused on learning other new things, will the error experiences get the attention deserved? We purposefully designed scenarios with error situations that could lead to both immediate problems (allergy and wrong route) or problems that would manifest over time (wrong fluid rate and sodium concentration), involving patients for which consequences could be severe. We also ensured that the potential severity of the consequences was conveyed. Details such as these may be important to the effectiveness of simulated error experiences presented in other ways.

Other questions remain related to how simulation should be integrated with error education programs. Are simulated error experiences most useful for students who have had limited clinical experiences to influence early attitude formation and so they are prepared when they enter clinical settings? Are they useful for practicing nurses who may be more susceptible to problems of complacency? Are repeated exposures helpful,^33,34 and if so, how many or how often? Our study provides limited evidence of generalizability to other error types. How important is the depth and breadth of error content covered?

This study has limitations beyond those already mentioned. We drew from a limited sample of student volunteer participants, thus limiting generalizability. The movies for the comparison condition were not selected using a formal process. A different selection may have had different results. Although the longitudinal nature of the study resulted in some understanding of the impact of these forms of education over time, we did not measure confounding factors, such as whether study participants encountered errors in practice. Facilitated debriefing was kept to a minimum for both interventions, so as not to bias comparisons between interventions. It is not clear how more in-depth debriefings may have impacted findings. We did not formally document participants’ qualitative feedback. We did not directly address whether simulation participants’ feelings of deception may have impacted their subsequent attitudes, both with respect to the goals of the current study and future simulation experiences. Most importantly, we did not evaluate the impact of education on subsequent student behaviors, performance, or patient outcomes. Although there is a theoretical basis to suggest that changes in attitudes will be reflected in performance, there is still limited evidence that links changes in the specific attitudes we measured to subsequent behaviors related to high-risk health care activities.

CONCLUSIONS

Our study provided limited evidence of an advantage of simulation over movies incorporating stories about errors with respect to changing attitudes related to error prevention. Simulations made participants more aware of how easily errors can occur and the movie education made participants more aware of the devastating consequences of errors. Our findings suggest that exposure to errors via simulation and/or review of error consequences through movie or other methods may sensitize clinicians to the risk of errors in a way that heightens their sensitivity to errors. It is not clear whether this will impact their long-term error prevention practices. Future research is warranted to further explore the value of simulated error experiences as an approach to reduce rates of preventable adverse events.

APPENDIX 1: Error Consequences Reading

Simulation Error Consequences

William Jones, ordering error and rate error.

In the scenario you performed, there was an error in the order for the patient’s medication. For example, a medication was ordered for which the patient had an allergy, a medication was ordered at a dose or concentration that was not appropriate for the patient, or a medication was ordered that had a contraindication with another medication the patient is taking.

Ordering errors are relatively common in health care. In children, the prescription error rate is as high as 5.7% (meaning that as many as 1 of every 17 prescriptions can have a mistake; Institute for Safe Medication Practices). Ordering errors can have a range of outcomes from no impact to death. In the former case, errors may well go undetected, which can lead to a false sense of security.

In the case of Mr Jones, a patient with congestive heart failure and upper respiratory infection, receiving a dose of penicillin could lead to a range of outcomes. At one end, the patient may have no reaction or a relatively innocuous reaction such as a case of hives. Even in such cases, however, patient trust and confidence can be significantly degraded when a patient receives a medication for which he has a documented allergy. At the other end of the spectrum, the patient could have an anaphylactic reaction, which combined with his congestive heart failure and upper respiratory infection could significantly exacerbate both respiratory and cardiac problems leading to escalation in care (need to call medical or rapid response team, transfer to intensive care unit, and more invasive procedures such as intubation) or even death.

In this scenario, there was also an administration error that was made before your transitioning to care for the patient. As many as one fifth (Han et al,² 2005) of all continuous infusions have some type of error, with the most common being an error in the rate of infusion. Like ordering errors, infusion errors frequently go undetected, leading clinicians to believe that accuracy may be higher than it actually is. In many cases, rate errors may have minor or no consequences. However, they may also have disastrous consequences.

In the case of underinfusing fluids to Mr Jones, a CHF patient with upper respiratory infection, underinfusing fluids could lead to problems such as dehydration. The longer a rate error continues for a patient such as Mr Jones, the more extreme the consequences may be. Significant negative effects caused by dehydration can include renal failure, which may require an escalation of care (transfer to intensive care unit, dialysis) and, if not managed appropriately, could lead to death.

Billy Smith, administration (fluid) error, stocking confusion.

In this scenario, there was an administration error that was made before your transitioning to care for the patient. As many as one fifth (Han et al,² 2005) of all continuous infusions have some type of error, with the most common being an error in the rate of infusion. Like ordering errors, infusion errors frequently go undetected, leading clinicians to believe that accuracy may be higher than it actually is. In many cases, rate errors may have minor or no consequences. However, they may also have disastrous consequences.

In the event of a 3 month old receiving fluids with an incorrect saline solution, the patient may receive an overdose of sodium. Although this may be easily tolerated in an adult, a pediatric patient with developing kidneys may not be able to handle the additional sodium and could suffer renal failure. The severity of the outcome depends largely on how long the error goes undetected. The patient may require an escalation in treatment, more invasive care, longer stay, or even death.

This scenario also had embedded an opportunity for confusion in the administration of penicillin. The physician ordered intravenous penicillin, but only oral penicillin was available on the medication cart. There are different potential errors that could occur if a clinician did not identify and obtain the ordered intravenous medication. If the clinician gave oral penicillin orally, the clinician would not have correct dosing information and the effects of the drug would be delayed from that intended via IV route by the physician. This could lead to either overdose and associated harmful impacts (nausea, vomiting, diarrhea) or delayed treatment, which may result in treatment delays or greater severity of the patients’ illness. If the clinician mistook an oral liquid for intravenous medication and delivered it intravenously, the patient would suffer severe harm requiring escalation in care and possibly death.

APPENDIX 2: Survey Questions

Survey Questions

For the purposes of this survey, we are defining patient care error events, hereafter referred to as “errors,” broadly as: “Situations caused by human error or system deficiencies that result in patient harm or increased risk of patient harm. They include mistakes, omissions, and failures to follow procedures by you or other care givers. They also include errors or deficiencies in orders, policies, systems, or technologies. They include near misses and situations where an error reaches the patient, whether or not the error causes harm.”

Please indicate your perceptions by circling the appropriate option. There are no right or wrong answers. Please answer honestly based on your personal feelings.

[Simulation participants only and only for the immediate post survey]

• 1.I thought the simulation experience was realistic.

Strongly agree

Agree

Neutral

Disagree

Strongly disagree

• 2.I felt as though I was being “set up” to make an error in a way that is not realistic.

Strongly agree

Agree

Neutral

Disagree

Strongly disagree

[Simulation and movie participants, only for the immediate post survey]

• 3.How memorable (likely to stick) was this educational (prereading and [simulation or movie]) experience?

1   2    3   4   5   6   7   8   9
Not at all memorable						One of the most memorable experiences in my nursing training

(Questions asked preintervention, immediately postintervention, 3 months after, and 6 months after]

• 4.How frequently do you believe medication and IV fluid errors occur in a typical 30-bed medical-surgical unit?

About once every few years

One or more times each year

About once a month

A few times a month or about once each week

A few times each week

About once each day

More than once every day

• 5.How frequently do you believe wrong patient, wrong procedure, or wrong site errors occur in a typical 30-bed medical-surgical unit?

About once every few years

One or more times each year

About once a month

A few times a month or about once each week

A few times each week

About once each day

More than once every day

• 6.How frequently do you believe falls occur in a typical 30-bed medical-surgical unit?

About once every few years

One or more times each year

About once a month

A few times a month or about once each week

A few times each week

About once each day

More than once every day

(For the immediate post survey, the language for the caution questions was modified slightly to frame as “Following this education, compared to peers who did not participate.”]

• 7.Compared to your peers, are you more or less cautious in the process of medication/IV fluid administration?

−4	−3	−2	−1	0	1	2	3	4
Much less cautious				About the same	Much more cautious

• 8.Compared to your peers, are you more or less cautious in checking patient identity before an intervention?

−4	−3	−2	−1	0	1	2	3	4
Much less cautious				About the same	Much more cautious

• 9.Compared to your peers, are you more or less cautious in protecting patients against falls?

−4	−3	−2	−1	0	1	2	3	4
Much less cautious				About the same	Much more cautious