Root Cause Analysis Design and Its Application to Pharmacy Education

Abstract

Objective. To evaluate third-year pharmacy students’ ability to effectively design and apply root cause analysis (RCA) to a sentinel event and to analyze student responses for areas of proficiency and deficiency.

Methods. This study involved a comprehensive review of RCA key assessments from 82 students in spring 2014. The performance assessments consisted of a review of each student’s RCA worksheet and flow diagram as well as the 1-2 page narrative reflection.

Results. The majority of students were able to successfully design an RCA, but had challenges with 3 components: fact-finding session, disclosure plan, and a flow diagram. Several students perceived challenges to conducting a formal RCA in certain health care settings, which included assembling an RCA team and gathering relevant facts, a lack of transparency/safe environment within organizations, and creating a plan for multiple/sequential errors.

Conclusion. Most students were able to design an RCA process, but encountered difficulty with some components. Students also identified challenges in applying the approach in certain health care arenas.

INTRODUCTION

In 1999, the Institute of Medicine reported that approximately 100 000 people die every year from preventable medical errors.¹ In 2010, the Department of Health and Human Services reported that poor hospital care was a contributing factor in 180 000 deaths for Medicare patients alone in a given year. In a 2013 Journal of Patient Safety study, James estimated that anywhere between 210 000 and 440 000 patients each year suffer some type of preventable medical event that eventually contributes to their death.² In fact, medication errors were the third leading cause of death in the United States. Considering these statistics, the Accreditation Council for Pharmacy Education (ACPE) guidelines for pharmacy school curriculum include patient safety components.³

To date, limited work has examined the integration of patient safety curricula into health professions education.^4,5 The most comprehensive review noted that research on instructional methods necessary to guide curricular change in patient safety is needed.⁵ In terms of patient safety instruction in pharmacy education, 2 studies provided information on current practices.^6,7 West-Strum et al found that, across 107 different pharmacy schools, safety science subjects were integrated into the majority of schools, but certain gaps were identified (eg, risk communication, FDA Sentinel Event Initiative).⁶ The safety science domains explored in their study encompassed several FDA initiatives including boxed warnings, post-marketing surveillance, and product safety communications with a focus on adverse drug reactions. However, the study was not an examination of instruction regarding preventable medication errors or actual formal investigational techniques (eg, RCA, failure mode, and effects analysis) to examine preventable events.⁶ Gilligan et al conducted an implementation assessment of a formal program (the EPIQ Curriculum) to educate pharmacy students on quality improvement at 7 pharmacy schools.⁷ Their findings supported the impact of the EPIQ program on students’ perceived knowledge of quality improvement. When examining student perceptions of knowledge gained, students rated the “ability to implement methods to reduce medication errors” as the lowest among the 9 items tested. This study did not determine actual student performance regarding patient safety practices either and did not examine student understanding of investigative techniques.⁷

One instructional method to address patient safety in pharmacy education is the application of RCA, a problem-solving method that identifies root causes or faulty system components contributing to an error rather than only addressing proximal causes. It can be applied as a retrospective approach to understanding why a medication error occurred.^8,9 Recent work in RCA application for errors at community pharmacies in Denmark demonstrated this technique was useful in examining patient safety.¹⁰

Published work on modeling or practicing the RCA process at pharmacy schools is sparse. Schafer et al implemented an RCA activity into a required medication safety course.¹¹ In this learning exercise, students were provided with predefined sentinel events, but they did not apply the RCA process to common areas of pharmacy practice. The instructors employed a team-based approach to allow students to practice the method in small groups, which focused on key elements to determine root causes. The formal RCA process was not designed, practiced, or followed explicitly. This approach allowed students, as basic participants in the RCA process, to develop a general understanding of the multitude of causes for a particular error. However, an RCA assignment done in groups rather than individually can limit the instructor’s ability to ascertain if each student understood the process. In addition, it is unclear whether students gained a deeper understanding because they were not asked to individually design or apply the formal steps of the RCA process. Since this study did not include the contributions of different disciplines to an RCA team, it may not have allowed students to fully appreciate the reasons behind the multidisciplinary approach when conducting an RCA.

To address the continuing problem of patient medication safety in alignment with the ACPE Standards, the University of New Mexico College of Pharmacy core curriculum includes a medication safety course for third-year pharmacy students. The course, which includes in-class lectures, out-of-class readings, examinations, and key assessments, allows students to develop fluency in discussing both historical and contemporary medication safety evidence and practices and to become advocates for effective approaches to improve patient safety in practice settings. For one of the key assessments, students are tasked to either identify or create a hypothetical preventable sentinel event with a medication error in the health care system and then design a comprehensive RCA to examine the etiologies of the event. The purpose of this study was to analyze the results of the RCA key assessment and to identify areas of proficiency and deficiency in students’ abilities to effectively design an RCA process and apply it to a sentinel event.

METHODS

The goals of this assignment were for students to gain better understanding of the multiple causes of sentinel events and of the RCA process and to assess how the process could work in the practice arena they chose to study. As noted in medication safety textbooks, the investigation of most sentinel events in health care end with a determination of human error and possible action against the front line operator rather than a deeper investigation into the chain of contributors to the event.^8,9 For our educational intervention, we reasoned that students completing the assignment individually would have a more comprehensive understanding of the actual RCA process from initial design to development of an action plan and final disclosure. Learning would be achieved by applying RCA, which would allow students to consider multiple causes of such an event. The overall goal was to not only develop fluency in the RCA process, but also help students understand the reasons why traditional approaches to medication errors (eg, invoking punitive action against front line providers) are flawed as they do not consider the multitude of failure nodes within the medication use process.

The instructional exercise included a lecture and reading assignment (mid-way through the course) specific to the components and application of the RCA process. The key assessment consisted of an RCA design for a sentinel event of each student’s own choosing to optimize interest in applying the RCA. Students were also required to provide a narrative reflection on their experience.

Students were provided with a worksheet listing the RCA steps, a sample flow diagram to guide the RCA process, and a rubric detailing how the assignment would be evaluated. The RCA components were taken from standard medication error textbooks that included chapters covering the process in detail.^8,9 Each item on the worksheet was assessed using a 5-point scale rubric (excellent, very good, meets expectations, needs improvement, not acceptable). For the narrative reflection, students were required to detail challenges they encountered with the assignment and any additional insight they developed. The narrative was graded as a single item using the same 5-point scale. Individual student grades for the key assessment were determined by applying the rubric to the RCA worksheet and flow diagram (45 points), as well as by reviewing the narrative portion (5 points). Students were given 8 weeks to complete the assignment.

This study involved a comprehensive review of collected and anonymized RCA key assessments of students. Student assignments were randomized and then assigned to 2 of 5 different reviewers so each assignment was independently reviewed by 2 unpaired reviewers. The data examined for each assignment consisted of 3 parts. First, student response to each component of the RCA worksheet was reviewed to determine the adequacy of the response in addressing each component. Student responses were scored on a 5-point Likert-type scale (1=unable, 2=below expectations, 3=able or met expectations, 4=above expectations, 5=excellent).

Reviewers then conducted an overall assessment of each RCA process presented and provided answers to 4 questions: (1) In what setting did the sentinel event occur? [We hypothesized that RCA within the inpatient setting would be more complete than those addressing events in the outpatient setting. Possible reviewer responses: inpatient or outpatient]; (2) Was the sentinel event correctly identified? [We hypothesized that correct identification of the sentinel event would depend on student ability to adequately describe the RCA components. Possible responses: yes or no]; (3) Was the RCA complete and adequately presented? [We hypothesized that a complete and adequately presented RCA would depend on student ability to adequately describe the RCA components. Possible reviewer responses: yes or no; (4) Was more than one root cause identified? [Given that more than one root cause usually contributes to the sentinel event, we hypothesized that identification of more than one root cause would depend on student ability to adequately describe the RCA components. Possible responses: yes or no]. Response data for these 4 questions are shown in Table 1. Lastly, the reflective narratives were reviewed to identify challenges, themes, and unanticipated findings that were then collapsed into qualitative categories for summary reporting. The study was reviewed and approved by the UNM Human Research Review Committee as exempt educational research.

Table 1.

Assessment of Student Ability to Conduct a Root Cause Analysis

The 2 reviewer scores for each student assignment component were averaged. Internal reliability, using Cronbach alpha for the 10 components, was 0.871, an acceptable indication of scale reliability. Component scores were then sorted into “able” (score of 3 or more) and “not able” (score less than 3) to estimate overall student ability to describe the RCA process steps in the worksheet. A chi-square test was run to determine the dependence of the student RCA component description on the practice setting. Student description of the RCA components was tested for dependence on student ability to describe a sentinel event, to identify more than one root cause, and to design an RCA. Assumptions for chi-square testing were met and the significance level set at 0.05. The data and significance of these associations are shown in Table 2. Descriptive and quantitative analysis was conducted using SPSSv22 (SPSS IBM, Armonk, NY).

Table 2.

Student Abilities to Correctly Identify Root Cause Analysis (RCA) Components

A general inductive approach was used to analyze narrative reflections.¹² Reviewers read the reflective narratives several times and noted challenges reported by students in completing the RCA process. Reviewer notes were examined and grouped into categories, which were then subjected to an iterative process of analysis and coding for refinement of recurrent themes. The frequency of challenges by theme is reported in Table 3. Recurring themes in student comments unrelated to the challenge of the RCA process (eg, role of patient counseling in preventing sentinel events) were also analyzed. These are also reported in Table 3 as unanticipated findings.

Table 3.

Thematic Analysis Results from Student Narratives

RESULTS

Eighty-two students completed the assignment. Fifty-one (62.2%) chose to study an event within the inpatient or institutional arena, and the remaining 31 (37.8%) chose the outpatient setting. The overall assessment demonstrated that more than 70% of students could correctly identify a sentinel event, consider more than one root cause for the event, and successfully design an RCA for a given practice setting (Table 1). The main findings from the 10 different RCA component analyses are presented in Table 2. Overall, the majority of students were able to successfully complete each of the RCA components. However, the majority of students were not able to adequately describe the RCA component for 3 items. Only 34% of the students were able to adequately articulate the development of a fact-finding session (item 5). Only 48% successfully presented a logical flow diagram of the steps that could have been contributors to the event (item 7). The final step of the RCA process, communicating the findings with a disclosure plan (item 10), challenged all but 24% of students.

The chi-square analysis (Table 2) was used to address the association of responses to the 4 questions with student ability to adequately describe RCA components. There was no significance in student ability to respond to RCA components based on practice setting. There was a significant association of student ability to adequately describe the RCA components for 5 of the 10 items and the other 3 parameters: identification of the sentinel event; ability to successfully complete the RCA process; and identification of more than one root cause.

The results of the reflective narrative analyses revealed 6 items students identified as challenges in completing the RCA process as well as 5 unanticipated findings (Table 3). In particular, identifying and assembling an RCA team and gathering the relevant facts were noted as challenges by at least a third of the students. Several students (22%) also perceived a lack of transparency or safe environment in the health care setting. Developing a plan for multiple/sequential errors (21%) was also a challenge to the RCA process. Major unanticipated findings were student choices to study an event based upon an actual incident (24.8%) and student perceptions of corporate control issues (14.6%) that might impede such investigations.

DISCUSSION

This curricular evaluation study revealed a number of findings regarding the practical teaching of the RCA process within a pharmacy school curriculum. The majority of students were able to adequately design an RCA, but approximately a third of the students had significant difficulties with components involved in the process. In particular, the majority of students could not adequately design a fact-finding session that included a neutral outsider and a detailed closure plan. Many students also did not adequately detail the review process in a logical flow diagram. Although it is unclear why deficiencies were demonstrated with these RCA components, one potential explanation is that some students’ limited pharmacy experiences may have prevented them from effectively translating the didactic material to the application assignment.

Two findings responsible for the deficiencies in items 5 (fact-finding session) and 10 (closure) were evident in reviewer assessments. For item 5, the lack of details on the fact-finding session frequently encompassed inadequate details regarding the interview process and lack of a neutral outsider. For item 10, students often did not include a disclosure plan beyond the health care personnel, so review of the findings with the patient/family was lacking. The latter deficiency may have been indicative of the status of the health care system regarding lack of routine disclosure following such events. These portions of the RCA process can be further emphasized when this material is presented and practiced within the curriculum. Chi-square analysis revealed that student ability to respond to RCA components varied by item and was related to overall performance success. This is the first evidence of which we are aware that indicates such components may be critical for students to grasp in order to achieve competency. In addition, identifying the most challenging components for students is important for constructing further classroom exercises on this process and for examining more precisely defined parameters to facilitate stronger quantitative analysis.

In terms of narrative findings, students noted that assembling the RCA team and gathering the relevant facts were the most common challenges. This could reveal that students perceive difficulty in health care settings lacking multidisciplinary team and in which providers are isolated from each other. Another challenge commonly perceived by students was the belief that health care organizations would not provide the supportive environment or transparency necessary to complete the RCA process. This suggests that some students believe safety culture may not yet be fully embraced in certain health care settings. The third major challenge that students encountered was in designing an RCA for multiple/sequential errors. This is indicative of the complex process in analyzing such events, and likely demonstrates that students working through a formal investigative technique were becoming more adept in visualizing multiple causes. Because all of the students were registered, presumably practicing interns at this point in their training, their perceptions may be valuable in interpreting the application of the RCA process to pharmacy practice.

Twelve students reporting on incidents in community pharmacy perceived that corporate culture issues were likely to be significant obstacles to initiating such a process. Six students interpreted business priorities of chain retail pharmacy focused on speed, efficiency, profit, and customer satisfaction as being more important to the organization than patient safety. Three of these students also perceived that organizations would not be sufficiently transparent regarding preventable sentinel events because of legal liability, thereby precluding a formal RCA process. Three students also perceived that corporate policy was to document such events for action against employees rather than for learning or conducting a formal RCA process. Six students who had some familiarity with organizational reporting systems commented that such reporting had created fear of punitive action. Despite working through an RCA process, 2 students retained the belief that punitive action was an appropriate response for some disciplines (eg, nursing) closely involved with the event. In addition, 4 students noted that after designing an RCA, they understood it would be much easier for a health care system to invoke punitive action for those involved rather than to study the event with a formal multidisciplinary process and address how the system was responsible for the event. These observations may be useful in structuring future class activities as well as designing additional research studies.

Some positive and/or unexpected findings emerged from the narrative component. Twenty students reported on actual events from their workplace, often contrasting the RCA process with what actually occurred. Eleven students indicated that working on an RCA process helped them consider multiple causes for such an event. Nine students who designed an RCA process for an outpatient arena also noted that thorough patient counseling was an important safety procedure that should be required. Two students who examined actual events also disclosed that after completing the process, they realized punitive action taken in the case was inappropriate because of the multiple causes involved.

There are no published studies of actual pharmacist understanding, perceptions, or challenges regarding implementation of the RCA process. Student beliefs that a corporate health care culture in an outpatient setting negatively impacts a successful RCA process suggest that cultural transformation in health care may be necessary to allow for a different approach to medication errors. Student belief that medication error investigations are commonly used to invoke punitive action in the outpatient setting against health care professionals is troubling. This belief is also supported by a recent finding that punitive action against pharmacists is a common response taken by pharmacy boards.¹³

While there were no differences in overall assignment success between the inpatient and outpatient/retail environments, 9 students who chose the outpatient environment noted the RCA process was difficult to accomplish in this setting. Students noted the difficulty in assembling the necessary RCA team because of logistical problems such as lack of proximity and established professional relationships among potential team members. This challenge is understandable, given the isolation of individual health care workers from each other in the outpatient environment. The relative isolation of community pharmacists from other health care providers is a barrier not only to an RCA process, but also to developing a broader provider role for pharmacists in the health care setting. Including interprofessional education (IPE) in health care curricula that specifically addresses limitations involving such outpatient practice silos could be helpful. Practicing RCA in the community setting could shift students’ thinking toward developing a more collaborative practice approach in this environment.

Although overall student success was 70%, approximately one third of students were not able to adequately design an RCA. Moreover, 22% of students were unable to identify a sentinel event, and 23% didn’t consider more than one root cause (23%). Clearly, some students did not fully grasp how the RCA process identifies the chain of causality that leads to a particular event. This suggests limitations with the formal RCA instruction and/or with the practice aspects of the assignment that required students to work individually. While we still see value in allowing students to individually develop and integrate foundational concepts through the design of an RCA process for an event of interest to them, practicing the process in interprofessional teams may provide a richer understanding of the components involved and identify remaining learning deficits (eg, multitude of causes, full disclosure plan).

Descriptions of RCA instruction and/practice have been published for other disciplines.^14,15 In nursing, proposed RCA instruction included a mock methodology after an error in the clinical setting (ie, random occurrence) and a standardized approach employing a simulation setting.¹⁴ The mock methodology was used with students involved with specific errors during their training. This was valuable for the students and for the department administration to examine a deeper set of reasons for the occurrences, but the random nature and rare occurrence of such events could not create a learning opportunity for most students. In the standardized simulation approach, students worked on case scenarios during their clinical practice experiences and those who made errors participated in an RCA exercise. Such simulation exercises are excellent for modeling high-risk situations in which RCA is an important component of the error debriefing process. This method allowed for more student participation, but the authors didn’t provide details on student understanding of the actual RCA process or its individual components. In another study of RCA instruction among midwifery students, students examined actual cases to promote critical-thinking skills within the paradigm of clinical decision-making.¹⁵ In this study, students worked in small groups (3-4) to examine different case scenarios. Students completed surveys after the exercise to evaluate its acceptability, educational impact, and preparation for practice. Students viewed the exercise as a positive instructional technique, especially regarding its role in promoting critical-thinking skills. Again, evaluations of actual student performance and understanding of the RCA process were not presented. While the use of RCA instruction for further development of critical thinking was not the purpose of our study, this midwifery education approach demonstrates additional value of employing such educational strategies in a health professional curriculum.

The paucity of published studies regarding the use of RCA as an instructional technique for health professions students is striking. The RCA process is not new, but its use and practice during formal instruction remains novel. If used and modeled appropriately, this approach could help students gain a deeper understanding of the multitude of causes of errors and the limitations of current approaches that employ punitive action. By being involved in such exercises, students could observe the value of a systematic approach to error analysis that remediates and refines systems rather than individuals. This learning model also may assist with further development of critical-thinking skills. Additional research could involve a multi-disciplinary learning exercise with at least pharmacy, medical, and nursing students to allow for the building of team components to support future practice models. Such a multi-disciplinary practice may also help teach pharmacy and other health professional students nuances of the RCA process and its deficits. Team-based instruction/modeling could also be practiced more broadly by including other health care and/or professional disciplines (eg, law school students), where appropriate.

The study did have several limitations. The first is that only one class of students at the college was evaluated using the current RCA curriculum and assignment. In addition, although the ability of students to complete an RCA in a controlled classroom assignment environment was evaluated, it is unclear whether students would be more likely to complete an RCA in the experiential setting or as a new practitioner. Moreover, conducting this type of key assessment with individual student-generated RCA designs was laborious for faculty members in terms of evaluation. We recommend providing students with clear instruction and a worksheet covering RCA components as a guide to successful completion. While students deriving their own events may be a potential study limitation, our findings suggest otherwise. We designed the assignment to allow students to choose their own event and take ownership from the beginning. By allowing them to identify an event important to them, we assumed students would be motivated to think more deeply about the causality than if the event occurred in a context not of interest to them. For example, we noted 8 students chose to study heparin overdose events, but each event was different. Twenty-four students selected a sentinel event based on the death of a family member, on observation of such an event in the workplace, or on personal interest in a practice area. The lack of granularity of data prohibited significant analysis of these findings. Future work should consider a survey to better yield generalizable information.

CONCLUSION

The majority of students were able to design a formal RCA process and to complete most of the detailed steps involved. However, students encountered difficulty with some aspects and also noted that application in certain health care systems could pose difficulty because of perceived cultural barriers. The study implications include the usefulness of such an educational exercise to broaden student understanding of the multitude of causes for sentinel events, as well as consideration for a broadened application of this educational strategy for interprofessional activities that model actual multidisciplinary involvement.