Improving Patient Safety Event Reporting Among Residents and Teaching Faculty

Abstract

Background:

A June 2012 site visit report from the Accreditation Council for Graduate Medical Education Clinical Learning Environment Review revealed that residents and physicians at TriHealth, Inc., a large, nonprofit independent academic medical center serving the Greater Cincinnati area in Ohio, had an opportunity to improve their awareness and understanding of the hospital's system for reporting patient safety concerns in 3 areas: (1) what constitutes a reportable patient safety event, (2) who is responsible for reporting, and (3) how to use the hospital's current reporting system.

Methods:

To improve the culture of patient safety, we designed a quality improvement project with the goal to increase patient safety event reporting among residents and teaching faculty. An anonymous questionnaire assessed physicians' and residents' attitudes and experience regarding patient safety event reporting. An educational intervention was provided in each graduate medical education program to improve knowledge and skills related to patient safety event reporting, and the anonymous questionnaire was distributed after the intervention. We compared the responses to the preintervention and postintervention questionnaires and tracked monthly patient safety event reports for 1 year postintervention.

Results:

The number of patient safety event reports increased following the educational intervention; however, we saw wide variability in reporting per month. On the postintervention questionnaire, participants demonstrated improved knowledge and attitudes toward patient safety event reporting.

Conclusion:

The goal of this unique project was to increase patient safety event reporting by both residents and teaching faculty in 6 residency programs through education. We achieved this goal through an educational intervention tailored to the institution's new event reporting system delivered to each residency program. We clearly understand that improvements in quality and patient safety require ongoing effort. The keys to ongoing sustainability include (1) developing patient safety faculty and resident experts in each training program to teach patient safety and to be role models, (2) working toward decreasing the barriers to reporting, and (3) providing timely feedback and system changes.

INTRODUCTION

The Joint Commission and the Accreditation Council for Graduate Medical Education (ACGME) place strong emphasis on timely reporting of patient safety events and mandate postgraduate educational programs to provide training in quality improvement because of residents' frontline involvement in patient safety.^1-3 The majority of faculty and resident physicians agree with the concept of reporting harm-causing hypothetical errors, but only a minority actually report patient safety events.^4,5 Recent studies have shown interesting results in this regard. For example, Madigosky et al demonstrated an unsustained improvement in reporting patient safety events by medical students although patient safety and medical feasibility had been incorporated into their medical school curriculum.⁶ Despite internal medicine residents' positive attitudes toward reporting, Boike et al were unable to achieve a long-term commitment from them at their institution.³ On the other hand, Jericho et al increased the number of adverse event reports and improved the attitudes toward reporting among anesthesiology resident physicians following their educational interventions.⁷

TriHealth, Inc. is a large, nonprofit independent academic medical center serving the Greater Cincinnati area in Ohio. At our institution, a June 2012 site visit report from the ACGME Clinical Learning Environment Review (CLER) revealed residents' and physicians' lack of awareness and understanding of the hospital's system for reporting patient safety concerns in 3 areas: (1) lack of knowledge of what constitutes a reportable patient safety event, (2) lack of awareness of the person responsible for reporting, and (3) lack of understanding of the hospital's current reporting system.

To improve the culture of patient safety at TriHealth, for our Alliance of Independent Academic Medical Centers (AIAMC) National Initiative IV project, we developed a quality improvement initiative to measure residents' and physicians' knowledge and self-reported behaviors related to patient safety event reporting through an anonymous questionnaire and paired the questionnaire with a mandatory curriculum to improve patient safety. Drawing from the above-mentioned studies demonstrating that an educational intervention can improve resident attitudes about patient safety, we administered the questionnaire prior to and after the educational intervention. Although previous studies demonstrated a change in a single program, we extended our initiative across all 6 of our residency and fellowship programs.

During the study period, TriHealth implemented a new patient safety event reporting system, giving the investigators the opportunity to tailor the educational intervention specifically to reporting via the new system. The goal of our quality improvement initiative was to determine if combining an educational intervention with a new patient safety event reporting system would improve event reporting among residents and teaching faculty.

METHODS

Setting and Participants

The quality improvement project was conducted from July 2014 to June 2015. The participants were residents and teaching faculty from the following graduate medical education (GME) training programs at TriHealth: family medicine, internal medicine, obstetrics and gynecology (Ob/Gyn; including urogynecology), and surgery (general surgery and vascular surgery). The TriHealth Institutional Review Board reviewed and approved the study (study no. 14-012).

All the GME training programs at TriHealth are accredited by the ACGME and incorporate efforts to engage residents and teaching faculty in the 6 focus areas of CLER: patient safety; healthcare quality; transitions in care; supervision; duty-hours oversight, fatigue management, and mitigation; and professionalism. As an effort to improve the culture of patient safety and encourage consistent participation in reporting patient safety events, we focused our quality improvement project on patient safety.

Questionnaire

We adapted a questionnaire from a published study to assess residents' and physicians' attitudes and experience regarding patient safety event reporting.³ The original questionnaire is a unique tool developed by Boike et al that is based on 9 core domains and has 60 items previously defined as variables that either facilitate or impede patient safety event reporting.³ We selected questions that were most relevant for the purpose of our study from the original questionnaire, and our final questionnaire included 27 items from 7 domains: (1) experience reporting, (2) responsibility to self, (3) responsibility to community, (4) responsibility to profession, (5) attitudinal barriers of reporting, (6) feelings of uncertainty, and (7) fears of reporting (Figure 1). Responses were measured on a 4-point Likert scale ranging from strongly disagree to strongly agree or very unlikely to very likely.

Figure 1.

Healthcare Providers' Perceptions of Safety Events Questionnaire.

We also solicited participant demographic information, including sex, years of practice, and medical error reporting training in medical school. A comments section was provided for feedback. The questionnaire was electronically distributed to be completed anonymously by the participants before and after the educational session.

Reported Events

Reported patient safety events were defined as the number of events reported per month by resident physicians and teaching faculty. The data were obtained from the Department of Patient Safety and Accreditation, and we retrospectively collected 3 years of reported events for our baseline data (June 2011 to June 2014) to determine improvement in event reporting. The baseline data consist only of events reported by residents because of limitations of our previous event reporting system. For the baseline data, we were not able to extract events reported by teaching faculty. However, following the intervention in July 2014, we specifically tracked events reported by both residents and teaching faculty.

Intervention

Event Reporting System.

A voluntary web-based reporting system, Incident Reporting Information System (IRIS), was adopted at TriHealth in July 2014 as part of an effort to encourage a culture of patient safety and simplify the reporting process. IRIS is the TriHealth enterprisewide system for reporting safety incidents and providing feedback. In contrast to the previous system, IRIS allows all hospital employees to easily access the system from any location and to submit reports quickly and easily using the capability of autopopulating certain fields with critical patient information.

Additionally, IRIS allows employees to anonymously report events that impact the safety of patients. When submitting a report, the reporter has the option to (1) disclose his/her identity; (2) remain anonymous; or (3) disclose only his/her role in the organization (such as physician/doctor, resident/fellow, or nurse) and keeping his/her identity confidential.

Although IRIS is intuitive, we made efforts to educate and train resident physicians and teaching faculty in each GME program on how to use the new system to increase awareness and to help ensure its successful implementation. We collaborated with the Department of Patient Safety and Accreditation and obtained their support for providing instructional education and training.

Educational Session.

We developed an educational lecture for each GME program to improve knowledge and skills related to patient safety and patient safety event reporting. The lecture emphasized the commitment to patient safety, focused on reportable patient safety events in each specialty, emphasized everyone's responsibility for ensuring patient safety, encouraged participation in reporting potential errors and patient safety events, and demonstrated the accessibility of the new online reporting system. Patient safety events were defined as National Quality Forum serious reportable events and our institution's guidelines on reportable patient safety events per specialty.⁸ To accomplish the goals of our quality improvement initiative, one or two resident physicians and one teaching faculty physician from each specialty delivered the educational lecture and served as patient safety role models within their specialty to promote and enhance safe care.

Statistical Analysis

Statistical analysis included simple descriptive statistics, including frequencies and cross-tabulations. We examined the responses to each item on the questionnaires completed before and after the intervention and dichotomized the Likert scale responses as follows: (1) very likely/somewhat likely vs very unlikely/somewhat unlikely and (2) strongly agree/agree vs strongly disagree/disagree. Responses were compared using chi-square or Fisher exact test as appropriate. An alpha level of 5% was used for significance. Computations were performed with the R statistical analysis software v.3.0.3 (R Foundation for Statistical Computing).⁹ The R extension package “likert” was used to graph responses from the questionnaire.¹⁰

RESULTS

We distributed the questionnaire to 105 residents and 78 teaching faculty. The response rate for the preintervention questionnaire was 71% (75) and 49% (38) among residents and teaching faculty, respectively. We distributed the postintervention questionnaire only to those who responded to the preintervention questionnaire, and the postintervention response rate was 80% (60/75) among residents and 68% (26/38) among teaching faculty (Table 1). Our cohort was 51% (58) male and 49% (55) female. Thirty-two percent (12) of teaching faculty reported <10 years of medical experience, 26% (10) reported 10-20 years, and 42% (16) reported >20 years of medical experience.

Table 1.

Preintervention and Postintervention Questionnaire Response Rate by Program

The number of reported patient safety events increased immediately following the intervention (Figure 2). The average number of reported events from the baseline data (June 2011-June 2014) was 1 per month (minimum=0; maximum=4). Following the intervention in July 2014, the average number of reported events was 4 per month (minimum=0; maximum=8).

Figure 2.

Resident and teaching faculty safety event reporting. From June 2011 to June 2014, the number of reported events by residents was x¯=1 (SD=1.07) compared to x¯=4 (SD=2.66) reported by both residents and teaching faculty from July 2014 to June 2015.

Overall, 62% (70) of respondents did not have medical error report training in their medical schools, and 71% (80) had never used any hospital's online error event reporting system. Thirty-three percent of all respondents indicated that they did not receive education or training on how to disclose medical errors to our hospital administration, yet 76% reported that they were likely to report medical errors.

The responses from GME programs were statistically significant for the statements “I do NOT know who is responsible for reporting medical errors” and “I do NOT know which incidents/errors to report” (P<0.05) (Figure 3). However, within individual residency programs, the responses between the preintervention and postintervention questionnaire were statistically significant for 3 questions: “If my senior resident, fellow, or attending does NOT agree that an error has occurred, I will still report what I perceive to be an error” (Ob/Gyn, P<0.05); “I do NOT know which incidents/errors to report” (surgery, internal medicine, and Ob/Gyn, P<0.05); and “I do NOT know who is responsible for reporting medical errors” (internal medicine and Ob/Gyn, P<0.05). For the questions “I do NOT know which incidents/errors to report” and “I do NOT know who is responsible for reporting medical errors,” all residency programs demonstrated an improvement following the educational intervention (Figures 4 and 5).

Figure 3.

Difference between the preintervention and postintervention responses to 4 questions on the Healthcare Providers' Perceptions of Safety Events Questionnaire. The distribution of responses is presented as percentages. The agree and strongly agree responses are aggregated on the right, and the disagree and strongly disagree responses are aggregated on the left. For example, results from preintervention indicate that 42% of respondents did not know which incidents/errors to report. However, following the intervention, this percentage significantly decreased to 8% (P<0.05). *The response to this question was statistically significant for obstetrics/gynecology only (P<0.05).

Figure 4.

Residents and teaching faculty demonstrated an improvement in their knowledge of who is responsible for reporting medical errors (internal medicine and obstetrics/gynecology: P<0.05).

Figure 5.

Residents and teaching faculty demonstrated an improvement in their knowledge of which incidents/errors to report (internal medicine, obstetrics/gynecology, and surgery: P<0.05).

DISCUSSION

While hospital administrators rely on incident reporting systems to help identify problems, physicians often do not want to be burdened with the tedious task of error reporting. Incident reporting requires timely input by the reporter, as well as detailed recollection of the incident. Thus, the safety reporting system must be simple to use and must capture quality information. The usefulness of this information is critical to improving patient safety. Our new reporting system is designed to make the process of reporting easier and more efficient.

After our intervention, we saw an average 4-fold increase in the number of patient safety events reported, and our residents' knowledge about what errors to report and who is responsible for reporting errors improved. Identifying what events to report was part of the educational session of the intervention, and we encourage multiple patient safety event reports from a single resident or teaching faculty if that individual recognizes different patient safety events.

Also after the intervention, a significantly higher percentage of residents said they would report an error even if “my senior resident or attending did not agree that an error has occurred.” Kaldjian et al report that many physicians' primary feelings of responsibility are toward their patients, their colleagues, and themselves and not necessarily toward the goal of increasing the culture of safety in an institution.⁴ The attitudes of the residents toward event reporting shifted in a positive way in 3 of the 4 residency program groups.

After the educational intervention, we saw a 6-fold increase in reporting from June 2014 to September 2014, but then no patient safety event reports were reported in October. Our safety reporting system does not provide positive or negative feedback, and the absence of feedback may have contributed to the lack of sustained reporting. Some of the decrease in reporting after several months is likely attributable to the lack of timely and direct feedback to the residents and teaching faculty who filed anonymous reports. However, nonanonymous reports are assigned an identification number that can be accessed for follow-up. The 2008 study by Kaldjian et al emphasizes the importance of convincing physicians that reporting errors is not fruitless and leads to results.⁵ Closing the loop with reporters will be critical in maintaining the motivation to submit patient safety events. Without a system in place to effect change based on incident reports, bringing about a long-term culture change directed at improved patient safety is much less likely.⁵

Providing an opportunity to report anonymously may be attractive to some reporters because it helps to alleviate some of the most common barriers to reporting events: fear of discipline, fear of fallout from colleagues, and fear that the mistake will be personally tied to the reporter.⁴ However, anonymous reporting has drawbacks. IRIS can tally anonymous reports but cannot track a report to its original reporter. Tracking the report to its original reporter provides an opportunity to collect additional information about the event if needed and to conduct follow-up interviews. This limitation can interfere with identifying error patterns and improving design intervention. However, an anonymous reporting system is important to encourage safety reporting without retaliation.

Continued education is important. Approximately 25% of all TriHealth residents graduate each June and are replaced by new interns in July. Providing a comprehensive curriculum and mandating continued education are necessary to influence the attitudes and awareness of residents about safety events. At the initiation of our study, our residents and teaching faculty had a knowledge deficit, identified not only by the CLER results but also by the responses on the preintervention questionnaire. As stated previously, 62% of all respondents had no training in error reporting in medical school, 71% had never reported an error, and 33% stated they had received no training in error reporting at our institution. Kaldjian et al demonstrated that knowledge deficits about reporting events is a leading factor in underreporting.⁵ Our residents and teaching faculty will continue to require education regarding error reporting and its importance to patient safety. The effectiveness of IRIS depends on educating physicians to identify and report appropriate patient safety events.

Patient safety role models can have an influence on improving patient safety event reporting and thus positively impact the healthcare organization. Role models help create and facilitate the Just Culture needed for sustainability. In our study, we did not just aim to increase patient safety event reporting but also invited faculty to act as role models and contribute to culture change through their participation in the educational intervention. Boike et al suggest that building a culture for reporting needs to be a simultaneous effort of both residents and faculty.³ However, in this initiative, we did not measure the impact of role models, so we can only speculate about their contribution.

We did not compare the number of events reported by residents vs teaching faculty (Figure 2). The number of reported events per month before the intervention only included events that had been reported by residents. Following our intervention, we combined resident and teaching faculty reports to determine if overall reporting improved. However, the total number events reported by residents and teaching faculty after the intervention were 40 and 6 events, respectively. Thus, faculty reports accounted for only 13% (6 of 46) of the postintervention reports. This information shows an improvement in reporting by residents despite not being represented in Figure 2. While the slightly different baseline dataset may be perceived as a limitation, our goal was not to differentiate reporting between residents and teaching faculty but rather to focus on improving patient safety in GME.

Because of the differences between the old reporting system (Quantros) and the new reporting system (IRIS), we were not able to compare the overall number of reports prior to the change in reporting system with the overall number of reports after the change. We would have liked to compare the relative increase in resident reports with the overall number of incidents reported across the organization. However, because of the marked improvement in reach of the IRIS system, this kind of comparison is not possible. IRIS not only allows reports to be completed by any hospital staff member in both the inpatient and outpatient settings, but patients and their families can also submit reports. Quantros did not provide this type of versatility. The number of overall reports is expected to increase disproportionately with the implementation of IRIS, thereby making a comparison of resident reports to the total number of reports misleading.

One of the limitations of our study was the inability to track anonymous reports that may have been filed by residents or teaching faculty (Table 2). Additionally, we identified some system barriers. First, physicians expressed frustration with the lack of feedback to anonymous reporters after the Department of Patient Safety and Accreditation had fully analyzed an incident. After a full analysis of an incident, a generalized educational feedback could be developed and disseminated to all residency programs with the goal of educating the residents about the changes that were implemented because of the incident and the improvement that resulted from those changes. Developing a system that offers targeted verbal feedback on nonanonymous reports will close the loop on safety reporting by providing reporters insight on changes or improvements that were made. Second, we received feedback that the previous error reporting system was a burden secondary to time constraints and administrative burden. The new system implemented at the time of our study was substantially simplified and more intuitive. However, to address these concerns, other options could be made available, such as a hotline or designated administrator to whom events can be reported. We also found differing responses to questions between the individual residency programs that may reflect variations in culture surrounding error reporting. Finally, many residents and members of the teaching had no previous education in error reporting. Continued education throughout the academic year and education for new residents are needed to increase error reporting and to foster the development of a Just Culture within GME. Further studies are needed to assess whether continued education and training lead to sustained increased reporting. Also, studies are needed to evaluate the effect of increased safety reporting on healthcare delivery systems.

Table 2.

Barriers Encountered, Lessons Learned, and Opportunities for Improvement

CONCLUSION

This unique quality improvement project was focused on patient safety event reporting and spanned 6 residency and fellowship programs. The goal of the project was to increase patient safety event reporting by both residents and teaching faculty through education. We achieved this goal through an educational intervention in each residency program tailored to the institution's new event reporting system. In addition, the residents and teaching faculty demonstrated an increase in knowledge of what patient safety events should be reported and who is responsible for reporting them.

Through our work with the AIAMC and our involvement in National Initiative IV, we clearly understand that improvements in quality and patient safety require ongoing effort. The keys to ongoing sustainability include (1) developing patient safety faculty and resident experts in each training program to teach patient safety and to be role models, (2) working toward decreasing the barriers to reporting, and (3) providing timely feedback and system changes.

This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Patient Care, Medical Knowledge, Practice-Based Learning and Improvement, and Systems-Based Practice..