Multifaceted Intervention to Improve Patient Safety Incident Reporting in Intensive Care Units

Abstract

Objectives:

Patient safety incident reporting in our institution’s intensive care units (ICUs) had fallen 30% below national benchmarks during the COVID-19 pandemic. Underreporting diminishes awareness of risks and precludes organizational learning from near misses. We aimed to increase the ICU number of patient safety incident reports by 30% from 27 to 35 reports/1000 patient-days without negatively impacting culture of safety as measured by patient-care staff surveys.

Methods:

Single-institution prospective interventional study with nine ICUs receiving a multifaceted intervention developed using quality improvement (QI) methodology during February–April 2022. Study intervention involved creation of patient safety peer-leadership role, feedback process, interactive dashboards for patient safety data, and education resources accessible via quick response codes. Primary outcome was patient safety incident reports/1000 patient-days. ICU patient-care staff culture of safety was assessed with surveys.

Results:

ICU patient safety incident reporting increased by 48% post-intervention (40 versus 27 reports/1000 patient-days [p=0.136]). Near misses were the most common incident report. ICU patient-care staff ratings of patient safety did not change; 80% rated patient safety as good or better post-intervention versus 78% at baseline, p=0.465. However, significant improvement was observed for subcomponents related to learning culture and support for staff involved in patient safety incidents. Most (>80%) reports were submitted by nurses.

Conclusions:

This multifaceted QI intervention increased patient safety incident reporting in the ICUs. Increases in ratings of learning culture and support for staff underline the importance of a well-functioning patient safety incident reporting system in an institutional culture of safety.

INTRODUCTION

Medical errors are the 3^rd leading cause of death in the U.S. and substantially increase healthcare costs.^1,2 Patient safety incident reports capture incidents that could or do result in harm to a patient. Increased reporting leads to robust process improvements and enhances trust and culture of safety. Most adverse events are systems/process issues and not overt human error.^3,4 These latent or hidden failures, when not identified, can eventually lead to error/patient harm.⁴ The patient safety incident reporting system at our institution is confidential, voluntary, online, and was recently integrated into the electronic health record (EHR).

The year 2020 posed unique challenges to intensive care units (ICUs) related to the COVID-19 pandemic with both high patient censuses and staff shortages. As part of monitoring patient safety incident report types and counts, leadership in our Patient Safety Office identified sustained trends of lower overall and near miss reporting in the ICUs while reports of incidents with patient harm had remained similar to historical levels. As the ICUs accounted for only 16% of all reports submitted across the institution in 2020, there was concern for significant underreporting as well as a shift in the culture of safety and collective understanding of the importance of near misses in an institution’s ability to proactively learn and respond to potential patient safety risks. Prior studies have shown that there are dozens of near misses for every adverse event, and that only 4–50% of patient safety incidents are actually reported, further supporting the potential for improvement.⁵⁻⁷

External benchmarking analysis showed underreporting in the ICUs by 30%, confirming the need for a targeted quality improvement (QI) project.^8–36 We aimed to increase the ICU number of patient safety incident reports by 30% from 27 to 35 reports/1000 patient-days without negatively impacting culture of safety as measured on patient-care staff surveys.

METHODS

This was a single-institution prospective interventional study and interventions were implemented between February–April 2022. The Define, Measure, Analyze, Improve, Control (DMAIC) framework was utilized; the Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines were followed; and institutional review board approval was not required.³⁷ A project timeline is provided in Figure 1.

Figure 1: Project Timeline.

Caption: Timeline for entire quality improvement project. ICU=intensive care unit; QI=quality improvement; QRG=quick resource guide; QR=quick reference

Study Setting

Our institution has two hospitals with a total of 2,059 inpatient beds. There are nine ICUs with 294 beds. The respiratory care unit for stable patients requiring ventilator weaning was excluded. During the study period, ICU patient-care staff totaled approximately 1,670 across all direct patient-care roles, including 149 attending physicians, 140 advanced practice providers (APPs; nurse practitioners and physician assistants), 1,145 registered nurses, 70 pharmacists, and 170 respiratory therapists. Residents/fellows are important team members in our ICUs, but rotational staffing models made it difficult to identify the numbers dedicated to the ICUs during the study period.

Patient Safety Incident Reporting System

Our institution utilizes a confidential and voluntary online patient safety incident reporting system for internal patient safety monitoring and QI. While this system assists in identification of events for mandatory reporting systems, it is separate from participation with The Leapfrog Group and those mandatory reporting systems under state law, which are consistent with the National Quality Forum’s Serious Reportable Events.^38,39 Reports are submitted by patient-care staff. Patient Safety Specialists analyze reports, review findings at multidisciplinary meetings, and conduct root cause analyses when indicated. Leadership in Patient Safety evaluate trends and identify opportunities for improvement. Patient Safety Specialists monitored ICU patient safety incident reports for completeness and ensured the final incident reports had all required program elements. Throughout the study there were no issues identified with incident report quality. Reporting process inefficiencies were identified by QI team members during process observations and waste walks.

Project Team

In response to this quality gap, Patient Safety collaborated with critical care leadership at our institution to form a multidisciplinary QI team with representation across direct patient-care roles and units. Consistent input from the 19 team members was maintained through regular meetings. Project oversight was provided by critical care leadership and experts in Patient Safety and QI.

Causes for Low Reporting

We performed fishbone cause-and-effect analysis with our multidisciplinary team to identify causes for lower patient safety incident reporting in the ICUs and validated the results during seven focus groups (Figure 2). There were 69 focus group participants across various ICUs and roles. The participants ranked the causes in order of importance on a survey; response rate was 93%. Pareto diagram was used to identify the most significant causes from the survey data, followed by the “5-Why’s” for further root cause analysis. The top three causes for low reporting included user difficulty with the online reporting system, lack of feedback following submission of patient safety incident reports, and unclear process for submitting reports. While our project was started during the COVID-19 pandemic, prior stakeholder analyses conducted by our institutional Patient Safety group had identified similar issues.

Figure 2:

Fishbone Cause and Effect Analysis

Interventions

Our change management strategy included stakeholder engagement and communication with an A3 problem solving template. We utilized a combination of “just-do-it” interventions and Plan-Do-Study-Act (PDSA) cycles as well as worked to provide increased accessibility to patient safety incident report data to ICU patient-care staff. “Just-do-it” interventions were implemented in February 2022 in all ICUs. The first phase of our feedback PDSA was implemented in two ICUs in February 2022 and the second phase of the feedback PDSA was implemented in the remaining ICUs in April 2022. Our project team, in collaboration with ICU data specialists and Patient Safety, created interactive dashboards on the Tableau platform (Tableau Software, Seattle, WA) that provided patient-care staff with the ability to visualize report numbers and trends, significance categories, and incident classes/types by specific locations and time periods. This collection of dashboards was called the ICU Patient Safety Tableau. Previously, data had been limited to hospital-wide summaries and monthly aggregated unit-specific reports. With the new dashboards, leadership are now able to trend patient safety risks within their units and across units over time and compare incident reporting to the national benchmark.

“Just-do-it” interventions are interventions that can be implemented quickly without iterative development. These consisted of educational resources for patient-care staff, including a new reporting quick resource guide linked to quick response (QR) codes posted at ICU computers, a handout highlighting ICU-specific patient safety incident examples posted across the ICUs, and a new multidisciplinary “Patient Safety Champion” peer-leadership role. The quick resource guide included short-cuts to make using the online reporting tool easier as well as incorporated elements from the model developed by Arnell et al to improve quality and completeness of incident reports.⁴⁰ The peer-leadership engages additional patient-care staff interested in patient safety issues (bedside nurses, pharmacists, APPs, etc.) with clinical nurse specialists (CNSs) and other unit leadership. They participate in monthly incident review and help communicate key learning points/action items and great catches to colleagues. They are also expected to serve as a point person for peers with questions about incident reporting. Access to the ICU Patient Safety Tableau was shared with Patient Safety Champions and unit leadership.

PDSA cycles were used to develop a feedback process, combining direct-to-reporter feedback from Patient Safety Specialists for patient safety incidents with moderate to high patient harm, monthly near miss trend analysis for the ICUs in aggregate, and monthly within-unit communication highlighting learning points/action items and great catches for patient-care staff based on monthly incident review and near miss trend information.

The quality, feasibility, and efficacy of interventions were assessed periodically via regular meetings with ICU CNSs and nurse managers, the multidisciplinary QI team members, Patient Safety, critical care leadership, and the Patient Safety Champions.

Measures

The primary outcome measure for this study was the number of patient safety incident reports/1000 patient-days. Aggregate incident report data from January–June 2021 served as the baseline and aggregate incident report data from April–June 2022 served as the post-intervention assessment (Figure 1). Data were updated monthly in the ICU Patient Safety Tableau to track progress throughout the improve phase (Figure 3).

Figure 3: Patient Safety Incident Reporting over Time.

Caption: Patient safety incident reporting over time stratified by near misses and incidents with patient harm. Arrows indicate the start date for implementation of project interventions (February) and the new online patient safety reporting (PSR) tool (July).

Secondary outcome measures included numbers of near miss reports and the percentage of reports submitted by various care team roles. From January 2019 to September 2020, near miss reporting had consistently been the most common report type; however, since September 2020, near miss reporting had a sustained decrease with numbers remaining below numbers of reports of incidents with patient harm. These secondary outcomes enabled our team to track the impact of project interventions on near miss reporting, an essential aspect of healthy patient safety reporting systems, and the distribution of reporters across the patient-care team.

Patient safety incident reports captured undesirable events in patient care that were not caused by the underlying disease and were classified into the following categories: near misses that have the capacity to cause harm but do not reach the patient; incidents with no harm that reach the patient but do not cause harm, incidents with temporary harm that result in patient harm requiring medical intervention but are not life-threatening, do not cause permanent harm, and do not prolong the hospital stay; and incidents with permanent harm (also known as adverse events) that require life-saving intervention, contribute to death, or prolong hospital stay.⁴¹

The balancing measure for this study was culture of safety as measured on anonymous ICU patient-care staff surveys utilizing Research Electronic Data Capture (RedCap) with 5-point Likert scales.^42,43 Perception of patient safety and culture of safety were assessed (Figure 4). While many workplace factors affecting perceptions of patient safety were out of the scope of this project, a higher level of risk awareness due to increased incident reporting could potentially result in decreased perceptions of patient safety.⁵ We wanted to make sure that there was not a negative impact of the project interventions on patient-care staff perceptions of patient safety. The baseline for the balancing measure used December 2021 baseline survey data. The post-intervention assessment used data from the post-intervention survey conducted in July 2022 (Figure 1).

Figure 4: Culture of Safety Survey Results.

Caption: Pre- and post-intervention culture of safety survey results with graph demonstrating percentages of responses on Likert scale (1=worst rating, 5=best rating). Individual questions listed on the left-hand side. P-values and green box indicate components (learning culture and support for staff) with significant improvement following the project interventions.

Statistical Analysis

Statistical analysis was performed using RStudio Version 4.0.3 (RStudio, PBC; Boston, MA) with statistical significance defined as p<0.05. Categorical variables are reported as number (%) and continuous variables are reported as median (interquartile range). Kruskal-Wallis rank sum tests and two sample tests for equality of proportions with continuity correction were used for subgroup comparisons where appropriate. There was 100% ascertainment for patient safety incident reports in the ICUs via the reporting system.

RESULTS

During January–June 2021, there were 692 ICU patient safety incident reports over 26,159 patient-days (27 reports/1000 patient-days). This aggregate incident report data served as our baseline for our primary outcome. The post-intervention assessment used aggregate incident report data from April–June 2022, during which there were 581 ICU patient safety incident reports over 14,516 patient-days (40 reports/1000 patient-days). The post-intervention period, April–June 2022, showed an overall increase in reporting by 48%; ICUs submitted 40 patient safety incident reports/1000 patient-days post-intervention compared to 27 patient safety incident reports/1000 patient-days at baseline, p=0.136. Although the overall ICU increase in reporting was not statistically significant, there were several units that had robust increases in reporting that were statistically significant (Table 1). A 48% increase was also observed for near miss reporting: 19 near miss reports/1000 patient-days post-intervention versus 13 before, p=0.373. Importantly, near miss reporting reached and sustained levels higher than reports of incidents with patient harm, marking a return to historical norms of near misses being the most common type of report (Figure 3). Overall, 6/9 (67%) ICUs met the national benchmark of 35 patient safety incident reports/1000 patient-days post-intervention versus 2/9 (22%) before (Table 1).

Table 1:

Increase in Patient Safety Incident Reporting by Intensive Care Unit

Location	Patient Safety Incident Reports a		P-Value
	PRE	POST
All ICUs	27	40	0.136
Location
ICU A	14	59	<0.001
ICU B	15	32	0.018
ICU C	22	21	1.000
ICU D	25	24	1.000
ICU E	27	36	0.306
ICU F	30	36	0.531
ICU G	32	56	0.012
ICU H	38	40	0.908
ICU I	39	84	<0.001

^(a)Patient safety incident reports per 1,000 patient-days

Statistically significant p-values in bold and italicized.

A new online patient safety reporting tool was implemented in July 2022, improving accessibility and ease of use with its integration into the EHR. This project’s QI methods, especially the process analyses, cause-and-effect analysis, and focus groups, provided not only additional support for the new reporting tool’s implementation but also helpful feedback for the design and functionality of the tool. The positive momentum and increases in reporting observed in the ICUs following the intervention were sustained throughout April–June 2022, a drop in reporting associated with the transition in the reporting tool was observed in July, and then further increases in reporting were observed for August and September 2022 (Figure 3).

The pre-intervention ICU patient-care staff culture of safety survey response rate was 33% (590 surveys completed), similar to the 34% response rate for an internal all-staff survey conducted across all patient care locations earlier that year. However, the ICU culture of safety survey had lower overall ratings of patient safety: only 78% of participants rated patient safety as good or better compared to 90% on the all-staff survey.

The post-intervention ICU patient-care staff culture of safety survey was identical to the baseline survey. The response rate was lower, 26% (464 surveys completed), compared to 33% at baseline, p<0.001. Post-intervention, 80% of participants rated patient safety as good or better compared to 78% at baseline, p=0.465. However, significant improvement was observed on the survey components related to learning culture and support for staff involved in patient safety incidents (Figure 4).

Most patient safety incident reports are submitted by nurses, and this did not change during the study period, 83.0% pre- versus 84.7% post-intervention, p=0.476. The percentages of patient safety incident reports submitted by other care team roles also showed no differences pre- or post-intervention: APPs 4.3% versus 4.4% (p=1.000), pharmacists 3.3% versus 4.7% (p=0.294), respiratory therapists 1.2% versus 2.2% (p=0.331), trainees 0.6% versus 0.5% (p=1.000), and attending physicians 0% versus 0.3% (p=0.565).

DISCUSSION

We demonstrate that a targeted, multifaceted QI intervention can successfully increase patient safety incident reporting in ICUs where underreporting is a concern. Root cause analysis identified three main barriers (user difficulty with online reporting system, lack of feedback, unclear process for submitting reports) that informed the study intervention, which involved creation of a patient safety peer-leadership role, feedback process, online interactive dashboards for patient safety data, and education resources accessible via QR codes. The multifaceted intervention increased overall reporting by 48% and near misses became the most common report type again, thereby closing the reporting quality gap in the ICUs and improving the institution’s ability to respond to and proactively target areas of concern for patient safety. Our study also shows that increased reporting is associated with improvements in patient-care staff perceptions of a learning culture and support for staff as regards patient safety, underlining the importance of a well-functioning patient safety incident reporting system in an institutional culture of safety. The implementation of the new reporting tool that is integrated into the EHR was an additional benefit and has continued the steady progress observed across the ICUs following our study interventions (Figure 3).

The main purposes for patient safety incident reporting systems include identifying risks with reporting, learning from incident review and providing feedback to frontline patient-care staff and leadership, addressing system issues that result in those risks, and determining if improvement efforts result in risk reduction for patients.^9,44–47 As many reporting systems are voluntary, participation bias can influence report data, especially in environments of underreporting, and result in an incomplete picture of the patient safety risks in the locations from where they are generated.^5,6,48 In this study, we confirmed underreporting in our institution’s ICUs, and this provided strong rationale for a QI project to improve our incident reporting system’s capabilities for learning by increasing the number of reports.

Consistent with prior literature, desire for feedback regarding patient safety incidents was a common theme among stakeholders and focus group participants.^5,47–51 Prior to this project, there was no feedback process at our institution for patient safety incident reports beyond incident analysis communication with Patient Safety Specialists. Feedback was a focus for our QI team, and we developed a comprehensive intervention utilizing two PDSA cycles. The feedback in our study involved direct-to-reporter feedback to provide closed-loop communication and unit-specific monthly communication from unit CNSs/leadership to patient-care staff regarding near miss trends, learning points/action items, and great catches/near misses. The feedback process was received positively and has been formally adopted. The new reporting tool implemented in July has automated several aspects of direct-to-reporter feedback, including status updates and incident review summaries, allowing for improved sustainability long-term.

Although increased reporting was seen in most ICUs, there were two units that had no increase post-intervention (Table 1). This lack of improvement is multifactorial, though likely related to user issues with the previous reporting tool and larger unit size. ICU C has shown substantial improvement since the implementation of the new reporting tool in July and is now meeting the national benchmark for reporting. ICU D is large (58 beds) and some areas within the unit showed significant increases while others had lower reporting. The heterogeneity in reporting and improvement following intervention in this unit suggests that there are potential challenges in some areas that our interventions did not address as well as variable stakeholder engagement. There are opportunities for continued improvement within these ICUs, and the higher performing units/areas can serve as aspirational peers for continued progress. Learning culture and support for staff are essential aspects of a positive culture around reporting among patient-care staff, especially when reporting systems are voluntary, and should be emphasized in additional improvement efforts.^5,45,52

Nurses submitted the majority of patient safety incident reports and physicians submitted the fewest, consistent with prior studies.^6,9,51,53 Although increased reporting was observed following the interventions, there was no change in the demographics of reporting. The goal of involving all members of the patient-care team in reporting is to improve incident report data’s representativeness of actual patient safety risks, as prior research shows that reports differ by role of the reporter.^5,9 Percentages of reports submitted by various roles was not a target for improvement, rather improving reporting numbers across all roles allows for a greater variety of risks to be identified and evaluated. Nonetheless, relatively low reporting continues to be observed among physicians, so efforts to further engage physicians in reporting would be beneficial. Interestingly, since the new reporting tool was implemented in July, the number of physician (attending and trainee) reports have increased; the ease of use as well as integration into the EHR have been cited as potential reasons for this change.

Prior studies have discussed the strengths and weaknesses of patient safety incident reporting systems as well as different areas of focus for improvement efforts. Work to increase the number of reports has been previously criticized and authors have suggested that more reports does not necessarily equate to improved quality or the data’s representativeness of the risks to patient safety.^6,48 Other concerns have been raised that important patient safety risk signals could be lost in the noise of more reports.⁴⁸ However, there are significant benefits to increasing reporting across an institution, especially in an environment of underreporting, including improved awareness of risks to patient safety, improved culture of safety among patient-care staff, and increased opportunities for internal quality improvement to mitigate risks to patient safety.^5,54 Importantly, when patient safety incident reports decline, the patient safety incident data cannot distinguish between improvements in patient safety or an institutional blind spot. We believe the risk of a blind spot is higher in an environment of underreporting.⁴⁸ Of note, no changes in report quality at our institution were observed throughout the study period despite a substantial increase in report numbers. The risk of losing important patient safety risk signals can be mitigated by analyzing all reports meeting certain harm criteria, as is the practice at our institution, as well as integrating data into dashboards that can separate reports by classes or types of events and various locations. Our institution’s ICU Patient Safety Tableau has this capability, allowing signals for lower-frequency incident types to be separated from overall report count noise.

Incorporating “just-do-it” interventions and utilizing handheld technology tools (QR codes) minimized our project’s strain on pandemic-related staffing crises and other stresses associated with the ICU environment. The QR codes were familiar to patient-care staff and made resources immediately accessible without introducing clutter to the work environment. Additionally, the linked resources were housed on our institution’s internal network, protecting confidentiality and preventing access by patients or visitors. Additional strengths of our interventions include using automated tools such as Tableau dashboards to not only measure progress throughout the intervention period but also improve accessibility to and understanding of patient safety data within and across ICUs.

While the interventions were designed and tested for the ICU environment, we believe several elements can be used in other patient care environments. These include QR codes to make resources easily accessible, interactive dashboards for patient safety data, the Patient Safety Champion peer-leadership role, and monthly feedback within a unit or work area highlighting relevant trends, important learning points/action items, and great catches.

Limitations

As a single-institution study conducted within the ICU, the interventions/results may not be fully generalizable to other settings. It is not possible to fully attribute the increases in reporting in the ICUs during the study period to our interventions as they were implemented in an uncontrolled environment, but we believe they are highly associated given the longstanding trend of decreased reporting prior to the study interventions and the lack of concomitant interventions promoting patient safety unrelated to our project during the study period. Of note, the new reporting tool occurred after our study and analysis. Finally, the study was not designed to assess the impact of project interventions and increased reporting on clinical outcomes.

CONCLUSION

A multifaceted QI intervention involving feedback regarding patient safety, the Patient Safety Champion peer-leadership role, interactive dashboards for patient safety data, and easily accessible education resources resulted in a substantial increase in patient safety incident reporting in the ICUs. The significant increases in patient-care staff ratings of learning culture and support for staff as regards patient safety underline the importance of a well-functioning patient safety incident reporting system in an institutional culture of safety. Increasing the number of patient safety incident reports in an environment of underreporting provides health care systems with the information needed to optimize patient safety.