Analysis of Intervention Employability in Pharmacy-Related Medication Safety Reports at a Tertiary Medical Center

Abstract

Background: The Institute for Safe Medication Practice (ISMP) suggests that patient safety reports be addressed with systematic, fail-safe, actions to prevent error recurrence. ISMP’s hierarchy of effectiveness of risk reduction strategies places education-related interventions as the least effective and fail-safes at the top. UNM Hospitals creates a positive environment for safety reporting, but often we are limited to education interventions due to resource and technology constraints. This study analyzes the intervention potential and quality of pharmacy-related medication safety reports. Methods: One thousand medication-related safety reports from selected time points between 2012 and 2022 were selected. Safety reports were included in our study if they were actionable by the pharmacy department. Each safety report was categorized by type of safety event and given an intervention potential score of 1 to 10 (1 indicating education-only, 10 being forcing function) by 2 student pharmacists and 1 pharmacy director based on their potential place on ISMP’s hierarchy. Safety report quality was graded based on professionalism, organization, clarity, and completeness. A standardized evaluation form was used for evaluation for all elements. Results: Six-hundred-sixty-five safety reports were pharmacy-related and evaluated by all 3 study team members for analysis. The 3 most common pharmacy-related safety reports were medication delivery, inappropriate order verification, and transcribing errors which accounted for over half of the reports (59.5%) and on average the intervention potential score of these types of safety reports was education only. Overall, safety reports were limited to a maximum actionability of education-only 75.4% of the time. Safety reports were found to be professionally written and well organized. Conclusion: The actionability of most pharmacy-related medication safety reports was limited to low leverage interventions likely because high leverage solutions were addressed with systematic change and did not recur. Errors limited to education interventions repeated and this increased relative counts of low leverage actionability of safety reports. The ISMP hierarchy of effectiveness of risk-reduction strategies is an important guide to intervening in medication-related safety events, but pharmacy staff should not be discouraged if most of the safety reports cannot be addressed through high-leverage interventions.

Introduction

Safety reporting at medical centers is considered vital to process improvement and a robust culture of safety by numerous organizations.^{1 –5} Incidence of underreporting of safety events in hospital medicine has been reported, however improving capture rates of safety events has proven difficult in medicine due to many barriers.^6,7 Even in facilities where the safety reporting environment is viewed positively by staff and mandated by the employer, patient safety events are largely underreported. ⁶ The Institute for Safe Medication Practices (ISMP) suggests that the best incentive for staff to submit a patient safety report is knowing a difference will be made because of their reporting efforts. ¹ To encourage patient safety reporting, administration should try to ensure that submitted safety reports result in a systematic, permanent change and close the loop with the staff involved.^1,8

In June of 2020, ISMP published an article which emphasized the importance of responding to safety reports with fail-safe strategies that focus on automation and computerization as opposed to staff education. ⁹ This sentiment was echoed by an editorial in the British Medical Journal which emphasized education as a supplement to system-based changes. ¹⁰ It is unclear what percentage of safety reports have a solution that can be resolved with strategies beyond education alone. Focusing on fail-safe interventions through improved use of technology is easier said than done for most safety reports due to budgetary and technological limitations. There is a paucity of literature that evaluates the quality potential intervention of medication-related safety reports.

The University of New Mexico (UNM) Hospital uses a Patient Safety Portal (PSP) to report adverse events and unsafe conditions. All hospital staff are encouraged to report safety events as often as possible. Hospital managers and administration are alerted when a new safety report is sent to the system and our department holds monthly interdisciplinary meetings to discuss the reports. In December 2021, UNM Hospital introduced an abbreviated PSP report type for use if the safety event did not reach the patient called the “Good Catch.” The goal of this new program was to facilitate reporting of “close call” safety reports by reducing documentation burden and recognizing employees’ efforts to intervene in a potentially harmful event. Staff were given adequate time to give feedback on the “Good Catch” system and they were educated about its launch in daily huddles and through hospital wide announcements. Staff are recognized at management meetings and given certificates for submitting a “Good Catch” event.

The objective of this study is to evaluate the feasibility of the ISMP suggestion that healthcare facilities respond to medication safety reports with high leverage, systematic responses in a pharmacy department. Our secondary objective was to analyze the quality and consistency of our pharmacy-related medication patient safety reports.

Methods

This descriptive, cross-sectional study examined quality and actionability of pharmacy-related safety reports at UNM Hospital. Two pharmacy interns employed at the UNM Hospital and 1 pharmacy director independently evaluated 1000 medication-related safety reports between the 3 time periods of December 21, 2011 and January 31, 2012 (9 reports), December 21, 2014 to October 31, 2016 (279 reports), and May 31, 2019 to June 30, 2022 (377 reports). The brief time period of December 21, 2011, to January 31, 2012, was included to sample distant safety reports for historical context. The sample size of 1000 reports was chosen to give a broad sample of about a year’s worth of reporting because our institution typically receives 1 to 2 medication-related safety reports per day. The various time points were included to create a sample that includes historical reports as well as reports before and after our “Good Catch” roll-out.

The selected safety reports were narrowed to events that could be addressed by pharmacy directly such as late delivery, medication mis-fills, and pharmacist transcribing errors. Medication errors that were nursing errors such as unclamped infusion bag and nursing administration errors. were excluded. The pharmacist decided which events were pharmacy-related for grading for the team. The quality of safety reports was graded using a standardized grading scale to evaluate the completeness of the following factors: professionalism, organization, explanation of causes and contributing factors, inclusion of actions taken, and recommendations to prevent the error in the future (see Appendix, Table A1). Only safety reports analyzed by both interns and the pharmacist were included for analysis.

Safety reports were also graded based on their potential place on the ISMP hierarchy of effectiveness of risk-reduction strategies (see Figure 1). The ISMP hierarchy consists of 10 risk-reduction strategies ranked from high leverage, system reliability interventions such as barriers and fail-safes, to low leverage, human reliability interventions such as suggestions to “be more careful.” ⁹ To facilitate accurate grading per the ISMP hierarchy of effectiveness of risk-reduction strategies, the pharmacist and pharmacy interns reflected on their experience at UNM Hospital and discussed currently available technology and potential system interventions given our current staffing, and resources. Using ISMP’s model, grades were assigned to the potential actionability of each safety report. For instance, an ISMP hierarchy grade of 1 corresponded to a safety report in which the institution had no resources to improve a situation beyond general “suggestions to be more careful” with an individual. A grade of 10 corresponded to a safety report in which the institution could reasonably respond to the safety report with a “forcing function” such removing an unsafe medication from formulary making it impossible to order (see Appendix, Table A2).

Figure 1.

The Institute for Safe Medication Practice’s hierarchy of effectiveness of risk-reduction strategies. The most effective safety interventions are those considered “High leverage” and are less dependent on human education and attention. “Low leverage” interventions are easiest to implement, but least effective. ⁹

Source. Reprinted with permission.

Our internal review board reviewed this project and declared it did not involve human subjects and therefore approval was not needed.

Results

Six-hundred-sixty-five safety reports were pharmacy related and used for evaluation. The maximum practical intervention was education-related intervention (ISMP hierarchy grade of 1-3) and low leverage intervention (ISMP hierarchy grade of 1-4) in 75.4% and 81.3% of the safety reports on average (see Figure 2). The ISMP grade for all reports was only 2.7 out-of-ten.

Figure 2.

Intervention potential scores of pharmacy-related medication safety reports. Low leverage intervention is defined by the Institute for Safe Medication Practices as interventions which are limited to policies and staff education, medium leverage interventions include warnings, double-checks, and standardization, high level interventions are fail-safes, automation and forcing-functions.

Table 1 displays the top-10 pharmacy-related medication safety event types. Most delivery issues were related to late delivery. Late deliveries were typically attributed to understaffing and inexperienced staff members unclear about procedures and locations of units. Inappropriate orders verified were typically the result of a pharmacist verifying duplicate therapy, or not considering a pertinent patient parameter (weight and kidney function). Reasons underscoring this type of event typically were inexperience, understaffing, and pop-up fatigue (where a pop-up is possible to be built). Communication events were typically the result of staff not utilizing the handoff tools available, or unclear communication. Mis-transcribing verbal orders, transitions of care orders, and mis-transcribing chemotherapy orders accounted for the majority of transcribing errors. Mislabeling or misfiling errors were due to the staff sticking the wrong label on a product or different tablets being loaded into an automated drug dispensing cubie because of inadvertent tablet mixing. The final 2 types of safety events in the table, unclear electronic medical administration record (eMAR) display and software/hardware malfunction are those types of safety events that were attributed to unclear orders due to CPOE builds and suboptimal optimization of new software, respectively.

Table 1.

Type of Pharmacy-Related Medication Safety Events and Institute for Safe Medication Practices Hierarchy Grade (n = 665).

Type of pharmacy-related medication safety event	Count (%)	Average ISMP hierarchy grade
Delivery	213 (32.0)	2.1
Inappropriate order verified	97 (14.6)	2.8
Transcribing errors	86 (12.9)	2.8
Communication	78 (11.7)	3.1
Misfill or mislabeling	70 (10.5)	1.6
Drug preparation	43 (6.5)	3.7
Existing procedure not followed	18 (2.7)	3.5
Expired medication	15 (2.3)	2.7
Unclear eMAR display	16 (2.4)	5.8
Software/hardware malfunction	10 (1.5)	4.6

Note. ISMP = Institute for Safe Medication Practices.

Table 2 describes the quality evaluation of our safety reports. Almost all safety reports were professionally written and logical (average score of 4.9 out-of-5 and 4.5 out-of-5, respectively). Safety reports were less likely to list contributing factors to the incident or contributing actions (average score of 3.3 out-of-5 and 3.5 out-of-5, respectively). Recommendations to prevent future events showed a bimodal peak, either they were well described or not present at all (average score of 2.7 out-of-5).

Table 2.

Qualitative Evaluation of Pharmacy-Related Medication Safety Reports (n = 665).

	5 (Highest) (%)	4 (%)	3 (%)	2 (%)	1 (Lowest) (%)
Professionally written	71.30	16.80	11.10	0.90	0.00
Presented logically and well organized	69.70	13.90	14.60	1.80	0.10
Actions clearly explained	48.50	8.50	11.10	7.50	24.40
Causes and contributing factors	36.10	11.10	18.20	13.00	21.60
Recommendation to prevent future events	32.30	7.40	8.40	6.00	45.90

Discussion

Maximizing the utility of each pharmacy-related safety report by creating fail-safes is challenging when technology and budget are limited. Even at our facility, where staff write professional and detailed reports and are encouraged to report, common safety event types such as delivery and order processing errors are bound to repeat. The ISMP Hierarchy of effectiveness of risk-reduction strategy diagram is a valuable tool in guiding response to safety events and it predicts the likelihood of a safety event type repeating, but one must be careful not to use it as a measuring stick by which to judge our responses to safety reporting. Even though ISMP ⁹ and Soong and Shojania ¹⁰ consider education the least effective intervention, our response to safety reports is often limited to education.

Barriers to medication safety reporting include the extra time involved in documenting a safety event and time-consuming systems, lack of feedback to the reporter, and lack of knowledge of usefulness of reporting medication-related safety events. Staff beliefs that reporting medication errors have little contribution to improving the quality of care and that they are punishing another staff member also disincentivize reporting. Facilitators to improve reporting include a non-accusatory environment, anonymity, perception that a change is being made, reporting education, and feedback.^1,7,12,13 Hewitt and Chreim ¹⁴ add that healthcare providers are less likely to report errors that are seen as one-time events, are seen as routine or inevitable events, or can be easily fixed in the moment. Hospital leadership should focus on removing barriers related to safety reporting because fail-safe changes are often impossible to implement.

To our knowledge, this is the first study that evaluated actionability of medication-related safety event reports. The actionability of over 75% of our safety reports were limited to education likely because when our facility has the technology and resources to address a safety event with a high leverage approach, the safety event did not repeat. For example, safety events that could be resolved by changing software settings or improving order sets were given higher leverage scores and occurred less often. Most of our safety reports were only actionable through low leverage interventions because those types of events recur multiple times and thus are likely reported more often. An example is the repeat incidence of safety reports due to late or incorrect delivery. Implementing a real-time medication tracking system or automated delivery robot would reduce the incidence of these events, but implementation is not financially feasible for our institution. An enthusiastic staff with an institution that rewards “Good Catch” reporting is less likely to forgo event reporting as described by Hewitt and Chreim. ¹⁴ Seeing errors repeat is discouraging for staff and a manager may feel they are not doing enough to address the issue. ¹ ISMP refers to education as “predictably disappointing,” ⁹ however in an institution with a healthy culture of safety, it may be a good sign that the actionability of most safety reports are limited to education-only interventions.

Our study’s limitations are primarily related to the subjective nature of the event reports and the subjective grading of the event reports. The number of potential sources of systematic failure for a medication error is vast and staff approach to reporting is subjective and individualized. ¹⁵ Designing an objective framework to categorize such diverse reporting is challenging. For instance, grading the actionability of the interventions depends on the limited information provided in the safety report and the knowledge and experience of our pharmacist and student graders. Sometimes the author of the safety report would offer a solution that helped the grading. In addition, our grading system has not been validated and may not reflect actual best practices, it is based on our review of suggestions from ISMP ¹ and our internal hospital safety reporting system. In aggregate, our grading was similar, but there were differences in opinions of individual safety reports. Efforts were made to standardize the grading and we worked through sample event reports together; however, it was difficult to categorize and score each specific event.

In Feeser et al, ¹⁶ up to 25% of patient safety reports are written in a punitive manner and educational emphasis on the importance of systematic improvements is important to improving utility and civility in the reporting system. While we did not evaluate punitive language in our study, we anecdotally noticed occasional language defined as punitive from Feeser et al ¹⁶ in our safety reports. Uncivil language in safety reports could reflect frustration and misunderstanding of how errors should be reported and processed. Our data shows that the percentage of patient safety reports that can be addressed via systematic change is low, however in institutions where a high volume of safety reports is encouraged, this is a positive sign that inexpensive, low-hanging fruit is being promptly addressed. Administrators should include in staff safety education that each individual patient safety report may not immediately result in desired systematic change, but they help in aggregate to prioritize large, systematic changes. ¹⁷

Safety event reporting systems are important for the improvement process of a healthcare facility and proper leadership response to safety reports is vital to encourage reporting. Traditionally leadership encourages high-volume safety reporting, ¹ but many safety reports cannot be leveraged to prevent recurrence of the event. Staff may feel discouraged when they are asked to spend time to fill out a report that they know will not result in meaningful change. Should we ask staff to prioritize reporting events when the event is a new event type or circumstance? Future research should evaluate how to best use safety reports for events that are bound to repeat and how to encourage a positive safety reporting culture, especially when most safety reports do not bring the change the reporting staff desires.

Appendix

Table A1.

Patient Safety Portal Event Professionalism Scoring Table.

PSP grading
1. Was the PSP professionally written?  • The PSP was concisely written with a focus on the facts and not editorializing.  • Focus on systems change, no finger pointing, or placing blame.
1-2: Blame was directly placed on those involved with limited facts	3-4: Facts were presented but hard to follow as evidence presented was out of order or not clear with some blame	5: Facts were presented with no blame and easy to follow or understand.	Score
2. Is the material presented logically, completely, and well organized?  • PSP is chronologically presented and easy to read and understand.  • All pertinent information is presented with no details left out.
1-2: Little to no information is presented and not clear on sequence of events	3-4: Some information is presented but hard to follow sequence of events	5: Information is presented chronologically, and event is easily understood	Score
3. Were the causes and contributing factors clearly presented?  • The cause(s) of the incident were presented clearly and concisely.  • Any contributing factors to the incident were mentioned.
1-2: Little to no causes or factors were accounted or documented	3-4: Some factors were mentioned, but not clearly presented to understand and follow if factors contributed to the event	5: At least 1 cause and contributing factors were presented in a clear and concise manner that was easy to follow	Score
4. Were the actions that were taken because of the event clearly explained?  • Actions done after the event were included in the PSP.  • Action taken well described and is related to the event.
1-2: No action after the event was included in reporting or action was irrelevant	3-4: Some action was included but was hard to follow and understand	5: Related actions taken after the event were included in the report and well described	Score
5. Was a recommendation provided for change to prevent future incidents?  • Recommendations for a system change or policy change are included in the report.
1-2: No recommendation was given to prevent a future event	3-4: Unclear or non-actionable recommendation given	5: A clear and understood recommendation was presented in the report	Score

Note. PSP = Patient Safety Portal.

Table A2.

Patient Safety Portal Event Table Based on the Institute for Safe Medication Practice’s Suggested Hierarchy of Intervention. ⁹

Intervention	Example	Score
Forcing function	Removal of product for use, specific option in CPOE eliminated.	10
Barriers and fail safes	Forced stop could reliably be implemented, but workaround possible. Hard stop on allergies.	9
Automation and computerization	Automated patient-specific dispensing, specific tablet size and suggested frequency loaded into CPOE. Barcode scanning.	8
Standardization and protocols	Standardized paper or electronic order sets in CPOE.	7
Redundancies	Independent double-check could be reasonably implemented.	6
Warnings, alerts, reminders, checklists	Pop-ups.	5
Rules and policies	Policies prohibiting borrowing doses from other areas. Pharmacy procedures for handling hazardous drugs or processing a specific type of medication.	4
Educational programs	Educational programs on high-alert medications, specific education on a drug or policy in a Lunch-and-Learn program. If someone overlooks renal adjustment, drug interaction, or enters something on a wrong patient it should usually be a “1.”	3
Available information	Education in huddle? Education in PSP meeting	2
Suggestions to “be more careful”	Pulling someone aside and educating them individually. An honest slip-up that we cannot really make an educational program or implement other strategies above (late delivery, lost dose, mis-transcribed dose that was not caught on double-check).	1

Note. CPOE = Computerized Physician Order Entry; PSP = Patient Safety Portal.