Using the Failure Mode and Effect Analysis Tool to Improve the Automatic Stop Order Process

Abstract

Introduction

Automatic stop orders (ASOs) in computerized prescription order entry (CPOE) systems predefine the length of treatment. This can improve resource use for select therapies (e.g., empirical antibiotics). However, root cause analysis of dose omission errors identified inappropriate ASO-directed termination of medications without prescriber notification. This quality improvement initiative aimed to identify potential failures of the medication ASO processes to develop a new workflow and anticipate issues that may arise after implementation.

Methods

A failure mode and effect analysis (FMEA) was conducted following Institute of Healthcare Improvement guidance. A multidisciplinary ASO-FMEA team reviewed the existing workflow. Failure modes, risk priority numbers (RPNs), and interventions were identified and assessed. The RPNs calculated for the proposed new workflow (assuming all recommendations were implemented) were compared with those of the existing workflow.

Results

Eight failure modes, 17 effects, and 31 causes were identified in the five workflow steps (mean RPN 365.4; median 280). Specific, measurable, achievable, realistic, and time-bound interventions were proposed. Assuming successful implementation of all recommendations, the RPNs of the proposed workflow (mean 117.6; median 112) were significantly lower (p < 0.05).

Conclusion

When modifying existing CPOE systems, FMEA may identify possible failures that can be addressed before the implementation of a new process. This may prevent errors, improving medication safety. Regardless, continuous audit and monitoring are required to ensure the effectiveness of implemented changes.

INTRODUCTION

Automatic stop orders (ASOs) in computerized prescription order entry (CPOE) systems leverage hard stops to enforce predetermined medication end dates.[1] The aim of an ASO is to prompt physicians to reassess the necessity for medications, thereby preventing unnecessary prolongation of therapies.[1] The ASO durations vary based on drug class, formulation, indication, and local regulations for specific medications (e.g., narcotic and controlled drugs). For example, at our institution (Table 1), an order for intravenous heparin is automatically stopped 24 hours after the order is signed. This should prompt reassessment of the patient’s condition, review of lab results to adjust the heparin dose, and further monitoring.

Table 1.

Default medication durations specified in Ministry of National Guard Health Affairs (MNGHA) policies relevant to Automatic Stop Orders (ASOs)

Medication	Default Time Frame
Empirical antimicrobial therapy	72 hours, unless the prescriber enters a specific duration
Antimicrobials for treatment, psychotropic substances, hypnotics, narcotics, sedatives	7 days, unless the prescriber enters a specific duration
Antimicrobial for surgical prophylaxis	24 hours, unless the prescriber enters a specific duration
Intravenous heparin	24 hours, unless the prescriber enters a specific duration
Parenteral pain management	72 hours, unless the prescriber enters a specific duration (e.g., intravenous (IV) acetaminophen and IV morphine IV)
Nebulized respiratory medications	96 hours, unless the prescriber entered a specific duration
All other medications	28 days, unless the prescriber enters a specific duration

The computerized prescription order entry system of the MNGHA Health Informatics System uses these defaults to prepopulate ASO dates when medications are ordered. Unless the start date is deferred, the order date is day 0. The ASO is triggered at 2400 hours on the day specified (i.e., a medication started on Apr 5 with a 7-day ASO will stop at midnight on Apr 12). Prescribers can adjust the ASO date manually when ordering medication. To extend a medication beyond the ASO date, prescribers must reorder the medication before the ASO is triggered to avoid missed doses.

The Joint Commission International (JCI) requires hospitals to establish policies and procedures for special orders, including ASOs.[2] Although ASOs can enhance patient safety, abrupt discontinuation of medication without notifying a prescriber may result in omissions.[3] Indeed, the Institute for Safe Medication Practices (ISMP) has reported several medication errors caused by inappropriate discontinuation of treatment by an ASO.[4]

Several factors exacerbate this system failure. Healthcare providers’ limited awareness of institutional ASO policies, unclear or hidden automatic stop dates in the CPOE, and the absence of appropriate alerts after discontinuation clearly contribute. Ambiguous policies that do not clearly define specific healthcare providers’ roles with respect to the review of medications’ ASOs create an additional layer of complexity. Some institutions attempt to address the issue with reeducation.[5] However, the efficacy of such reminders is limited, so relapses occur.[5]

The development of successful, relevant, sustainable interventions requires a detailed assessment. Root cause analysis (RCA) can pinpoint the cause of past failures but cannot identify flaws in a system before an incident occurs. Furthermore, complex processes such as the medication ASO can have many potential failure modes that may not be identified by an RCA focused on a specific incident. Thus, the Institute for Healthcare Improvement (IHI) advocates the use of failure mode and effect analysis (FMEA) to address this gap.[6]

Anticipatory in nature, FMEA proactively analyzes healthcare systems for potential weaknesses before errors occur. Evaluating each step’s susceptibility to various failure modes allows identification of vulnerabilities and hazardous situations. This foresight enables implementation of preventive measures, minimizing the risk of future failures in healthcare settings.[7–15] The use of FMEA has reduced the risks associated with in-hospital transfers,[12] dispensing drugs,[13] psychiatric emergency care,[14] and cancer treatment.[15]

When conducting an FMEA, a multidisciplinary team must create a process map, analyze each step for potential failures and then evaluate their severity, likelihood of occurrence and detectability to calculate risk priority numbers (RPNs). Corrective actions are then proposed and prioritized by RPN. Calculation of the RPN for the new proposed workflow allows the potential impact of the recommendations to be considered before their implementation.

The aim of this quality improvement initiative was to identify the potential failure modes in our institution’s existing medication ASO process. These data would be used to develop a new ASO workflow, anticipating any issues that may arise after implementation of the new processes.

METHODS

Retrospective analysis of medication administration practices was approved by the institutional review board, with a waiver of informed consent.

Setting

The Ministry of National Guard Health Affairs (MNGHA (operates primary, secondary, and tertiary healthcare facilities throughout the five main regions of the Kingdom of Saudi Arabia (KSA). The largest facility is King Abdulaziz Medical City, Riyadh. This JCI-accredited, university-affiliated tertiary center operates around 1800 beds. The institutional review board approved this study [mention consent as well].

The MNGHA mandates the reporting of all witnessed medication errors through a voluntary intranet-based reporting system. In 2022, around 4000 medication events were reported in the central region of KSA alone. A medication safety officer (MSO) systematically reviews, analyzes, and categorizes each report. This allows prompt intervention for critical events and ongoing monitoring of trends.

Index Events

Retrospective review of selected medication omission errors, using the standardized mini-RCA (mini-RCA2) methodology described by Al Mardawi et al[16] and Al Mardawi and Rajendram[17] allowed for identification of recurrent themes. This revealed a concerning trend, that lapses in medication administration often stemmed from medication orders being stopped by an ASO without prescriber notification. This concerning finding prompted the medication safety program (MSP) to proactively address potential flaws in the ASO process. FMEA was employed to systematically identify potential weaknesses and develop mitigation strategies.

Study Design

This quality improvement initiative used FMEA to optimize the workflow of the medication ASO process at our institution as described below and illustrated in Figure 1. The primary outcome measure was the difference in RPN between the existing and proposed medication ASO workflows (assuming all recommendations had been implemented).

Figure 1.

How to conduct a failure mode and effect analysis (FMEA). A step-by-step approach to the conduct of an FMEA and calculating risk priority numbers (RPNs).

Failure Mode and Effect Analysis

The MSP assembled a multidisciplinary FMEA-ASO team (an MSO, a quality specialist, pharmacy staff, nursing staff, medical staff, and an information technology specialist). The team familiarized themselves with the IHI FMEA tools prior to the project. The MSO examined and labeled each step in the existing workflow (Fig. 2).

Figure 2.

Process map of the medication ASO workflow. This process map illustrates the 5 main steps of the medication automated stop order workflow. CPOE: computerized prescription order entry.

The FMEA-ASO team proposed potential failure modes, causes, and effects for each step of the existing workflow. The final list of all potential failure modes was then agreed upon. The FMEA-ASO team members independently assessed the severity, likelihood of occurrence, and likelihood of detection of each failure using the FMEA scoring model. The team then met to agree the final scores.

The FMEA-ASO team reviewed other hospitals’ experiences, the ISMP recommendations,[2] and relevant national and international standards. The individual team members discussed the recommendations with their respective departments. The FMEA-ASO team then met to agree on the applicable remedial actions. The FMEA-ASO team then evaluated the proposed new workflow assuming that all recommendations were implemented. The severity, likelihood of occurrence, and likelihood of detection of potential failure modes were evaluated. A new RPN was then calculated for each step in the proposed workflow. The RPNs for each step of the existing and proposed workflows (assuming implementation of all recommendations) were then compared.

The MSO presented the FMEA-ASO team’s recommendations to the other MSOs of the MNGHA network. Suggestions and approval for implementation of the agreed actions were sought. The MSO summarized the recommended actions and their rationale and strengths, and the actions were prioritized and target dates for their implementation were set. This summary of the ASO-FMEA report was communicated to the departments tasked with the implementation of each action.

Statistical Analysis

The RPN for each step of the existing and proposed workflows (assuming all recommendations had been implemented) were calculated by multiplying the scores for severity, likelihood of occurrence, and likelihood of detection. The RPN were summarized as median and mean ± SD and were compared using the Wilcoxon signed rank test for paired samples. All statistical analyses were performed using Microsoft Excel (version 2016).

RESULTS

Existing Workflow

The team identified five major steps in the ASO workflow (Fig. 2):

1. A prescriber orders a medication.

2. A pharmacist reviews and approves the medication order.

3. The medication is administered by nursing staff every day while the order is active.

4. The most responsible physician or a member of their team (i.e., a prescriber) and the patient’s primary nurse for that day should review the patient’s active medications list. Orders that should be continued but will trigger an ASO within the next 24 hours must be identified.

5. To continue the medication and prevent omissions, it must be reordered by a physician, ideally before the ASO is triggered. The ASO is triggered if the medication is not reordered before midnight on the predefined date. If the ASO is triggered, the medication disappears from the list of active medications in the CPOE system in the physicians’ module, the active medications list in the pharmacy module, and the medication administration record in the nursing module.

As shown in Table 2, the FMEA-ASO team identified eight potential failure modes, 17 failure effects, and 31 causes of failure in the existing ASO workflow. Note that the severity of the failure effect of each step was assigned the same score (7) because failure of the ASO process ultimately results in a medication omission error. Assigning a severity score for every medication class would be impractical. Thus, for simplicity the score of 7 was assigned based on the judgement of the ASO-FMEA team on the potential consequences of commonly omitted medications (e.g., antibiotics) and the index event (i.e., a seizure induced by omission of an antiepileptic agent). The mean RPN of the existing workflow was 365.4 (SD 114.5; median 280). Steps 4 (RPN 567) and 5 (RPN 420) were the highest risk points in the existing workflow (Table 2). These findings guided the development of a new workflow to minimize medication omission errors.

Table 2.

Failure modes in the existing medication Automatic Stop Order (ASO) workflow, risk priority numbers and recommended interventions

Process Step	Failure Modes	Failure Effect	Failure Causes	Severity (1–10)	Likelihood of Occurrence (1–10)	Likelihood of Detection (1–10)	RPN (0–1000)
Step 1: Prescriber orders a medication (e.g., Levetiracetam 1 g q12h IV for 7 days)	Entering undesirable duration Intended duration was not entered when medication was ordered.	Duration of therapy will be insufficient.	Default durations for therapies are preset in the HIS. Hospital policies specify an ASO duration for medications (see Table 1). Pharmacological properties of medication. Physicians’ lack of knowledge of the ASO process. Physicians’ lack of awareness about the importance of specifying the correct duration of therapy when ordering medications. The system changes the duration of antimicrobial therapy after the selection of the intention of therapy (i.e., prophylactic, empirical, or therapeutic).	7	8	5	280
Step 2: Pharmacist reviews the ordered medication	Reviewing and approving an order with an undesirable duration	Duration of therapy will be insufficient.	The proper duration is not known. Technical limitation of default medication durations set in the HIS. Editing an inappropriate order (e.g., to change the dilution) automatically changes the duration) Hospital policy specifies an ASO duration for medications.	7	8	5	280
Step 3: Medication administration by nursing staff	A medication that triggers an ASO and is not renewed is removed from the active list in the MAR in the nursing module and the active orders panel of the physicians’ module The default physicians’ order entry tab does not show MAR Failure of the nurse to request the renewal of medication that will trigger an ASO within the next 24 hours	Dose omission medication errors Treatment failure Patient harm	Policies do not clearly define healthcare providers’ responsibilities for reviewing medications that may trigger an ASO. The absence of a clear warning that a medication will trigger an ASO or prescriber notification when ASO is triggered. HIS system downtime	7	8	5	280
Step 4: 24 hours before the ASO date, the prescriber and the nurse should review ASO medications. The remaining duration of the order appears in (1) a. Physician and pharmacy module in a column beside drug name as red zero; and (2) Nursing module, beside drug name as 4/4.	Medication not reordered or renewed Medication will stop	Patient does not receive medication. Complications or harm due to missed doses of medication. The last day in countdown may be missed by prescriber. The last day in count up could be missed by the nurse. Therapy interruption Treatment failure Staff stress	Red color of the number zero is not clear on screen. The use of count up in the nurses’ module and countdown in the physicians’ module in the HIS may cause miscommunication. The healthcare provider may think that the course of treatment has been completed and the medication does not need to be renewed. Unable to filter medication list in the HIS according to ASO duration. The current policies do not clearly define healthcare providers’ roles in relation to the review of medications’ ASO dates and reordering of medications. Lack of proper alert in the HIS for medications that will trigger an ASO. Workload Absence of color change in the medication duration on the day that the ASO will be triggered in the nursing module. The medication list in the HIS is not arranged by ASO date. Nurse does not ask physician to renew the order. Nurse may ask the prescriber, but they forget to renew medication. Nurse forgets to hand over the need to renew medications to the next shift. Nurse endorses the issue over to the next shift but the next shift fails to act on this. After the ASO is triggered, the medication is removed from the active list so the order may be forgotten. There is no alert highlighting that the ASO was triggered and the medication was discontinued. Physicians do not do a written handover. Healthcare providers may overlook some medications when several need to be reordered.	7	9	9	567
Step 5: ASO medications disappear from the HIS	Healthcare professionals are unable to review medications for which ASO has been triggered.	Patient does not receive medication. Patient develops complication due to omission of medication. Increased pharmacy workload as the missed dose will have to be provided as a STAT order. Transfer patient to high dependency units. Consultation of other specialties and services	Physician does not notice that medication has automatically stopped. Lack of ASO medication list for all healthcare professionals. Entering the wrong duration when the medication was first ordered.	7	6	10	420

The RPN for each failure mode was calculated by multiplying the scores for severity, likelihood of occurrence, and likelihood of detection. As each factor is assigned a score between 1 and 10, the highest RPN possible is 1000.

HIS: health informatics system; IV, intravenous; MAR: medication administration record; MRN: medical record number; RPN: risk priority number.

Proposed New Workflow

For each step of the existing process, specific, measurable, achievable, realistic, and timed (SMART) interventions were recommended (Table 3). The strength of the action, the department responsible for their implementation, and a target date for completion are shown in Supplemental Table S1. The actions recommended by the FMEA-ASO team were:

Table 3.

Rescoring of failure modes and recalculation of the risk priority numbers for new workflow assuming implementation of recommended actions

Process Step Number and Failure Mode	Recommended Action	Revised Severity (1–10)	Revised Occurrence (1–10)	Revised Detection (1–10)	Revised RPN (0–1000)
Step 1: Entering undesirable duration. Intended duration was not entered when medication was ordered.	Increase physicians’ awareness of the need to: Adjust the ASO date to enter the correct duration when ordering medications. Review ASO stop dates and remaining medication durations daily. Reorder medications before ASO is triggered to prevent dose omissions when necessary.	7	3	4	84
Step 2: Pharmacist or clinical pharmacist reviews and approves order with undesirable duration.	Increase pharmacists’ and clinical pharmacists’ awareness of the need to review medication duration.	7	4	6	168
Step 3. Medication order not listed in nursing module. Physician cannot view the administration of ordered medications within the order entry screen.	Increase nurses’ awareness of the need to review medication duration.	7	4	6	168
Step 4. Medication not ordered or renewed. Medication will discontinue.	Create an application within the HIS to allow prescribers, pharmacists, clinical pharmacists, and nurses to search for and print lists of patients’ medications sorted by ASO date. The tool should allow patients’ medications lists to be identified by MRN or by ward. This app should be accessible by all physicians, pharmacists, clinical pharmacists, and nurses and should be able to display medications by ASO date. The application should allow prescribers to acknowledge that they reviewed the medications list and ASO dates, add comments and renew medications. The application should allow clinical pharmacists and nurses to acknowledge that they reviewed the medications list and ASO dates and write comments. A pop-up alert should remind the nurse and physician to use the application to review the medications list and ASO dates every day. Modify hospital policy to define healthcare providers’ responsibility to review and renew medication and terminology of counting up or counting down to completion of medication course. The appearance of the term ASO needs to be made more obvious at the top of column for ASO dates in HIS order list. A sort function should be accessible when the mouse cursor hovers over the definition of the ASO.	7	4	4	112
Step 5. Healthcare professional unable to review ASO medication list.	Combine the order list screen with administration record and display medications administered for at least 7 days before the current date. Create the application described in step 4.	7	4	2	56

The RPNs were calculated by multiplying the scores for severity, likelihood of occurrence, and likelihood of detection. As each factor is assigned a score between 1 and 10, the highest RPN possible is 1000.

ASO: automatic stop order; HIS: health informatics system; MRN: medical record number; RPN: risk priority number.

1. Modifications to the HIS

   • Create an application to search the HIS for medications by ASO date. This application should be available to all healthcare professionals.
   • Allow prescribers to renew medications through this application.
   • Create a pop-up alert listing medications for which an ASO will be triggered within the next 24 hours whenever prescribers or nurses access the patient’s profile.
   • Create a new display for medications that combines the order entry screen and active order list with an administration record that displays the active medications and all medications administered during the preceding 7 days.
   • Increase the clarity of the appearance of the ASO column header in the order list in the HIS for all healthcare professionals.
2. Modifications to policies and procedures

   • Modify hospital policies to clarify the role of each healthcare provider in reviewing, ordering, and reordering of medication as well as the ASO process.
3. Awareness and education

   • Train all healthcare professionals on the existing ASO process and display of ASO dates in the HIS. Retrain all healthcare professionals on the new ASO workflow after implementation of the recommended actions.

4. Audit and monitoring

Potential Impact

The RPN of the proposed new workflow (assuming that all recommendations were implemented successfully), suggests that implementation of the FMEA-ASO team’s recommendations would decrease the RPN at each step (Table 3). Table 4 compares the RPNs of the existing and proposed workflows. The mean RPN of the proposed workflow is 117.6 (SD 44.8; median 112). The mean difference from the existing workflow is 247.8 (SD 44.8; median 196). A one-tailed Wilcoxon signed rank test indicated that the RPN scores for the proposed new workflow are significantly lower than the RPN scores of the existing workflow (n = 5; z = −2.02; W = 0; p < 0.05).

Table 4.

Comparison of the RPNs of the current process with the proposed new workflow (assuming all recommended modifications are made).

Process Step	RPN		Difference in RPN
	Existing ASO Workflow	Proposed ASO Workflow
1	280	84	196
2	280	168	112
3	280	168	112
4	567	112	455
5	420	56	464
Total (mean ± SD)	365.4 ± 114.5	117.6 ± 44.8	247.8 ± 44.8

ASO: automatic stop order; RPN: risk priority number.

DISCUSSION

The present quality improvement initiative demonstrates that FMEA can detect the flaws in CPOE systems that may cause medication omission errors. This requires a multidisciplinary team with an appropriate skill mix to comprehensively review the process, identify failure modes, stratify their risk (i.e., calculate RPN), and suggest preventative actions.[18]

The RPN can focus priorities. Steps with higher RPNs require more attention and effort to prevent failures. The theoretical recalculation of predicted RPN for the proposed new workflow (assuming implementation of all recommended interventions) can help to subjectively evaluate whether the proposed changes are likely to reduce the risk of errors.

The medication ASO process in our institution’s CPOE system had five main steps. The highest RPN was associated with step 4 (RPN = 567). A prescriber and a nurse must review the ordered medications every day and identify those orders with an impending ASO trigger date. If these medications should be continued, they must be reordered before the ASO is triggered to prevent a dose omission error. The problem is exacerbated by differences in the way the remaining duration of the medication order is displayed in the nursing, pharmacy, and physician modules of the CPOE (Supplemental Fig. S1, available online).

In the physicians’ and pharmacists’ modules, a column beside that for the drug name displays a countdown for the remaining duration in days (Supplemental Fig. S1A). On the day that the ASO will be triggered a red zero is displayed in this column. However, in the nursing module, the number of days the medication order has been active is shown next to the drug as a fraction of the total duration of the course (e.g., 3/4; Supplemental Fig. S1B). On the day that the ASO will be triggered, a fraction that represents 1 is displayed (e.g., 4/4).

The second-highest RPN was associated with step 5 (RPN = 420). The medications disappear from the active orders list in the CPOE at midnight on the day the ASO is triggered (i.e., on-call time when the primary medical team is not on duty). These steps are critical points when errors are most likely to occur. So, when modifying the existing ASO workflow it was important to prioritize the consideration and implementation of interventions that would reduce the risk of errors occurring at these points.

However, as several of the ASO-FMEA team’s recommendations have not yet been implemented, the impact of the proposed new workflow on medication omission errors remains uncertain. The MSP will audit the implementation of the recommendations and monitor the reported medication errors to determine the impact of the changes. Yet, previous studies have demonstrated the benefit of FMEA on medication safety in other settings. For example, FMEA identified potential risks before implementation of CPOE for chemotherapy.[19] Thus, FMEA can effectively mitigate risks and support the development of safer hospital systems.[20]

Limitations

Conducting an FMEA is labor intensive. It requires experts, stakeholders, and champions from the relevant departments. Several meetings are needed. Moreover, the tool used to score the severity, likelihood of occurrence, and likelihood of detection of failure modes is subjective. However, the objectivity of calculated RPN can be improved if relevant experts not directly involved in the FMEA independently review the conclusions of the FMEA team.

CONCLUSION

The present study describes the use of FMEA to redesign a complex medication ASO system that mini-RCA2 identified as the cause of several medication omission errors. FMEA is a practical, proactive tool that may help to predict system failures, their causes, and likelihood of occurrence (i.e., risks). Future studies should explore the potential benefits of this use of FMEA in other healthcare settings with different CPOE systems to explore the external validity of the findings of the current study.

Supplemental Material

Supplemental material is available online with the article.