Abstract
Objective: Medication order voiding allows clinicians to indicate that an existing order was placed in error. We explored whether the order voiding function could be used to record and study medication ordering errors.
Materials and Methods: We examined medication orders from an academic medical center for a 6-year period (2006–2011; n = 5 804 150). We categorized orders based on status (void, not void) and clinician-provided reasons for voiding. We used multivariable logistic regression to investigate the association between order voiding and clinician, patient, and order characteristics. We conducted chart reviews on a random sample of voided orders (n = 198) to investigate the rate of medication ordering errors among voided orders, and the accuracy of clinician-provided reasons for voiding.
Results: We found that 0.49% of all orders were voided. Order voiding was associated with clinician type (physician, pharmacist, nurse, student, other) and order type (inpatient, prescription, home medications by history). An estimated 70 ± 10% of voided orders were due to medication ordering errors. Clinician-provided reasons for voiding were reasonably predictive of the actual cause of error for duplicate orders (72%), but not for other reasons.
Discussion and Conclusion: Medication safety initiatives require availability of error data to create repositories for learning and training. The voiding function is available in several electronic health record systems, so order voiding could provide a low-effort mechanism for self-reporting of medication ordering errors. Additional clinician training could help increase the quality of such reporting.
Keywords: medication errors, patient safety, CPOE, medication order voiding
BACKGROUND AND SIGNIFICANCE
Computerized provider order entry (CPOE) systems promote medication order standardization through electronic entry of legible and complete medication orders,1 mitigating the potential for medication errors.2–4 In addition to supporting medication safety, CPOEs can improve clinical workflow efficiency through effective collaboration and coordination of tasks,5 reduction of misinterpreted orders, and effective medication administration.6 Despite these advantages, medication safety concerns persist.2,7 A recent Agency for Healthcare Research and Quality report identified that adverse drug events resulting from medication errors affect nearly 5% of hospitalized patients.8
Past research on CPOE-based medication errors relied on retrospective chart reviews,9,10 medication error reports,11 nonexperimental pre-post studies,12 malpractice claims,13 voluntary self-reports,14,15 and observational studies.7,16 For example, Schiff et al.17 developed a taxonomy of CPOE-based medication errors using a retrospective review of the United States Pharmacopeia’s MEDMARX database of error reports. Their taxonomy included missing patient instructions, wrong dosages, wrong schedules, duplicate orders, delayed orders, routing issues, wrong times selected, wrong patients, and nurse administration errors. Koppel et al.7 used an observational study with interviews, surveys, and focus groups to characterize medication risks with CPOE, including fragmented information displays, mistaken guidelines, and inflexible ordering formats; similar issues have also been reported by other researchers.18–20
Retrospective approaches for investigating medication errors are valuable. However, they have several limitations. For example, use of malpractice claim data has been reported as a “crude” and “questionable” measure of errors21,22; similarly, incident report data have been described as “unreliable” and insensitive.23 Additionally, many studies, especially those that relied on observational approaches or malpractice claims, used small samples of errors, with a majority reporting on serious patient harm or death.24
Besides these methods, researchers have used electronic health record (EHR)-integrated indication alerts to identify drug-name confusions and wrong-patient errors.25–27 However, because these alerts were designed to detect and recover from specific medication ordering errors, they are limited in their ability to capture other types of ordering errors. Additionally, using alerts to capture the entire spectrum of erroneous medication orders can lead to unintended consequences, such as alert fatigue.28 Currently, we lack a simple, effective, and clinically relevant approach to identify and record erroneous medication orders without disrupting clinician workflow.
MEDICATION ORDER VOIDING
Medication order voiding is a CPOE-based function by which a clinician can identify and remove erroneous medication orders from a patient’s active medication list.29 By voiding an order, the clinician indicates that it was placed in error. A voided order can serve as an index of a medication ordering error; analysis of such voided orders can help in tracing the source and cause of these errors and their potential impact on patient care.
The process of medication order voiding within the Cerner EHR involves the following steps: first, the clinician selects a medication order to be voided; second, the “void” option is selected from a menu list; and finally, the clinician selects a reason for the voiding action from a dropdown menu. We refer to this as the clinician-provided reason for voiding. Within this dropdown menu, there are 8 choices: blank (no reason provided), order on wrong encounter, wrong patient, incorrect ordering physician, duplicate order, system date error, voiding student order, and improperly composed order. Figure 1 illustrates the voiding workflow.
Figure 1.
Clinical workflow of the medication order voiding process within the study site implementation of Cerner; the presented case is for a test patient.
Based on a cursory appraisal of prominent EHR systems, the voiding function, or a similar alternative suggesting an erroneous medication order, is available in both Cerner and Epic systems. The outpatient Longitudinal Medical Record system at Partners Healthcare also has a similar functionality, labeled “Error (erroneous entry).” However, it must be noted that voiding functions are optional and are often enabled based on institutional preferences.
In this paper, we investigate the viability of using the medication order voiding function to identify and study medication ordering errors. Using a large retrospective sample of medication orders and chart reviews, we describe the characteristics of voided orders, the proportion of voided orders that were medication ordering errors, and the accuracy of clinician-provided reasons for voiding.
METHOD
Setting
This study was conducted at the University of Illinois Hospital and Health Sciences System (UI Health). UI Health is a tertiary urban academic medical center that includes a 495-bed hospital, an emergency department, and 23 primary and specialty care clinics. In fiscal year 2015, UI Health had approximately 47 000 emergency department visits, 20 000 hospitalizations, and 7000 inpatient surgeries. At the time of this study, medication orders were placed with CPOE using Cerner Powerchart® and Firstnet®. Inpatient CPOE use started on November 1, 1999, and outpatient use started on March 1, 2007.
Per prescribing laws in Illinois, pharmacists and nurses can initiate medication orders based on verbal, written, or protocol orders from physicians. Pharmacy and medical students can propose medication orders, but such orders require a physician’s approval.
Data
We retrieved data on all original medication orders at UI Health during the period between January 1, 2006, and December 31, 2011. This dataset included medication order status (voided, not voided), patient characteristics (race/ethnicity, sex, age), ordering clinician characteristics (physician, pharmacist, nurse, student, other), and type of medication order (inpatient, prescription, home medication by history) with timestamp.
Data analysis was conducted in 2 stages. First, we performed a retrospective analysis of the 6-year dataset of all medication orders to describe the characteristics of voided orders. Second, using a random sample of voided orders, we performed patient chart reviews to determine the proportion of voided orders that were medication ordering errors. The Institutional Review Board of the University of Illinois at Chicago approved this study.
Analysis of medication orders
Data variables and processing
Order status was the outcome variable of interest. Explanatory variables included patient characteristics, original order characteristics (order type, shift, weekday), and clinician characteristics. Reported race/ethnicity was categorized into white, black, Hispanic, and other; order type was classified as inpatient (ie, medication order for a hospitalized patient), prescription, and home medication by history. Home medication by history included a nonactionable record of medications that patients were taking at home and were not recorded as prescriptions. These can be recorded at any time: during an inpatient admission, at discharge, or at an ambulatory setting. Shift was categorized based on shift time: day (7 a.m. to 5 p.m.), night (5 p.m. to 12 midnight), or overnight (12 midnight to 7 a.m.); weekday type was classified as work weekday (Monday through Friday) or weekend (Saturday or Sunday). Clinician type referred to the clinical provider who was entering orders and was categorized as physician, pharmacist, nurse, student, or other. Age was treated as a continuous variable. A detailed description of each variable is provided in Appendix Table 1.
Table 1.
Odds of an order being voided, adjusted odds ratios, and 95% confidence intervals with all exploratory variables
| Variable | AOR (95% CI) | P valuea |
|---|---|---|
| Sex (ref = female) | ||
| Male | 0.98 (0.96–1.01) | .25 |
| Race/Ethnicity (ref = white) | ||
| Black | 0.93 (0.90–0.96) | <.0001 |
| Hispanic | 0.91 (0.88–0.95) | <.0001 |
| Other | 0.92 (0.89–0.96) | .0001 |
| Ageb | 0.98 (0.94–0.99) | <.0001 |
| Order Type (ref = inpatient order) | ||
| Outpatient prescription | 0.57 (0.55–0.59) | <.0001 |
| Home medications by history | 1.88 (1.81–1.95) | <.0001 |
| Weekday (ref = work weekday) | ||
| Weekend | 0.90 (0.86–0.93) | <.0001 |
| Shift (ref = day) | ||
| Night | 0.93 (0.90–0.96) | <.0001 |
| Overnight | 0.69 (0.66–0.72) | <.0001 |
| Clinician Type (ref = physician) | ||
| Pharmacist | 0.82 (0.78–0.85) | <.0001 |
| Nurse | 1.72 (1.66–1.78) | <.0001 |
| Student | 6.76 (6.39–7.15) | <.0001 |
| Other | 1.01 (0.95–1.07) | 0.66 |
aWald chi-square P value, significance at P < .05.
bAge was standardized in the model.
Statistical analysis
Univariate analysis was performed for outcome and explanatory variables to develop a descriptive characterization of order voiding. Bivariate analysis was conducted for all explanatory variables by order status. Significance of association was measured using Chi-square tests. Crude odds ratios were generated with 95% confidence intervals to measure the strength and direction of association between explanatory variables and order status. Before introducing new variables into the model, the potential for high correlation among variables was assessed using Pearson’s correlation coefficient. Results of the bivariate analysis informed the selection of a multivariate logistic regression model that included all explanatory variables. The strength of association was interpreted using adjusted odds ratio (AOR) with 95% confidence intervals.
Based on this preliminary analysis, we developed 2 models: first, a logistic regression model with all explanatory variables, and second, a logistic regression model with all explanatory variables and an interaction between clinician and order type. All analyses were performed using SAS v 9.4 (SAS Institute, Inc., Cary, NC, USA).
Chart review of voided orders
Patient chart selection for review
Voided orders were grouped based on the 8 clinician-provided reasons for voiding. A simple randomization approach was used to select 25 voided orders from each of these 8 reasons. Because of incomplete data on 2 categories, order on wrong encounter and system-date error (each with 24 voided orders), we ended up with a total of 198 charts for review.
Data coding and validation
We defined medication ordering errors as those that met the following criteria: “any preventable event that may cause or lead to inappropriate medication use or patient harm while medication is in the control of the healthcare professional.”30
Chart review was conducted in 2 phases. In the first phase, 2 of 3 physician reviewers (authors WLG, AS, and SGP) independently reviewed each patient’s chart to assess whether a voided order was a medication ordering error. In case of a disagreement between the 2 reviewers, or if 1 reviewer noted an inability to make a decision, a third adjudicator independently conducted the chart review. Final dispositions on disagreements were based on majority decisions among the 3 reviewers.
In the second phase, 1 reviewer (WLG) evaluated all voided orders that were classified as medication ordering errors (n = 152). The purpose of this second review was to verify whether the clinician-provided reasons for order voiding were indicative of the actual reasons for medication ordering errors. In this secondary analysis, for the subset of medication ordering errors, we identified the “actual reasons” for errors and compared them with the clinician-provided reasons.
Statistical analysis of chart review data
The proportion of voided orders that were medication ordering errors was determined by the individual proportions for each reason weighted by the total number of voided orders for that reason. The standard error of the proportion (SEP) for the entire population was estimated in a similar manner, using the SEP for each reason weighted by the number of voided orders for that reason. A significance level of P < .05 was used for all comparisons.
RESULTS
Analysis of medication orders
Sample characteristics
There was a total of 5 804 150 medication orders over a 6-year period, and voiding occurred at a rate of 4.9 voids per 1000 orders (0.49%). Patients were primarily female (59.6%), black (50.9%), and middle-aged (M = 45.5, S.D. = 22.3 years). Physicians generated most of the orders (68.4%) in inpatient settings (62.7%).
Patients who had voided orders were also primarily female (59.0%), black (49.1%), and middle-aged (M = 46.5, S.D. = 22.2 years). Voided orders were primarily placed by physicians (56.8%) and originated in inpatient settings (60.8%). Based on bivariate analysis, all explanatory variables were found to be significantly associated with order status (P < .05). A summary can be found in the Appendix Table 2.
Table 2.
Adjusted odds ratios and 95% confidence intervals considering an interaction between order type and clinician type
| Order type | AORa (95% CI) | P valueb |
|---|---|---|
| Inpatient orders (ref = physician) | ||
| Pharmacist | 0.46 (0.43, 0.49) | <.0001 |
| Nurse | 1.56 (1.49–1.63) | <.0001 |
| Student | 11.4 (10.74–12.12) | <.0001 |
| Other | 0.93 (0.85–1.03) | .15 |
| Outpatient prescriptions (ref = physician) | ||
| Pharmacist | 1.89 (1.74–2.04) | <.0001 |
| Nurse | 1.95 (1.81–2.09) | <.0001 |
| Student | 22.6 (9.20–55.69) | <.0001 |
| Other | 1.11 (0.94–1.32) | 21 |
| Home medications by history (ref = physician) | ||
| Pharmacist | 1.68 (1.46–1.94) | <.0001 |
| Nurse | 1.90 (1.68–2.15) | <.0001 |
| Student | 3.37 (2.90–3.91) | <.0001 |
| Other | 1.09 (0.95–1.26) | .21 |
aOdds ratios are adjusted for sex, race/ethnicity, age, weekday, and shift type.
bWald chi-square P value, significance at P < .05.
Characteristics of voided orders
The variables were not highly correlated (r < 0.30), and all explanatory variables were significant in bivariate analysis. Therefore, all explanatory variables – patient, order, and clinician characteristics – were used to develop a multivariable logistic regression model. Patient characteristics were not significantly associated with order status; however, order and clinician characteristics were significantly associated with order status. Compared to inpatient orders, prescriptions had lower odds (AOR = 0.57) and home medications by history had higher odds (AOR = 1.89) of being voided. Compared to physicians, students and nurses had higher odds of their orders being voided (students AOR = 6.76, nurses AOR = 1.73), while pharmacists had lower odds of their orders being voided (AOR = 0.82). A summary of the AORs for all explanatory variables is provided in Table 1.
Interaction between clinician and order types
We introduced an interaction term to the multivariable logistic regression. This was based on our post hoc hypothesis that order voiding was more strongly related to clinician and order type than patient characteristics. Compared to physicians, there were lower odds of inpatient pharmacist orders being voided (AOR = 0.46) and higher odds of outpatient pharmacist orders being voided (AOR = 1.68). Overall, student and nursing orders had higher odds of being voided compared to physician orders. A summary is provided in Table 2.
Medication ordering errors among voided orders from chart reviews
We found that 152 of 198 voided orders were medication ordering errors. These errors were distributed across 7 “actual reasons” for errors: duplicate order, incorrect ordering physician, not clinically appropriate, wrong drug, order on wrong encounter, wrong patient, and wrong route/drug/dose/strength. A description of the 7 reasons, the frequency of medication ordering errors in each error category, and the relationship of actual reasons to original clinician-provided reasons for voiding is provided in Table 3.
Table 3.
Medication ordering errors (ie, actual reasons for medication ordering errors), their descriptions, their frequency of occurrence (N), and the relationship of actual reasons to clinician-provided reasons at the time of voiding
| Actual Reason for Medication Ordering Error (from Chart Reviews) | Description of Medication Ordering Error | N | Relationship to Clinician- Provided Voiding Reason |
|---|---|---|---|
| Wrong route/dose/schedule/strength | Any error in the construction of the order: route, dose, schedule, or strength | 60 | Improperly composed order |
| Duplicate order | An identical order to one already active in the list of medications | 36 | Duplicate order |
| Not clinically appropriate | An order that was cancelled rapidly, or prior to administration without an indication noted in the patient chart | 34 | New |
| Wrong patient | An order indicating that the medication was given to the wrong patient | 15 | Wrong patient |
| Incorrect ordering physician | An RN or PharmD chooses an incorrect physician giving a verbal order | 3 | Incorrect ordering physician |
| Order on wrong encounter | An order placed on the incorrect patient encounter episode that causes it to be nonactionable for the intended care of the patient | 3 | Order on wrong encounter |
| Wrong drug | An order indicating the wrong medication for the patient | 1 | New |
The new “actual reasons” for errors are italicized.
Based on our comparison of the actual and clinician-provided reasons for medication ordering errors, we found that 3 of the clinician-provided reasons were not indicative of the actual reasons: no reason provided (ie, reason left blank), voiding a student order, and system-date error. For example, voiding a student order was not indicative of the actual reason for a medication ordering error; often the actual reason was voiding an existing order (ie, voiding a duplicate order). In other words, although the voided medication order represented an ordering error, the attributed reason was inaccurate.
We identified 2 new actual reasons for medication ordering errors that were not in the clinician-provided reasons for order voiding: wrong drug error and not clinically appropriate. “Improperly composed order” was renamed as “Wrong route/dose/schedule/strength.”
The most common cause for a medication ordering error was “wrong route/dose/schedule/strength,” (n = 60) followed by duplicate ordering (n = 36). Given that the charts for review were not randomly selected from the overall population, the voided medication orders were not uniformly distributed. Hence, we estimated the overall rate of medication ordering errors in the entire population by weighting the medication ordering error rate for each reason with the proportion of each particular reason. Based on this estimation, the overall proportion of voided orders that were medication ordering errors was 70 ± 10%.
The proportion of medication ordering errors for each voiding reason varied from a high of 72% for duplicate orders to 48% for wrong patient and wrong route/dose/schedule/strength, to a low of 8% for orders on wrong encounters (P < .001). Table 4 provides a summary of the proportions of clinician-provided and actual reasons for medication ordering errors.
Table 4.
Proportion of medication ordering errors for clinician-provided reasons and actual reasons for order voiding
| Clinician-Provided Reason for Voiding | N | Proportion of Medication Ordering Error (±SEP) (%) | Corresponding Actual Reason for Medication Ordering Error (Based on Chart Review) | Proportion of Medication Ordering Error (± SEP) |
|---|---|---|---|---|
| Duplicate order | 25 | 72 ± 9 | Duplicate order | 72 ± 9% |
| Incorrect ordering physician | 25 | 76 ± 9 | Incorrect ordering physician | 12 ± 6% |
| Order on wrong encounter | 24 | 79 ± 8 | Order on wrong encounter | 8 ± 6% |
| Wrong patient | 25 | 100 | Wrong patient | 48 ± 10% |
| Improperly composed order | 25 | 80 ± 8 | Wrong route/dose/schedule/strength | 48 ± 10% |
| System date error | 24 | 75 ± 9 | Not applicable | NA |
| Voiding student order | 25 | 80 ± 8 | NA | NA |
| No reason given | 25 | 52 ± 10 | NA | NA |
DISCUSSION
Based on a retrospective analysis of medication orders over a 6-year period, we found that 0.49% of all orders were voided. Chart reviews for a sample of those orders revealed that at least two-thirds of the voided orders were indeed medication ordering errors (70 ± 10%). The proportion of overall medication ordering errors among voided orders is considerably higher than medication error rates reported in current literature, which are estimated to be around 5–10%,31 with a broad overall range.32 Such a high proportion of medication ordering errors among voided orders offers significant promise for the use of voiding as a trigger mechanism for identifying and tracking errors.
Multivariable logistic regression analysis revealed several patterns. We found that order voiding was associated with clinician and order types: inpatient pharmacist orders had lower odds of being voided, while nurse and student inpatient orders had higher odds of being voided. These higher odds could point to greater error surveillance of such orders; currently, student and nurse orders require an attending physician’s co-signature, providing additional oversight. Although pharmacist orders also require a co-signature, their lower odds suggest that pharmacists might be more accurate in their ordering. However, further investigation is necessary.
We found an increased likelihood of order voiding by pharmacists, nurses, and students for home medications by history, a type of “pre-order.” Many clinicians use home medications by history to pre-order medications that are not yet finalized and recorded. However, these pre-orders pose a risk in situations where they get documented in patients’ visit/discharge summaries and medication lists. Such documentation raises medication safety concerns. For instance, pre-orders can lead to confusion for patients regarding their medication regimens. Similarly, they can also raise questions for other clinicians regarding patients’ medication management and reconciliation. Other physicians may also unwittingly trust those who placed the pre-orders and turn pre-order entries into orders with minimal scrutiny and review. Medical student orders also act as pre-orders; often physicians place orders without realizing that they are duplicating active medical student orders. In our sample, nearly 50% of “voiding student orders” were duplicate orders.
Our results also demonstrate that clinician-provided reasons for voiding did not always accurately reflect the reasons found during chart review, except for duplicate and wrong patient orders. This finding is in agreement with prior research showing that options selected from CPOE dropdown lists can be unreliable.33 Based on our results, the voiding function can be especially useful for identifying wrong patient errors that are often difficult to identify without alerting tools such as “retract and reorder.”25
There are several characteristics that make the voiding function a viable mechanism for identifying medication ordering errors. First, a clinician can complete the entire process within the patient’s medical record, including selecting an erroneous medication order and classifying it as a voided order. As a result, this process is within the clinician’s medication ordering workflow. No additional cognitive or physical effort, such as additional clicks or logging in to an external system, is required to complete this task. Second, voided orders are automatically documented as such, hence serve as a rich database of medication ordering errors for further analysis and quality improvement. Creating such a database is simple and can provide valuable information for medication and patient safety units of a hospital. Medication error reporting systems often require a 2-page report to detail an incident.34 As a result, clinicians are unlikely to use these or similar systems to report routine errors that may have had no direct effects on patients.
Third, order voiding provides a potentially standardized approach for documenting medication ordering errors detected by clinicians. Such standardization can help in sharing ordering errors among local and national patient safety organizations. Fourth, although a key development over the last several years has focused on training clinicians to identify and recover from errors,35 data on error correction and recovery have been scant. Much of the data have been based on retrospective reports using case studies of error recovery.36 Medication order voiding can provide a platform for training clinicians to learn from the different types of prevalent medication ordering errors, their root contributors, and proactive strategies to identify and correct them.
For the order voiding function to be more universally and practically useful, 4 things need to occur: (1) the order voiding function needs to be incorporated across all clinical settings; (2) clinicians must be adequately trained on its purpose and procedures for optimal use, along with sufficient support and motivation to use the voiding function; (3) medication ordering error data must be shared with institutional medication safety officers; and (4) classification of voiding reasons needs to be standardized across CPOE systems, with clearly understood operational definitions.
Limitations
This study was based on an analysis of a large set of medication orders from a single academic medical center. Medication order voiding relies on clinicians to identify, intercept, and record ordering errors. In other words, the voiding mechanism can only be used to identify those orders that clinicians recognize as erroneous and utilize the voiding function to appropriately categorize them as voided orders (ie, indicating an ordering error). Although this puts the onus on clinicians, such a self-regulating, meta-cognitive approach has been suggested as a potential mechanism for error mitigation.36 Medication ordering errors identified through the voiding process can have a range of effects, from no effects (eg, voided prior to administration), to administered to the patient with no harm, to administered to the patient with adverse consequences.
The number of medication ordering errors that were not voided was not determined, so we were unable to calculate the sensitivity of the sample of voided orders as a proxy for medication ordering errors. We calculated only the proportion of voided orders that were medication ordering errors. Additionally, because of the retrospective nature of the study, we could not ascertain whether clinicians’ intended purpose for voiding was to indicate erroneous orders. This is particularly the case for the “not clinically appropriate” reason for medication ordering errors. For example, orders in this category were voided prior to administration, indicating a potentially erroneous order; however, there were insufficient data in the chart to verify whether the intent was indeed to void an erroneous order. If all the “not clinically appropriate” orders were not ordering errors as we had classified, the medication ordering error rate would still be high (>50%), not significantly affecting the main result of this study. We also cannot fully explain differences between clinician-provided and actual reasons for voiding obtained from chart reviews. Further research using order voiding alerts and follow-up phone calls with voiding clinicians may be required to identify clinician intent with regard to order voiding.
The variation in adjusted odd ratios does not necessarily imply that more or fewer errors occurred; instead, it suggests that more errors were intercepted through order voiding. Thus, we do not have any conclusive evidence suggesting that pharmacists or nurses place a larger or smaller proportion of erroneous medication orders than physicians, except that more of their orders were voided. Although this finding is interesting, more research is necessary to draw conclusions regarding the voiding rate across different clinician groups. Finally, given the large sample of orders that was used, it is possible that some of the associations may have occurred by chance alone.
CONCLUSION
Review of voided medication orders may be a promising approach to detect, categorize, and track medication ordering errors. Our findings show potential for using voided medication orders as a mechanism for documenting medication ordering errors. This approach to error surveillance could also help to promote a learning health system for medication safety. Although our results are preliminary, further research investigating the reasons for the associations of order voiding with clinician and medication order type might be fruitful in understanding error generation, error recovery, and medication safety surveillance.
ACKNOWLEDGMENTS
We would like to thank Sabita Acharya and Amy Solsman for their assistance with the data organization, data cleaning, and preliminary analysis.
Funding
This work was supported by contract HHSF223201000008I/HHSF22301005T from the US Food and Drug Administration (Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research) and supported in part by grant number U19HS021093 from the Agency for Healthcare Research and Quality and a pilot interdisciplinary award from the College of Applied Health Sciences at the University of Illinois at Chicago. The content is solely the responsibility of the authors and does not necessarily represent the official views of Agency for Healthcare Research and Quality or Food and Drug Administration.
Financial disclosure and competing interests
BLL provides software and consulting services designed to prevent wrong-drug medication errors. His companies had no access to these data or involvement in the study.
Contributors
WLG, TGK, JA, and GDS conceptualized and designed the study. WLG, TGK, SGP, and AS organized the data, conducted the analyses, and drafted the initial manuscript. TGK, WLG, JA, BLL, GDS, and AW were involved in interpreting the results and critically reviewing, revising, and finalizing the manuscript. All authors were significantly involved in all stages of the study and approved the final version of the manuscript.
References
- 1. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Int Med. 2003;16312:1409–16. [DOI] [PubMed] [Google Scholar]
- 2. Bates DW, Leape LL, Cullen DJ et al. . Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311–16. [DOI] [PubMed] [Google Scholar]
- 3. Bates DW, Teich JM, Lee J et al. . The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;64:313–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Kuperman GJ, Gibson RF. Computer physician order entry: Benefits, costs, and issues. Ann Intern Med. 2003;1391:31–39. [DOI] [PubMed] [Google Scholar]
- 5. Potts AL, Barr FE, Gregory DF et al. . Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004;1131:59–63. [DOI] [PubMed] [Google Scholar]
- 6. Cordero L, Kuehn L, Kumar RR et al. . Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol. 2004;242:88–93. [DOI] [PubMed] [Google Scholar]
- 7. Koppel R, Metlay JP, Cohen A et al. . Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;29310:1197–203. [DOI] [PubMed] [Google Scholar]
- 8. AHRQ Patient Safety Network. Computerized provider order entry. AHRQ PSNet Patient Safety Primer 2015 [cited 2015 February 9, 2015]. https://psnet.ahrq.gov/primers/primer/6/. Accessed January 19, 2017.
- 9. Westbrook JI, Baysari MT, Ling L et al. . The safety of electronic prescribing: Manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc. 2013;206:1159–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Her QL, Amato MG, Seger DL et al. . The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting. J Am Med Inform Assoc. 2016;235:924–33. [DOI] [PubMed] [Google Scholar]
- 11. Slight SP, Eguale T, Amato MG et al. . The vulnerabilities of computerized physician order entry systems: A qualitative study. J Am Med Inform Assoc. 2015;232:311–16. [DOI] [PubMed] [Google Scholar]
- 12. Lyons AM, Sward KA, Deshmukh VG et al. . Impact of computerized provider order entry (CPOE) on length of stay and mortality. J Am Med Inform Assoc. 2016;242:303–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Bishop TF, Ryan AM, Casalino LP. Paid malpractice claims for adverse events in inpatient and outpatient settings. JAMA. 2011;30523:2427–31. [DOI] [PubMed] [Google Scholar]
- 14. Perrow C. Normal Accidents: Living with High Risk Technologies. Princeton, NJ: Princeton University Press; 2011. [Google Scholar]
- 15. Radley DC, Wasserman MR, Olsho LE et al. . Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc. 2013;203:470–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Park YS, Chung SP, You JS et al. . Effectiveness of a multidisciplinary critical pathway based on a computerised physician order entry system for st-segment elevation myocardial infarction management in the emergency department: a retrospective observational study. BMJ Open. 2016;68:e011429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Schiff GD, Amato MG, Eguale T et al. . Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Safety. 2015;24:264–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Donyai P, O’Grady K, Jacklin A et al. . The effects of electronic prescribing on the quality of prescribing. Br J Clin Pharmacol. 2007;652:230–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Lesar TS, Briceland LL, Delcoure K et al. . Medication prescribing errors in a teaching hospital. JAMA. 1990;26317:2329–34. [PubMed] [Google Scholar]
- 20. Vaidya V, Sowan AK, Mills ME et al. . Evaluating the safety and efficiency of a cpoe system for continuous medication infusions in a pediatric ICU. In American Medical Informatics Association Annual Symposium. Washington DC: AMIA; 2006. [PMC free article] [PubMed] [Google Scholar]
- 21. Brennan TA, Leape LL, Laird NM et al. . Incidence of adverse events and negligence in hospitalized patients. New Engl J Med. 1991;3246:370–76. [DOI] [PubMed] [Google Scholar]
- 22. Brennan TA, Sox CM, Burstin HR. Relation between negligent adverse events and the outcomes of medical-malpractice litigation. New Engl J Med. 1996;33526:1963–67. [DOI] [PubMed] [Google Scholar]
- 23. Anderson JE, Kodate N, Walters R et al. . Can incident reporting improve safety? Healthcare practitioners’ views of the effectiveness of incident reporting. Int J Qual Healthcare. 2013;252:141–50. [DOI] [PubMed] [Google Scholar]
- 24. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc. 2005;124:377–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Adelman JS, Kalkut GE, Schechter CB et al. . Understanding and preventing wrong-patient electronic orders: a randomized controlled trial. JAMA. 2013;202:305–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Galanter W, Falck S, Burns M et al. . Indication-based prescribing prevents wrong-patient medication errors in computerized provider order entry (CPOE). J Am Med Inform Assoc. 2013;203:477–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Galanter WL, Bryson ML, Falck S et al. . Indication alerts intercept drug name confusion errors during computerized entry of medication orders. PloS ONE. 2014;97:e101977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Kesselheim AS, Cresswell K, Phansalkar S et al. . Clinical decision support systems could be modified to reduce ‘alert fatigue’while still minimizing the risk of litigation. Health Affairs. 2011;3012:2310–17. [DOI] [PubMed] [Google Scholar]
- 29. Cerner. Ucern. [cited May 2, 2016]. https://wiki.ucern.com/pages/viewpageattachments.action?pageId=1067811766. Accessed May 2, 2016.
- 30. National Coordinating Council for Medication Error Reporting and Prevention. NCC MERP Taxonomy of Medication Errors. Taxonomy of Medication Errors. 2001. http://www.nccmerp.org/sites/default/files/taxonomy2001-07-31.pdf. Accessed January 19, 2017. [DOI] [PubMed] [Google Scholar]
- 31. Lewis PJ, Dornan T, Taylor D et al. . Prevalence, incidence and nature of prescribing errors in hospital inpatients: a systematic review. Drug Safety. 2009;325:379–89. [DOI] [PubMed] [Google Scholar]
- 32. Franklin BD, Vincent C, Schachter M et al. . The incidence of prescribing errors in hospital inpatients: an overview of research. Drug Safety. 2005;2810:891–900. [DOI] [PubMed] [Google Scholar]
- 33. Khajouei R, Jaspers MWM. The impact of cpoe medication systems’ design aspects on usability, workflow and medication orders. Methods Inform Med. 2010;491:3–19. [DOI] [PubMed] [Google Scholar]
- 34. Shojania KG, Duncan BW, McDonald K et al. . Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Agency for Healthcare Research and Quality: Rockville, MD; 2001. [PMC free article] [PubMed] [Google Scholar]
- 35. Dior I, Schmidt P, O’Connor L. A cognitive perspective on technology enhanced learning in medical training: Great opportunities, pitfalls and challenges. Med Teacher. 2011;334:291–96. [DOI] [PubMed] [Google Scholar]
- 36. Patel VL, Kannampallil TG, Shortliffe EH. Role of cognition in generating and mitigating clinical errors. BMJ Qual Safety. 2015;247:468–74. [DOI] [PubMed] [Google Scholar]