Abstract
Objective
We sought to examine the impact of limiting the number of open active charts on wrong patient order entry events among 13 emergency departments (EDs) in a large integrated health system.
Methods
A retrospective chart review of all orders placed between September 2017 and September 2019 was conducted. The rate of retract and reorder events was analyzed with no overlap in both the period pre‐ and post‐intervention period. Secondary analysis of error rate by clinician type, clinician patient load, and time of day was performed.
Results
The order retraction rate was not improved pre‐ and post‐intervention. Retraction rates varied by clinician type with residents retracting more often than physicians (odds ratio [OR] = 1.443 [1.349, 1.545]). Advanced practice providers also showed a slightly higher rate than physicians (OR = 1.114 [1.071, 1.160]). Pharmacists showed very low rates compared to physicians (OR = 0.191 [0.048, 0.764]). Time of day and staffing ratios appear to be a factor with wrong patient order entry rates slightly lower during the night (1900–0700) than the day (OR 0.958 [0.923, 0.995]), and increasing slightly with every additional patient per provider (OR 1.019 [1.005, 1.032]). The Academic Medical Center had more retractions that the other EDs. OR for the various ED types compared to the Academic Medical Center included Community (OR 0.908 [0.859, 0.959]), Teaching Hospitals (OR 0.850 [0.802, 0.900]), and Freestanding (OR 0.932 [0.864, 1.006]).
Conclusions
Limiting the number of open active charts from 4 to 2 did not significantly reduce the incidence of wrong patient order entry. Further investigation into other factors contributing to order entry errors is warranted.
Keywords: electronic health record, emergency department, information technology, patient safety, prescribing errors, quality improvement, workflow
1. INTRODUCTION
1.1. Background
The Health Information Technology for Economic and Clinical Health (HITECH) act of 2009 mandated the adoption and meaningful use of electronic health records in health care facilities across the United States. 1 By 2016, 99% of eligible hospitals had demonstrated meaningful use and either adopted or upgraded electronic health records. 2 Computerized order entry is a core‐meaningful use objective designed to support standardized, legible, and complete orders to facilitate accurate and efficient processes of care. 3 Several studies have reported the positive impact of computerized order entry on safety and cost. 4 , 5 , 6 Conversely, adverse events resulting from computerized order entry have also been noted in the literature. Unintended consequences of computerized order entry include increased workload burden, disruption of workflow, and patient order errors resulting from cognitive overload, alert fatigue, and incorrect patient selection. 7
1.2. Importance
The use of the electronic health records in the emergency department is associated with specific challenges and points of vulnerability not seen in other health care delivery environments. The time‐critical, multi‐patient workflow of the ED necessitates frequent clinician interaction with the electronic health records. A study by Hill et al 8 found that emergency physicians spend significantly more time entering data into electronic health records than any other activity, amounting to 30%– 40% of a workday. Studies aimed at quantifying order entry errors in the ED reflect variation in error rates among ED practice setting and computerized order entry task. Ratwani et al 9 found that computerized order entry error rates varied among electronic health records functions and noted that error rates as high as 50% occurred during more complex ordering tasks. Error type varied from omission of order components, wrong time and wrong dose, frequency, route, or rate. 9 Pham et al 10 found that 12% of medication errors in the ED were due to patient selection errors. These errors may be exacerbated by the frequent interruptions that emergency physicians encounter on a given shift. 11
Patient selection errors are actions that are performed for 1 patient that were intended for another patient. 12 These error types are typically underreported because they require self‐reporting, are often recognized and corrected by the ordering clinician, are intercepted during the orders verification process, or they may go unrecognized. 13 , 14 In the early 1990s, Classen et al 15 described use of an integrated database that detected potential adverse medication events based on sudden stop orders, orders for antidotes, and abnormal lab values. This initial effort at isolating wrong patient orders led to the development of systems designed to identify rapid discontinuation of an order as a proxy for a prescribing error. 13 This phenomenon, now termed retraction‐reorder, or retract‐and‐reorder, describes patient selection errors as near‐miss events in which orders are placed on a patient, then retracted and reordered for a different patient. 14
The Bottom Line
Limiting the number of open active charts in the electronic medical record from 4 to 2 did not reduce the incidence of wrong patient order entry in the emergency department.
A number of electronic medical record adaptations designed to improve accuracy of patient selection have been proposed or trialed 16 (Table 1). Effectiveness of these interventions varies because introducing additional steps into the ordering process increases the amount of time clinicians spend in the electronic medical record.
TABLE 1.
Strategies to improve accuracy of patient selection
| Watermark with the patient's ED bed space 17 |
| Patient pictures 16 , 18 , 19 |
| Personalized patient lists 16 |
| Same/similar name alerts 16 |
| Elimination of cropped or truncated name displays 16 |
| Consistency among computer screens 16 |
| Pop‐up boxes for confirmation of patient identity 16 |
| Changing the location of the “submit” button 16 |
| Limiting the number of open patient charts 14 |
1.3. Goals of this investigation
In an effort to reduce wrong patient order entry events at our health system, the maximum number of active simultaneously open charts was decreased from 4 to 2. Emergency physicians care for multiple patients at a time and often need to switch tasks frequently. We therefore sought to examine the impact of limiting the number of open active electronic medical record charts on retract and reorder errors in the EDs of a large integrated health system. Secondarily, we sought to examine wrong patient order entry error rates by clinician type, patient load, ED type, and time of day to understand the epidemiology of wrong patient orders in the ED.
2. METHODS
2.1. Study design and setting
A retrospective chart review was performed on all patient encounters presenting to 1 of 12 EDs in a large integrated health system that uses a single instance of an electronic medical record (Epic Systems, Verona, WI) for 12 months before and 12 months after the intervention of limiting simultaneously open charts was enacted. The month in which the intervention was introduced was excluded from analysis. The health system Institutional Review Board approved the study.
2.2. Selection of participants
Any patient who received a patient care order during an active ED visit at any of the health system's 12 hospital‐based or freestanding EDs during the study period was considered for inclusion in the study. All orders generated from September 2017 through September 2019, excluding September 2018, were eligible for analysis. Eligible ordering clinician types included: physicians, advanced practice providers (physician assistants and nurse practitioners), fellows, residents, pharmacists, registered nurses, students, health unit coordinators. Nurses, students, and pharmacists required the identification of a co‐signer on their orders.
2.3. Measurements
Order types were categorized as medication or procedure orders. A procedure order was defined as any patient care order not directly related to a medication, such as imaging orders, laboratory tests, or patient care instructions. Retract and reorder was defined as any procedure or medication orders placed, retracted, and re‐entered on a different patient within 10 minutes. Secondary analysis of error rate by clinician type, patient load, and time of day was performed.
We characterized the error rate by instance, such that multiple simultaneously entered wrong patient orders were considered as 1 episode of error. Simultaneously entered orders are defined as those orders, which may be part of an order group, a standard multi‐order set or treatment care path, which are placed or signed at once during a single order entry session. Medication orders were listed separately from other orders, even if ordered simultaneously. A combined rate without overlap was calculated.
2.4. Analysis
Patient and clinician identifiers were removed, resulting in a de‐identified data set of time‐stamped orders, identifying the location, hour, and role for every distinct patient‐clinician pair. This data set was matched with an additional data set extracted from the electronic medical record of number of patients in the ED for each hour as well as the number of clinicians (attending physicians, residents, and advanced practice providers) for each hour. The number of new patients in each hour was added to the number of patients who arrived in the previous hour, and divided by the number of clinicians to obtain clinician staffing ratios. We used the first 2 hours to calculate clinician workload as we assumed orders were more likely to be placed during the first 2 hours of the patients’ arrival and workup.
We analyzed the error rate per 1000 orders using a logistic regression incorporating predictor variables to represent providers, time of day, patient volume and patient clinician ratio, and relationship to the intervention. To evaluate whether decreasing trends in error rates changed after the intervention, a linear term for month within time period, time period, and a monthly time period interaction were included in the models, and the interaction term was used to assess whether the rate of decline changed with the intervention.
Data were compiled using a SQL Server database (Microsoft SQL Server 2012) and then extracted into SAS software (Cary, NC) for analysis. All data are expressed in error rate per 1000 orders. A logistic regression model was created for all orders, using a backward stepwise selection method, excluding 1 variable at a time, by highest P‐value, until all variables had P‐values <0.05. The initial set of variables included several possible measures of time of day and several possible measures of patient load, each of which had moderate‐to‐high correlations (>0.30) to other possible measures of the same type. As these variables were removed from the model, a second test was done with the previously removed related variables to confirm that they were still not significant. None of this ultimately changed the course of the stepwise selection.
3. RESULTS
3.1. Study subjects
A total of 21,783,373 individual orders in 5,988,914 instances (matched patient, clinician, and order time) for 1,108,757 unique patient encounters were included in the analysis.
3.2. Primary results
The combined order retraction rate by instance was 2.4 per thousand before the intervention and 2.2 per thousand afterward. Overall, order retraction rates decreased by 0.03 retractions per thousand with each additional month in both time periods (Figure 1). The linear trend of reductions in overall order retraction rates in the year before and year after this intervention is consistent through both periods. This relative change in retraction over time did not significantly differ between time periods (P = 0.52). After adjustment for other factors, the odds of retraction decreased by 1.3% per month before the intervention, and by 1.7% per month after the intervention (Figure 2).
FIGURE 1.
Order Retraction Rate (per thousand), by Month
FIGURE 2.
Odds ratios for all variables
3.3. Secondary results
3.3.1. Clinician types
Across the intervention period, error rates varied by clinician type and ED type. In the multivariate model, clinician types were a significant predictor of error rates. Residents were reliably higher than attending physicians with an OR of 1.443 (1.349, 1.545). Advanced practice providers also showed a slightly higher rate than the physicians with an OR of 1.114 (1.071, 1.160). Pharmacists showed very low rates for errors compared to physicians with an OR of 0.191 (0.048, 0.764). Unit coordinators (clerks) had very high rates, with an OR of 1.613 (1.292, 2.014) compared to physicians for the few types of procedure orders they can enter (Figure 2).
3.3.2. ED types
Among the EDs included in the study, small Community Hospitals (n = 6), medium‐sized Teaching Hospitals (n = 2), and Freestanding EDs (n = 3) were each slightly less likely to have retracted orders than the large Academic Medical Center (n = 1) in the set, but were not significantly different from each other: Community OR 0.908 (0.859, 0.959), Teaching Hospitals OR 0.850 (0.802, 0.900), Freestanding OR 0.932 (0.864, 1.006) (Figure 2).
3.3.3. Time of day
Time of day and staffing ratios appear to be a factor with wrong patient order entry rates slightly lower during the night (1900–0700) than the day with an OR of 0.958 (0.923, 0.995) (Figure 3), and increasing slightly with every additional patient per provider with an OR of 1.019 (1.005, 1.032) (Figure 4).
FIGURE 3.
Order retraction by hour
FIGURE 4.
Order rate (per 1000) by patient–provider ratio
4. DISCUSSION
In our primary analysis, there was no difference in the rates of retract and reorder errors before and after limiting the number of concurrently open charts. Further, our results are concordant both in direction and size, with previously published results of other studies in an ED setting. For instance, a retrospective study by Kannampallil et al, 20 found no difference in error rates when the number of open charts was restricted to 2 from 4. Likewise, Adelman et al 21 also found no difference in error rates in their randomized study throughout a health system. In this study, the restricted group could only have 1 chart open at a time, and the unrestricted group could have up to 4 charts open, although ultimately 66% of the unrestricted group had only 1 chart open when they placed orders.
These results suggest that the uniqueness of the high‐stress ED environment and the number of charts open have no impact on the genesis of such errors. We speculate therefore that as yet undefined human factors and human‐computer interface issues may be more relevant in the etiology of these errors. Nonetheless, it is important to determine if the impact of numerical limitation is different in other high stress environments such as the intensive care unit and whether workload has an impact. This may help elucidate whether differences in practitioner types or workflows in these different settings may influence these errors, as suggested by the results of our secondary analyses.
In the secondary analyses, we find evidence of substantial variation in the ordering accuracy between different groups of clinicians in the ED setting. Health unit coordinators demonstrated the highest overall error rate for procedure orders, particularly orders for patient transport or supplies. Although the explanation for this is unclear, it may be related to the fact that they are frequently receiving verbal orders and may lack training in patient safety and identification. It is also possible that these orders have little clinical or significant impact, and thus the diligence present on orders with patient safety implications may be lacking.
Among clinician groups, residents demonstrated the highest rates of wrong patient order entry. It is not unreasonable to assume that the higher rates in the academic center ED may be in part attributable to the presence of residents. A single site pilot study by Murray et al 22 noted higher error rates among emergency medicine residents than emergency medicine attending physicians (9.2% versus 6.9%). Residents’ higher error rates may be related to the fact that as trainees, they may still be overly task‐focused, trying to get the work done, but may not yet have developed the skills to self‐audit in real‐time to catch errors. However, we do not have a breakdown of error rates by level of residency training that may support this hypothesis.
Increased patient‐to‐provider ratios were linked to increased rates of wrong patient order entry errors, and this is may be due to the workload burden and associated distractions when providing care for a high patient load. Diurnal variation was noted in errors for procedure orders but not for medication orders. Rates were lowest and most stable during the nighttime hours (1900–‐0700), and higher and more variable during the day. Again, the reasons for this difference are unclear given that both types of orders should be subject to any influence of lower patient volumes at night. Moreover, insofar as an unknown number of these errors will not be caught before a patient receives an unintended therapy, it is particularly concerning with medication order errors as these are presumably more likely to cause harm, should they reach the patient. 23
Limiting the number of simultaneously open patient charts, attempts to balance the need for smooth and efficient provider workflows (including multi‐tasking), with the imperative to minimize opportunities for patient harm. Indeed, the Office of the National Coordinator for Health Information Technology and the Joint Commission have issued even stricter recommendations to restrict the allowable number of open charts to just one. 21 Yet, our findings add to the body of evidence that run counter to expectations given the implicit underlying assumption that limiting the number of open charts reduces the opportunity for mix‐ups in patient selection.
Several proposals have been designed to curb these errors. Some of these have been implemented with yielding varying degrees of effectiveness. 16 Strategies include passive measures such as displaying patient pictures on the screen and ensuring patient names are fully displayed and not truncated by space constraints. Others include intrusive measures such as hard stops that require active provider action such as patient identity verification (by inputting patient initials or date of birth) before orders can be signed and submitted. 24 Even such measures have not been uniformly effective and do add a level of friction and inefficiency to the user experience.
Accordingly, we call for more research into human factors design in the clinical electronic medical record space to mitigate these risks. We do note that our electronic medical record system features automated real‐time checks for allergies and drug interactions errors in medication orders, with alerts presented to the provider before signing. It would be helpful if such functionality can be extended to include checks that notify prescribers when contextually inappropriate medications are being ordered before said orders are signed. This would be one immediately useful clinical application of the burgeoning field of artificial intelligence (AI) in medicine. Before this promise of AI can be realized, changes to the electronic medical record user interface and display as outlined in Table 1 can be harnessed to mitigate these errors.
5. LIMITATIONS
This study has several limitations. It measures retraction and reorder rates, which is an objective measure of wrong patient order entry, but it is unclear how this relates to true errors that reach the patient and any associated patient harm. We are unable to assess whether the ordering clinician themselves discovered the error and self‐corrected or whether they were notified of this by another professional providing checks and balances such as a pharmacist verifying medication orders or a nurse acknowledging laboratory studies. Presumably, these orders were retracted before they were carried out, but we are unable to determine the actual rate of errors that reached the patient.
We were also unable to retrospectively obtain true patient workload at the individual clinician level from the electronic medical record or whether patients were in the lobby, in ED treatment rooms or already admitted, all of which represent decreased labor intensity. Nevertheless, we believe that the proxy of new patient arrivals gives a snapshot into the workload in the ED and a point of reference for other times of the day that can be compared across EDs.
This study was strictly observational and designed to study the impact of our intervention, yet there may have been other confounding factors that affected the error rates. Finally, our results may not be generalizable to other electronic medical record systems.
6. CONCLUSION
Limiting the number of active, open charts in the electronic medical record from 4 to 2 did not significantly reduce the rate of retract and reorder in the ED setting. Our study also highlights the need for pilot trials before implementing widespread workflow changes, no matter how intuitive they may seem. Further investigation into wrong patient order events is warranted to better direct preventive solutions.
CONFLICTS OF INTEREST
The authors declare no conflicts of interest to disclose.
AUTHOR CONTRIBUTIONS
CC and BSF conceived and designed the study. CC provided the first draft with BSF. HB acquired and analyzed the data. HB, BF and CC interpreted the data. All authors were involved in revising for critically important intellectual content. BT takes final responsibility of the paper as a whole.
ACKNOWLEDGMENTS
The authors thank James Bena, MS, of the Department of Quantitative Health Sciences of the Cleveland Clinic for his assistance with statistical analysis.
Biography
Baruch S. Fertel, MD, is the Director of Operations and Quality Improvement Officer for the Emergency Services Institute and all of the Cleveland Clinic Emergency Departments.
Canfield C, Udeh C, Blonsky H, Hamilton AC, Fertel BS. Limiting the number of open charts does not impact wrong patient order entry in the emergency department. JACEP Open. 2020;1:1071–1077. 10.1002/emp2.12129
Funding and support: By JACEP Open policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that no such relationships exist.
Supervising Editor: Chadd K. Kraus, DO, DrPH.
REFERENCES
- 1. Halamka JD, Tripathi M. The HITECH era in retrospect. N Engl J Med. 2017; 377(10):907‐909. [DOI] [PubMed] [Google Scholar]
- 2. Office of the National Coordinator for Health Information Technology . What is computerized provider order entry? https://www.healthit.gov/faq/what-computerized-provider-order-entry. Accessed October 9, 2019.
- 3. Meaningful use and the shift to the merit‐based incentive payment system. HealthIT.gov. https://www.healthit.gov/topic/meaningful-use-and-macra/meaningful-use. Published October 18, 2019.
- 4. Forrester SH, Hepp Z, Roth JA, et al. Cost‐effectiveness of a computerized provider order entry system in improving medication safety ambulatory care. Value Health. 2014;17(4):340‐349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Vermeulen KM, van Doormaal JE, Zaal RJ, et al. Cost‐effectiveness of an electronic medication ordering system (CPOE/CDSS) in hospitalized patients. Int J Med Inf. 2014;83(8):572‐580. [DOI] [PubMed] [Google Scholar]
- 6. van Rosse F, Maat B, Rademaker CMA, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics. 2009;123 (4):1184‐1190. [DOI] [PubMed] [Google Scholar]
- 7. Brown CL, Mulcaster HL, Triffitt KL, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc. 2017;24(2):432‐440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Hill RG, Sears LM, Melanson SW. 4000 Clicks: a productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med. 2013;31(11):1591‐1594. [DOI] [PubMed] [Google Scholar]
- 9. Ratwani RM, Savage E, Will A, et al. A usability and safety analysis of electronic health records: a multi‐center study. J Am Med Inform Assoc. 2018;25(9):1197‐1201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pham JC, Story JL, Hicks RW, et al. National study on the frequency, types, causes, and consequences of voluntarily reported emergency department medication errors. J Emerg Med. 2011;40(5):485‐492. [DOI] [PubMed] [Google Scholar]
- 11. Chisholm CD, Collison EK, Nelson DR, et al. Emergency department workplace interruptions: are emergency physicians “interrupt‐driven” and “multitasking?” Acad Emerg Med. 2000;7(11):1239‐1243. [DOI] [PubMed] [Google Scholar]
- 12. Lowry SZ, Quinn MT, Ramaiah M, et al. Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records: Empirically Based Use Cases for Validating Safety‐Enhanced Usability and Guidelines for Standardization. Washington, DC: U.S. Department of Commerce; 2015. [Google Scholar]
- 13. Koppel R, Leonard CE, Localio AR, et al. Identifying and quantifying medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized physician order entry system. J Am Med Inform Assoc. 2008;15(4):461‐465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Adelman JS, Kalkut GE, Schechter CB, et al. Understanding and preventing wrong‐patient electronic orders: a randomized controlled trial. J Am Med Inform Assoc. 2013;20(2):305‐310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Classen DC, Pestotnik SL, Evans RS, et al. Description of a computerized adverse drug event monitor using a hospital information system. Hosp Pharm. 1992;27(9):774, 776–779, 783. [PubMed] [Google Scholar]
- 16. Sopan A, Plaisant C, Powsner S, et al. Reducing wrong patient selection errors: exploring the design space of user interface techniques. AMIA Annu Symp Proc. 2014;1056‐1065. [PMC free article] [PubMed] [Google Scholar]
- 17. Yamamoto LG. Reducing emergency department charting and ordering errors with a room number watermark on the electronic medical record display. Hawaii J Med Public Health. 2014;73(10):322‐328. [PMC free article] [PubMed] [Google Scholar]
- 18. Hyman D, Laire M, Redmond D, et al. The use of patient pictures and verification screens to reduce computerized provider order entry errors. Pediatrics. 2012;130(1):e211‐e219. [DOI] [PubMed] [Google Scholar]
- 19. Wilcox AB, Chen Y‐H, Hripcsak G. Minimizing electronic health record patient‐note mismatches. J Am Med Inform Assoc. 2011;18(4):511‐514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Kannampallil TG, Manning JD, Chestek DW, et al. Effect of number of open charts on intercepted wrong‐patient medication orders in an emergency department. J Am Med Inform Assoc. 2018;25(6):739‐743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Adelman JS, Applebaum JR, Schechter CB, et al. Effect of restriction of the number of concurrently open records in an electronic health record on wrong‐patient order errors: a randomized clinical trial. J Am Med Assoc. 2019;321(18):1780. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Murray KA, Belanger A, Devine LT, et al. Emergency department discharge prescription errors in an academic medical center. Bayl Univ Med Cent Proc. 2017;30(2):143‐146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Grissinger M. Oops, sorry, wrong patient! Pharm Ther. 2014;39(8):535‐537. [PMC free article] [PubMed] [Google Scholar]
- 24. Green RA, Hripcsak G, Salmasian H, et al. Intercepting wrong‐patient orders in a computerized provider order entry system. Ann Emerg Med. 2015;65(6):679‐686.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]